Surveys the live ServUO tree against everything the bridge already surfaces and records the full gap list (17 items), then specs the four features scoped for 3.0. 3.0 has three scope areas: - A: the visibility framework. Admin-configurable, per-feature and per-field audience control over all ten shard-derived surfaces (the four new ones plus the six that already ship), on an anonymous -> logged_in -> player -> staff -> admin ladder. Every default reproduces today's behavior, so the retrofit is a no-op until an admin changes something. Two rules an admin cannot override: acct and webId are admin-only always, and an unmapped event kind is never broadcast below admin. This also fixes a verified leak - guild leader acct/webId are readable today on the anonymous /public/shard/guilds. - B: three new wire streams - world.ruleset, points.board, and vendor.listing/vendor.listing.remove. - C: the spawn atlas, built from static ServUO data files with no wire involvement. Visibility lives entirely on the website; the sidecar stays a dumb forwarder that defines no access parameters and advertises no capabilities. PROTOCOL_VERSION goes 2 -> 3 once, at the end: every part PRs into an edge branch per repo, and the coordinated edge -> main merge is the cutover. A schema migration moves uo_link_config.protocol so operators don't have to. Co-Authored-By: Claude <noreply@anthropic.com>
34 KiB
Protocol 3.0 — Shard content, standings & the visibility framework
Status: Planned, approved 2026-07-28. Not yet built. All work lands on an edge branch in each repo; edge → main is the v3 cutover.
Date: 2026-07-28
Codebase: ServUO 57.4, <servuo>, net48 / x64, Expansion EJ.
Companion to PLAN.md (1.0 read/event plane), PROTOCOL_2.md (2.0 provisioning + world-state streams), ADMIN_CONTROLS.md (staff write plane), INTEGRATION.md (website API).
1. Why 3.0
A survey of the live ServUO tree against everything the bridge already surfaces end-to-end found that the bridge covers live activity well and covers shard content and standings almost not at all.
Covered by 1.0 + 2.0: presence/online, region transitions, char vitals + profile + roster, house registry + IDOC decay, champion spawns, guild board, city governors + term history, help-page queue, total gold supply, player-vendor sales log, deaths/murders/kills, skill gains, fame/karma, quest completes, staff/cheat audit, account linking + creation, town crier + news.
Not covered by anything: every leaderboard, every ruleset fact, every "where do I find X", and the entire player economy outside a player's own vendors.
3.0 has three scope areas:
- A — The visibility framework (§3). Admin-configurable, per-feature and per-field audience control over every shard-derived surface on the website. Ships first; the rest depends on it.
- B — Three new wire streams (§5, §7, §8).
world.ruleset,points.board,vendor.listing/vendor.listing.remove. - C — One website-only feature (§6). The spawn atlas, built from static ServUO data files with no wire involvement at all.
2. Survey: the full gap list
Recorded so the items not scoped for 3.0 aren't re-derived later.
| # | Gap | Source on the shard | Value | Cost | Status |
|---|---|---|---|---|---|
| 1 | Points/loyalty leaderboards — 25 point currencies | Scripts/Services/PointsSystems/PointsSystem.cs → static List<PointsSystem> Systems, each List<PointsEntry>{Player,Points} |
Very high | Low | 3.0 §7 |
| 2 | Shard ruleset page | Config/*.cfg via Server.Config.Get |
High | Very low | 3.0 §5 |
| 3 | Shard-wide marketplace | PlayerVendor.PlayerVendors + VendorSearch.cs |
Very high | High | 3.0 §8 |
| 4 | Spawn atlas / bestiary | Spawns/*.xml (6,455 spawners), RevampedSpawns/*.xml (333), Data/Regions.xml, Data/Locations/*.xml, Config/ChampionSpawns.xml, Data/teleporters.csv, Data/HarvestLocs/* |
High | Medium | 3.0 §6 |
| 5 | VvV standings + battle status | Services/ViceVsVirtue/{ViceVsVirtueSystem,GuildStats,VvVBattle}.cs |
High | Medium | PROTOCOL_2.md §10.4 deferred this pending "which PvP system does this shard run?" — now answered: VvV.cfg Enabled=True, Factions.cfg off. Unblocked, not scoped here |
| 6 | Skill leaderboards + shard census | Services/Reports/Reports.cs → GetSkillDistribution(), CompileGeneralStats(), StaffHistory |
High | Low | Spec'd PROTOCOL_2.md §14 (Part B phase 6), unbuilt. Shares §7's UI — fold in after |
| 7 | Custom mounts/pets codex — ~35 across 4 tiers | Scripts/Custom/{Companions,Legendary,Named,New Legacy} |
Medium-high | Very low | Pure wiki/CMS content, zero bridge work. The shard's most distinctive content, with zero site presence |
| 8 | Community Collections progress | Services/CommunityCollections/CollectionsSystem.cs |
Medium | Low | Natural public "community goal" widget |
| 9 | Seasonal/holiday event calendar | Services/Seasonal Events/SeasonalEventSystem.cs, Krampus, Forsaken Foes |
Medium | Low | "What's live now / what's next" |
| 10 | Crafting / taming / harvesting feeds | EventSink.CraftSuccess / TameCreature / ResourceHarvestSuccess |
Medium | Low | Spec'd PROTOCOL_2.md §11 #3/#4/#5, unbuilt |
| 11 | Virtue progression | EventSink.VirtueLevelChange, Services/Ethics/ |
Medium | Low | Spec'd §11 #6, unbuilt |
| 12 | Bulk Order Deeds + reward tables | Services/BulkOrders/, Data/Bulk Orders/* |
Medium | Low | Feed spec'd §11 #7; the static reward tables are a free wiki page |
| 13 | Guild wars | war state on Guild |
Low-medium | Low | Spec'd §11 #8, unbuilt |
| 14 | Astronomy discovery log | Services/Astronomy/AstronomySystem.cs (104 KB save) |
Low | Low | Niche completion leaderboard |
| 15 | In-game chat relay | Services/Chat/, Logs/Chat/{General,Help,Trade,LFG} |
Low | Medium | Privacy-sensitive; staff-only at most |
| 16 | Shard health telemetry | Crash logs, LayerConflict.log, throttle.log, world.save.after counts, AutoSave/AutoRestart schedule |
Low-medium | Low | world.save.after is already ingested but never charted — world-size-over-time is nearly free |
| 17 | Ultima Store / Sovereigns balance | Store.cfg Enabled=True, CurrencyName=Sovereigns; UltimaStore.GetCurrency |
? | Medium | Only worth it if sovereigns are actually sold |
Excluded permanently — see ADMIN_CONTROLS.md Tier H/N: firewall/IP-block,
kill/resurrect, jail, item/gold grants, set-access-level, arbitrary [set/[add.
Noticed during the survey, out of scope: Scripts/Custom/PerryOwnerFix.cs hardcodes an
EventSink.Login hook granting AccessLevel.Owner to account "ShardOnwerPerry". Worth reviewing
independently of this work.
3. Part A — The visibility framework
3.1 The leak this replaces (verified 2026-07-28)
BridgeJson.Actor() (BridgeJson.cs:85-117) writes serial, name, acct, webId,
player. shardState.model.js:346 shapeGuild() returns r.payload verbatim, and
GET /api/v1/public/shard/guilds (anonymous, shard.controller.js:131) serves it. A guild
leader's game account name and website user id are readable on an anonymous public endpoint today.
The same path exists for shapeGovernor → /public/shard/governors. Actor also feeds
guild.join, city.update and region.enter, all three in PUBLIC_KINDS on the anonymous SSE
stream.
The framework below is the vehicle for the fix, and the reason it ships before anything else.
3.2 Where visibility lives
On the website, never in the sidecar. The sidecar's job for 3.0 is unchanged in character: accept frames, persist them to its SQLite store, forward them verbatim over WS, and serve store-backed reads that survive a shard outage. It defines no access parameters, no audiences, no field projection, and advertises no capabilities.
3.3 The audience ladder
anonymous → logged_in → player → staff → admin, each rung implying the ones below it.
viewerLevel(req) resolves: no session ⇒ anonymous; authenticated ⇒ logged_in; authenticated
with a linked shard account ⇒ player; moderator/admin role ⇒ staff/admin. Staff always
satisfy the player rung even without a linked game account, consistent with the existing rule
that /player/* is role-agnostic self-service.
3.4 Two limits an admin cannot override
acctandwebIdare admin-only, always. They are not in-game-visible and are not exposed as configurable fields.- A kind absent from the kind→feature map is never broadcast below
admin. Fail closed. This preserves the property that today's staticPUBLIC_KINDSallowlist is a security boundary rather than a convenience filter.
3.5 Configuration
CREATE TABLE IF NOT EXISTS shard_feature_visibility (
feature VARCHAR(48) NOT NULL PRIMARY KEY,
enabled TINYINT(1) NOT NULL DEFAULT 1,
audience VARCHAR(20) NOT NULL DEFAULT 'anonymous',
field_rules JSON NULL, -- {"<field>": "<rung>"} for sensitive fields only
updated_by INT NULL,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
Seeded on boot in server.js, one row per feature. All ten shard features are covered — the four
new ones and the six that already ship — and every default reproduces today's behavior, so the
retrofit is a no-op until an admin changes something.
| Feature | Default audience | Sensitive fields (default rung) |
|---|---|---|
status, activity, champs, guilds, governors |
anonymous |
guilds/governors: leaderAcct / leaderWebId → admin (locked) |
houses |
anonymous |
owner → staff, price → staff (matches today's IDOC-only public view) |
presence |
anonymous |
location → staff (matches today's staff-only, location-gated /online) |
ruleset (new) |
anonymous |
connect → anonymous |
atlas (new) |
anonymous |
— |
leaderboards (new) |
anonymous |
characterName → anonymous |
market (new) |
anonymous |
ownerName → anonymous, location → anonymous |
3.6 Enforcement — three points, one config
New website/server/src/utils/shardVisibility.js:
LADDER = ['anonymous','logged_in','player','staff','admin'],rank(),meets(viewer, required)viewerLevel(req)(§3.3)KIND_FEATURE— every event kind → its feature; unmapped ⇒ admin-only (§3.4)getConfig()— DB-backed, cached ~5 s likeuoLinkClient's config cache, busted on adminPUTrequireFeature(name)— 404 when disabled (don't leak existence), 403 when enabled but the viewer is below the audienceprojectFeature(name, payload, viewerLevel)— strips fields whose rung the viewer doesn't meet;acct/webIdalways stripped belowadmin
Applied at:
- Routes —
requireFeature(…)on every/public/shard/*,/public/atlas/*and the shard-derived player routes;projectFeaturein the controllers, replacing the ad-hocshapeGuild-returns-payload-verbatim path. - SSE —
shardBroadcast.jsmoves from "one public channel with a staticPUBLIC_KINDSallowlist plus one admin channel" to per-connection filtering: each subscriber carries itsviewerLevel; each frame is mapped kind→feature, gated onenabled && meets(...), then passed throughprojectFeaturebefore write.PUBLIC_KINDSbecomes the seed data forKIND_FEATURErather than a hardcoded gate. This is the largest single change in Part A and where the security boundary now lives. - Nav —
GET /api/v1/public/shard/featuresreturns only the features the calling viewer can see, so the SPA hides nav entries rather than rendering links that 403.
3.7 Admin surface
GET / PUT /api/v1/admin/shard/visibility (admin-only). Validate feature names against the known
set and rungs against the ladder; reject any attempt to set acct/webId below admin. Writes an
admin.audit-style row so visibility changes are traceable. New client panel
routes/admin/ShardVisibility.jsx at /admin/shard-visibility, linked from ShardAdmin.jsx.
4. The version bump and the rollout
PROTOCOL_VERSION 2 → 3 in link/sidecar/src/main.rs:27. v3 is defined as "adds
world.ruleset, points.board, vendor.listing / vendor.listing.remove".
A bump is an operator-visible hard cutover — web.rs::gate returns 409 on every protected route on
mismatch, uoLinkSocket.js::handleHello closes the WS, and the website's declared version is the
admin-set uo_link_config.protocol column — so it happens exactly once, at the end:
- Cut an
edgebranch frommainin each ofwebsite/,link/,servuo-plugins/,docs/. - Every phase PRs into
edge, nevermain. Feature branches are cut fromedge. - Part A lands first, alone.
- When all parts are built and tested, one
edge→mainPR per repo, merged together. That merge is the v3 cutover. - A schema migration sets
uo_link_config.protocol(the existing row and the column default) from 2 to 3, so the cutover doesn't require a manual admin edit.UOLINK_PROTOCOLstill overrides. - No feature-negotiation array anywhere — v3 implies all three kinds.
5. Part B/1 — world.ruleset
PROTOCOL_2.md §10.4 sketches a world.systems capability frame that was never implemented
(grep returns nothing across all four repos). world.ruleset subsumes it, carrying a systems
sub-object with the cityLoyalty / vvv / factions booleans §10.4 asked for. §10.4 is marked
superseded; no orphan kind is left behind.
5.1 Plugin
NEW servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeRuleset.cs, modelled on
BridgeBoot.EmitHello — not a sweep. Subscribes BridgeLink.Connected_Core += Emit so a sidecar
that comes up second still learns the ruleset.
Built from an explicit allowlist of Server.Config.Get<T> calls. Never enumerate
Config.Entries (Server/Config.cs:162) — it would sweep in secrets. An FNV-1a rev over the body
makes an unchanged reconnect a site-side no-op (String.GetHashCode() is not stable across runs and
must not be used).
BridgeConfig.cs + overlay/Config/Bridge.cfg: RulesetEnabled=true, PublicConnectAddress="",
RulesetIncludeSchedule=true. BridgeBoot.cs: reload → re-emit, status → rev/bytes. Not wired
to sweepnow; it isn't a sweep.
5.2 Payload
Every block optional, omitted when its system is off:
shard, expansion, connect (only from PublicConnectAddress),
systems{cityLoyalty,vvv,factions,siege,chat,store,dailyRares,honesty,shadowguard,treasureMaps,vetRewards,testCenter},
caps{skill:1000,totalSkill:7000,stat:225,str/dex/int:125,strMax/dexMax/intMax:150},
housing{accountHouseLimit:1}, accounts{perIp:3,charSlots:7,autoCreate},
vetRewards{enabled,rewardIntervalDays:30},
loot{feluccaLuckBonus:1000,feluccaBudgetBonus:100,feluccaMaxProps:11},
vendors{restockDelayMinutes,maxSell,economyStockAmount},
champions{powerScrolls:6,statScrolls:16,scrollChance,transcendenceChance,rankThresholds},
treasureMaps, vvv{enabled,startSilver:2000,enhancedRules}, store{enabled,currencyName},
schedule{autoSaveFrequencyMinutes,autoRestart*}.
Excluded by name — in a code comment and here: Server.cfg (Address/Listen/Port; only
PublicConnectAddress is published), Staff.cfg, Email.cfg, DataPath.cfg, Bridge.cfg,
Compiler.cfg, Reports.cfg, Client.cfg.
5.3 Sidecar and website
Sidecar — store.rs: singleton ruleset(id CHECK(id=1), rev, json, updated_t) + upsert/get;
main.rs: new arm in the board-projection match; web.rs: GET /ruleset served from the store, so
it answers during a shard outage (PROTOCOL_2.md §12.2).
Website — uoLinkClient.getRuleset(); uoLinkSocket.backfill() (object-shaped, so follow the
getPresence() block's explicit form, not the array-only snapshot() helper); shardIngest.js →
shardState.setRuleset, not in LOGGED_KINDS (it re-arrives every reconnect and server.hello
already marks those); KIND_FEATURE['world.ruleset'] = 'ruleset'; shard_ruleset singleton table
(rev, expansion, payload JSON, t); GET /public/shard/ruleset behind
requireFeature('ruleset'), returning null ⇒ "not published yet".
Client — NEW routes/public/Rules.jsx at /site/rules, alongside
/site/champs|guilds|governors|houses; live via useShardFeed({ filter: new Set(['world.ruleset']) }).
5.4 Risk
Perf is nil (~3 KB per connect). The only real risk is publishing a secret, mitigated by the explicit
allowlist, the no-Config.Entries rule, the named exclusion list, and a manual eyeball of the emitted
frame during verification.
6. Part C — Spawn atlas / bestiary (website-only)
No plugin, no sidecar, no Bridge.cfg knob, no new kinds. Not part of the v3 wire change.
Decision: committed generated artifact + idempotent DB import, split in two because the build needs the ServUO tree (which the website container does not have) and the import does not. Not runtime import (10.5 MB of XML per boot), not a browser-served blob.
New in website/server/:
src/utils/spawnAtlasParse.js— pure functions, no fs, so they are unit-testable in CI without a ServUO tree:parseObjects2(),parsePoints(),parseRegions(),parseLocations(),resolveRegion().scripts/buildSpawnAtlas.js(--servuo <path> --out db/data/) andscripts/importSpawnAtlas.js(TRUNCATE + batched INSERT in one transaction);package.jsonscriptsatlas:build,atlas:import.db/data/spawnAtlas.<facet>.json×13 +spawnAtlas.index.json(creatures, champions, regions, landmarks, meta with per-source-file hashes).src/model/shardAtlas/{shardAtlas.db.js,shardAtlas.model.js}following theshardStatesplit.src/router/v1/public/atlas.{router,controller}.js;test/spawnAtlas.parse.test.js.
Two parsing notes that matter:
<Objects2>isType:MX=n:SB=…segments joined by:OBJ=— verified againsttrammel.xml, where a single point carries six types. Split on:OBJ=; the token before the first:is the type.- The high-value transform: point-in-rect each spawn against the facet's
Regions.xmlrects (highestprioritywins), falling back to the nearestData/Locationslandmark, else"Wilderness". This is what turns "lizardman at 5411,1234" into "Despise, Felucca" and is the entire reason the page is worth building.Regions.xmlis genuinely nested and needs a ~120-line recursive tokenizer or one devDependency (fast-xml-parser) — the server has zero XML deps today, so that is an explicit call to make at implementation time. The flat<Points>files need only regex/streaming; do not put 10.5 MB through a DOM parser.
Tables: shard_spawn_creatures (slug PK, name, total, facets JSON), shard_spawn_points (slug,
facet, x, y, region, landmark, max_count, tod_*), shard_regions, shard_landmarks,
shard_champion_spawns, shard_atlas_meta. Plain INDEX on name, not FULLTEXT — ~1,500
creature rows makes a LIKE scan free, and FULLTEXT brings min-token-length trouble for names like
"orc". No FKs, consistent with every existing shard_* table.
Routes at /api/v1/public/atlas, not under /shard — the atlas is static shard content, not
live shard state; it must not look sidecar-dependent, and unlike /shard/* it should be
siteMode-gated like /posts and /wiki. GET /creatures?q=&facet=, /creatures/:slug,
/regions, /landmarks, /champions, /meta, all behind requireFeature('atlas'). Admin:
GET /admin/shard/atlas/status (artifact-vs-DB drift) and POST /admin/shard/atlas/import. Build
stays CLI-only.
Client: routes/public/Atlas.jsx (/site/atlas) and AtlasCreature.jsx (/site/atlas/:slug).
Payload risk — a monolithic artifact would be 2–3 MB of committed JSON. Shard per facet and drop
every <Points> field the site cannot use (UniqueId, all trigger/refractory/proximity/sequential
fields, sound ids), keeping Name/Map/X/Y/W/H/Range/MaxCount/MinDelay/MaxDelay/TOD*/types — well under
1 MB. The artifact never reaches the browser; the browser sees only paginated API responses.
Operator re-run story — spawns changed → npm run atlas:build -- --servuo <path> on a machine
with the tree → commit the regenerated db/data/spawnAtlas.*.json → deploy → npm run atlas:import
(or POST /admin/shard/atlas/import). shard_atlas_meta.source holds per-file hashes, so
GET /admin/shard/atlas/status reports when the DB is behind the artifact. Full detail in
docs/website/SPAWN_ATLAS.md.
7. Part B/2 — points.board
Two deliverables: a diff sweep for the boards, and a points block folded into char.profile —
the PROTOCOL_2.md §10.3 titles precedent (read-model enrichment, no new request kind).
7.1 Plugin
NEW BridgePoints.cs, copying the BridgeHousing.cs diff-sweep shape (Initialize →
ServerStarted, Connected_Core += OnConnected clearing _last + Rearm(), SweepOnce(),
Status(), skip when !BridgeLink.Connected, try/catch throughout).
Which systems: default to PointsSystem.Systems filtered to ShowOnLoyaltyGump == true — reuse the
shard's own "this is player-facing" signal rather than inventing one. Bridge.cfg PointsSystems=
overrides. Null-guard PointsSystem.Systems; it is a mutable static populated by 25 separate
subsystem constructors.
The perf trap. PlayerTable is a plain List<PointsEntry>, and QueensLoyalty has AutoAdd, so
it can hold an entry for every PlayerMobile that ever existed. A naive
.OrderByDescending().Take(N) across 25 systems is 25 full sorts — at 20,000 historical characters,
~7.5 M comparisons, tens of ms on the Core thread. BRIDGE_PLUGIN_PLAN.md §1 found that nothing
except bulk profile generation comes close to a frame budget; this would be the second thing that
does.
Mitigation — single-pass bounded selection into a fixed N-element sorted array (N=10): O(n·N) with
tiny constants and one allocation. Skip Player == null || Deleted and Points <= 0. ~500 k cheap
iterations at a 300 s interval.
Diff signature per system: concat(serial + ":" + (long)points) over the top N, plus the entry count.
No points.remove — the system set is fixed, the same argument city.update already uses.
7.2 Payload — one frame per system
25 × ~600 B rather than one 12 KB frame, matching champ.update / guild.update:
{"t":…,"kind":"points.board","system":"QueensLoyalty",
"nameString":"Queen's Loyalty","nameNumber":1114938,
"maxPoints":30000,"showOnGump":true,"players":842,
"top":[{"rank":1,"serial":"0x1A2B","name":"Darrow","points":29500}, …]}
nameString and nameNumber are both emitted (a TextDefinition may be a cliloc), resolved
website-side — the contract titles.reward already documents at BridgeProfile.cs:107-110.
Entries are written inline as {serial, name} — never via BridgeJson.Actor. Deliberate even
though the website can now reveal fields by rung: acct/webId are not needed here, because the
website resolves serial→user from its own shard_account_links mirror for staff views. Keep the wire
minimal.
7.3 char.profile enrichment
BridgeProfile.cs gains WritePoints(sb, m) alongside WriteTitles:
"points":[{system,nameString,points,maxPoints}], omitting systems with no entry or 0 points.
Deliberately no rank — computing it means scanning each system's PlayerTable once per profile
(25 × n), which would dominate the measured 0.069 ms/profile budget. The website derives rank from
the board when the character appears in the top N. Gate behind PointsProfileRank=false if it is
ever wanted.
7.4 Config, sidecar, website
Bridge.cfg: PointsSweepSeconds=300, PointsLeaderboardEnabled=true, PointsTopN=10,
PointsSystems= (blank ⇒ auto), PointsProfileEnabled=true, PointsProfileRank=false.
BridgeBoot.cs: Rearm() in reload, SweepOnce() in sweepnow, Status() in both.
Sidecar — points_boards(system PK, name, json, updated_t); main.rs arm keyed on system;
GET /points and GET /points/:system.
Website — shard_points_boards(system PK, name, name_cliloc, max_points, players, show_on_gump, payload JSON, t). The top-N list stays in payload — a fixed-size list read whole, exactly like
shard_governors.candidates. Do not normalize into a shard_points_entries table until a
per-character reverse lookup is actually needed. shardIngest.js → upsertPointsBoard, not in
LOGGED_KINDS (board state, like guild.update). KIND_FEATURE['points.board'] = 'leaderboards',
with characterName as its per-field rule. GET /public/shard/points and /points/:system behind
requireFeature('leaderboards'); validate system ≤ 48 chars.
No new player route — per-character points ride inside char.profile, already served by
GET /player/shard/char/:serial with its shardLinks.ownsAccount check.
Client — NEW routes/public/Leaderboards.jsx at /site/leaderboards; a "Loyalty & Points" section
added to components/CharacterSheet.jsx, one edit serving both PlayerCharacter.jsx and
AdminCharacter.jsx.
8. Part B/3 — vendor.listing
8.1 It cannot be an RPC, and this is load-bearing
rpc.rs::try_route correlates on the first frame carrying a matching reqId and resolves a
single oneshot. A chunked reply sharing one reqId would deliver chunk 1 to the HTTP caller and
leak chunks 2..N onto the broadcast feed. REPLY_TIMEOUT is 10 s (the client waits 12 s), so a
whole-world snapshot could not fit regardless.
⇒ a per-vendor diff sweep on the broadcast stream, like champ.update / house.update. The
existing per-account vendor.snapshot RPC is untouched; the player portal keeps using it.
Kinds: vendor.listing (one frame per vendor, authoritative for that vendor) and
vendor.listing.remove. Payload: serial, shopName, owner:{serial,name}, map, x, y, region, house, count, truncated, items:[{serial,itemId,hue,amount,price,name,cliloc,child}].
8.2 Two perf traps
Measured baseline (BRIDGE_PLUGIN_PLAN.md §1): 30 vendors / 1,200 listings = 0.343 ms via
pack.Items + GetVendorItem; extrapolated to 500 vendors / 40,000 listings ≈ 12 ms per full pass.
Except:
VendorSearch.GetItemName(Item)is a packet builder, not a field read. It constructs anObjectPropertyList, callsGetProperties, serialises, then byte-parses the packet (VendorSearch.cs:681-789) — per item. Across 40,000 items in one tick that is a multi-hundred-millisecond stall. Mandatory: never call it in the sweep. EmititemId,hue,amount,price,item.Name(the plain field, null for most) anditem.LabelNumber, resolving display names website-side — exactly whatchar.profile.equipmentalready does (BridgeProfile.cs:173).VendorSearch.GetItems(PlayerVendor)is private (:791). The reusable public API isGetItems(Container, List<Item>)(:807), which recurses into sub-containers, so real item counts run above the top-levelpack.Itemsthe 0.343 ms measurement used. Budget accordingly.
8.3 Mitigations
- Amortized round-robin sweep —
MarketSweepSeconds=60, at mostMarketSweepBatch=25vendors per tick, with a persistent cursor overPlayerVendor.PlayerVendors. Full coverage inceil(vendors/25) × 60 s, with per-tick cost bounded independent of world size. This is the one genuinely new pattern versus the existing sweeps and should be flagged in review. - Per-vendor signature diff (
count | Σ(serial ^ price) | x | y | shopName), asBridgeHousingdoes — most vendors are static, so steady-state emission is near zero. MarketMaxListings=250, then"truncated":true.BridgeJson.Parsecaps inbound at 1 MB; outbound is uncapped andshard.rs::read_linewill allocate whatever arrives.- On
Connected_Core, clear_lastand reset the cursor; the re-emit is self-throttled by the round-robin window.
8.4 Player opt-out and privacy
Honour pv.VendorSearch — ServUO's own per-vendor opt-out, which DoSearch filters on (:62).
Skip opted-out vendors entirely; the seen-set removal then drops them from the board, so a player
who hid their vendor in game is hidden on the website too. Also skip Map == null || Map.Internal
and Backpack == null, matching DoSearch.
A vendor's shop name, owner character name and location are already globally visible in-game — the
stock Vendor Search gump surfaces exactly this set to any player — which is why they default to
anonymous. They remain per-field configurable (ownerName, location) so an admin can tighten
them. Account name and website user id never go on the wire.
8.5 Sidecar and website
Sidecar — one table vendors(serial PK, shop_name, owner_name, map, x, y, region, count, json, updated_t) storing the whole-vendor blob. No vendor_items table — the sidecar's job here is
outage resilience (PROTOCOL_2.md §12.2), not search; search lives in MariaDB. Endpoint is
GET /market, not /vendors — axum would route the latter fine, but the collision with the
per-account RPC is a readability trap.
Website — shard_vendors + shard_vendor_items (indexes on vendor_serial, price, item_id,
display_name; delete-then-insert per vendor in one transaction; no FKs). shardIngest.js handles
both kinds; not in LOGGED_KINDS.
KIND_FEATURE['vendor.listing'] = 'market', but the market feature's SSE mapping is disabled by
default: a live firehose of full vendor inventories would be the site's single biggest bandwidth
consumer, and no page needs it live. The page is a paginated DB query with a staleness stamp; an
admin can turn the stream on. uoLinkSocket paginates /market on reconnect, bounded by
MARKET_SNAPSHOT_MAX = 5000 vendors so a pathological world cannot hang startup.
GET /public/shard/market?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=
(limit 1..100, default 50; q ≤ 60 chars; sort ∈ {price_asc, price_desc, recent}) and
/market/vendors/:serial, behind requireFeature('market'). Rate-limit it — this is the first
genuinely expensive public endpoint; express-rate-limit is already a dependency.
8.6 The open dependency — cliloc names
CharacterSheet.jsx:14-15 already documents the gap ("without a cliloc table on the site we can only
show literals") and renders equipment as id {itemId}. Search-by-name needs that table.
- Recommended:
scripts/buildClilocs.jsreads the UO client'sCliloc.enu→ committeddb/data/clilocs.json; ingest denormalizes intoshard_vendor_items.display_name. Same build-artifact pattern as §6, and it also fixes the character sheet. - Fallback: ship with item-art + price + region filters, and name search only over renamed items.
This decision is the reason §8 is sequenced last.
8.7 Client
routes/public/Market.jsx at /site/market, with a "prices last refreshed N minutes ago" banner
driven by staleAt (the oldest shard_vendors.updated_at). The round-robin sweep means data is
inherently up to one full cycle old, and the UI must say so.
9. Sequencing
| Order | Part | Repos touched | Wire change |
|---|---|---|---|
| 1 | A — visibility framework + actor-leak fix | website, docs | none |
| 2 | B/1 — world.ruleset (§5) |
all four | new kind |
| 3 | C — spawn atlas (§6) | website, docs | none |
| 4 | B/2 — points.board (§7) |
all four | new kind + char.profile field |
| 5 | B/3 — vendor.listing (§8) |
all four | new kinds |
| 6 | Cutover — PROTOCOL_VERSION 2→3, edge → main |
all four | the bump |
10. Documentation obligations
- This file (
link/v3.md) is the canonical 3.0 design. PROTOCOL_2.md§10.4 gains a note thatworld.systemsis superseded byworld.ruleset, and that the deferred VvV question is answered (VvV.cfg Enabled=True, Factions off).INTEGRATION.md— catalog entries and §6 consumer sections for each new kind, plus the v2→v3 upgrade note for operators.PLAN.md— phasing.website/BACKEND_DESIGN.md— every new table and route, and the visibility framework as a security contract: the audience ladder, the two locked rules, and the fail-closed kind map belong in the security section.- NEW
website/SHARD_VISIBILITY.md— admin-facing: what each feature exposes, what each rung means, what cannot be loosened. - NEW
website/SPAWN_ATLAS.md, NEWwebsite/MARKETPLACE.md. PROJECT_TREE.mdin each touched repo.npm run swaggerandnpm run routes:manifeston every route-touching PR — both are committed artifacts, andtest/routeManifest.test.jsfails on drift.
Follow-up, not scoped for 3.0: the Android app consumes the same public/player shard API and will
need /public/shard/features to hide its own nav. Track separately against android-app/.
11. Verification
Plugin — servuo-plugins\deploy.ps1 -ServerPath <servuo> -Verify, inspect the ADD/CHANGE list,
then re-run without -Verify (ServUO must be stopped). Boot with tools/stub_sidecar.ps1 listening
and confirm the compile banner in the console, not merely the absence of errors —
BRIDGE_PLUGIN_PLAN.md §1 warns that a failing build is silently ignored and the previous
Scripts.dll reloads. Then [bridge status, [bridge sweepnow, [bridge reload.
- §5: eyeball the emitted
world.rulesetframe for anything sourced fromServer.cfg,Staff.cfg,Email.cfg,DataPath.cfgorBridge.cfg. - §8: with a seeded world, time one sweep tick and confirm the batch cap holds it under ~1 ms.
Sidecar — cargo build && cargo clippy; curl -H "Authorization: Bearer <token>" localhost:8080/ruleset (and /points, /market); confirm X-UOLink-Version: 3 and that a client
declaring 2 receives a 409.
Website server — DB_HOST=127.0.0.1 DB_PORT=59999 node --test. New tests, each modelled on an
existing sibling: test/shardVisibility.test.js, test/shardBroadcast.visibility.test.js,
test/shardIngest.{ruleset,points,market}.test.js (after shardIngest.protocol2.test.js — stubbed
deps, asserting routing and logged flags), test/spawnAtlas.parse.test.js (pure functions, inline
fixtures). Then npm run routes:manifest and npm run swagger, committing both.
Full stack — against a local MariaDB: apply db/schema.sql (idempotent), start the server,
confirm uoLinkSocket backfill logs the new snapshot lines and that uo_link_config.protocol
migrated to 3, then load /site/rules, /site/atlas, /site/leaderboards, /site/market.
Visibility smoke test — for each of the five rungs, walk every shard page and confirm gating and field projection match the configured matrix. Same shape as the 200-routes × 5-access-levels sweep already run for the domain split.
12. Critical files
| File | Why |
|---|---|
website/server/src/utils/shardBroadcast.js |
The security boundary; reworked from a static allowlist to per-connection audience filtering. The highest-risk file in 3.0. |
website/server/src/utils/shardVisibility.js (new) |
Ladder, kind→feature map, projection |
website/server/src/utils/shardIngest.js |
The dispatcher every new kind routes through |
website/server/src/model/shardState/shardState.model.js |
The shape* projections, including the shapeGuild leak §3.1 fixes |
servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeHousing.cs |
Cleanest copy of the diff-sweep pattern; template for BridgePoints.cs and BridgeMarket.cs |
link/sidecar/src/main.rs |
PROTOCOL_VERSION 2→3 and the board-projection match |
website/server/db/schema.sql |
All new shard_* tables plus the uo_link_config.protocol migration |