feat(bridge): publish the player-vendor market index as vendor.listing #5

wtclaude · 2026-07-29T14:54:10Z

wtclaude commented

2026-07-29 14:54:10 +00:00

What & why

Protocol 3.0 §8 (docs/link/v3.md) — order 5b, the shard side. Docs in docs #71; sidecar in link #19; website in website #116.

Every player vendor's shop name, owner, location and priced inventory, published as one authoritative vendor.listing frame per vendor, so the website can offer the search the in-game Vendor Search gump offers — from outside the game, and honouring the same per-player opt-out.

⚠️ It cannot be an RPC, and that shapes everything

The obvious design is a market.snapshot request/reply like vendor.snapshot next door. It cannot work: rpc.rs::try_route correlates a reply on the first frame carrying a matching reqId and resolves a single oneshot, so a chunked reply sharing one reqId would deliver chunk 1 to the HTTP caller and leak chunks 2..N onto the broadcast feed. A whole-world snapshot would not fit in one frame inside the 10 s reply timeout either.

So it is a diff sweep on the broadcast stream. The per-account vendor.snapshot RPC is untouched; the player portal keeps using it.

The one genuinely new pattern: an amortized round-robin

Worth a reviewer's attention because it is unlike every other sweep here. BridgeHousing, BridgePoints, BridgeSocial and the rest walk their whole collection each tick — fine for tens of houses or a fixed set of ~25 point systems, and not fine for a world of shops whose inventories recurse into containers.

BridgeMarket inventories at most MarketSweepBatch (25) vendors per tick from a persistent cursor, so per-tick cost is bounded by the batch rather than by world size. Full coverage takes ceil(vendors / batch) × MarketSweepSeconds.

The subtlety the batching creates is the removal pass: _last holds every vendor seen in any previous tick, so "not in this tick's window" does not mean gone. Removals are decided against the current vendor list, which is a cheap pass over serials rather than a second inventory walk.

`VendorSearch.GetItemName` is never called

It builds an ObjectPropertyList, calls GetProperties, serialises it and then byte-parses the resulting packet — per item. Across a full pass that is a multi-hundred-millisecond stall on the Core thread. The frame carries itemId, hue, amount, price, the plain item.Name field (null for most items) and item.LabelNumber; the website resolves names against its own cliloc table, exactly as char.profile.equipment already does.

It would not work anyway: every current client ships its cliloc files compressed, ServUO's bundled Ultima.StringList reads only the older plain layout, so VendorSearch.StringList is null and GetItemName returns item.Name regardless — the in-game gump has the same gap.

Design notes worth a look

location is ONE nested object, not flat map/x/y/region. The website's visibility projection matches literal JSON keys, so nesting is what lets a single market.location rule hide a vendor's whereabouts on both the live frame and the stored read model. Flat keys would need five rules that could drift apart — and Part A had already pre-wired a location rule that flat keys would have made inert, exactly like the characterName miss one part earlier.
Owner is flat ownerSerial/ownerName, never BridgeJson.Actor, which would add acct and webId. Same argument points.board makes: this is the widest-audience surface the bridge has.
pv.VendorSearch is honoured — ServUO's own per-vendor opt-out, which DoSearch filters on. A shop hidden in game is hidden on the site, and the seen-set removal then emits vendor.listing.remove, so revoking consent takes effect rather than merely stopping refreshes.
Container-priced items carry child: true, exactly as DoSearch reports them. ServUO prices a container as a unit; a site that printed the container's price against each item inside it would be lying about the shard.
The diff signature is the full listing set, not §8.3's proposed count | Σ(serial ^ price) — that hash collides on two items swapping prices, which is exactly what re-pricing a shop looks like.
The item walk uses the public VendorSearch.GetItems(Container, List<Item>) rather than a hand-rolled recursion, so ServUO's rule about which containers are sold whole (quivers, seed boxes, jewelry boxes, …) stays ServUO's to define — the predicate that decides it is private, and a copy here would silently diverge.

How it was tested

Deployed to the local ServUO tree and compiled offline first (dotnet build Scripts.csproj -c Release → 0 Error(s)), because a boot-time rebuild failure silently reloads the previous Scripts.dll. Boot confirmed the compile banner, not merely the absence of errors.

Run against the real shard (209,116 items / 43,011 mobiles / 27 player vendors) with the Rust sidecar listening:


Vendors / listings published	27 / 1,040
First tick — 25 vendors, cold	15.4 ms
Second tick — the remaining 2	3.4 ms
Steady state — nothing changed	0.3 ms

Every frame checked for shape: nested location with real facets, regions and house-sign names (26 of 27 in a house — the null path exercised), flat owner fields, count/total/truncated, and 1,040 items carrying cliloc with name: null, which is the normal case.

Sweep cost is now reported, because the batch cap is a claim about it

[bridge status gained lastMs/maxMs, and a tick over 50 ms prints a rate-limited warning naming the knob. An operator tuning MarketSweepBatch was otherwise tuning blind.

Note the arithmetic the warning exists for: 25 full shops at the 250-listing cap is 6,250 items ≈ 95 ms, over budget. Real shops hold tens, which is why 25 is the default — but a shard of commodity resellers should lower the batch, and now it will be told to.

Two small things the live run caught

BridgeJson.Escape takes a NON-NULL string (it dereferences value.Length) and BridgeJson.Str writes its own ,"key": prefix, so neither serves a value inside a hand-built object. Nearly everything this frame writes is legitimately null. BridgeMarket.Text() is the two-line writer that was missing.
The ServUO console writes in the OS code page, so an em dash in a Console.WriteLine renders as ??? in the log an operator would paste into an issue. Fixed to ASCII.

Not covered: vendor.listing.remove from the plugin side needs a vendor dismissed or opted out in game, which needs a client. The removal path was verified at the sidecar and website layers instead (synthetic insert-then-remove over the loopback socket, and a routing test).

CI note: pr-checks.yml only triggers on PRs into main, so this PR (targeting edge) runs no CI — local verification above is the gate.

Checklist

I have read CONTRIBUTING.md.
The change builds and existing tests/checks pass locally.
I have added or updated tests/docs where it makes sense.
My commits are reasonably scoped with clear messages.

AI-assisted contributions (required)

No AI tools were used to produce this contribution.
AI tools were used. Tool(s): Claude Code (Opus 5). I have reviewed and understand every change, and take responsibility for it. AI-authored commits are marked with a Co-Authored-By trailer.

License

I agree that my contribution is licensed under this project's license (GNU GPL v3.0 or later), and I have the right to contribute it.

## What & why Protocol 3.0 **§8** ([`docs/link/v3.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/edge/link/v3.md)) — order 5b, the shard side. Docs in docs #71; sidecar in link #19; website in website #116. Every player vendor's shop name, owner, location and priced inventory, published as one authoritative `vendor.listing` frame per vendor, so the website can offer the search the in-game **Vendor Search** gump offers — from outside the game, and honouring the same per-player opt-out. ### ⚠️ It cannot be an RPC, and that shapes everything The obvious design is a `market.snapshot` request/reply like `vendor.snapshot` next door. It cannot work: `rpc.rs::try_route` correlates a reply on the **first** frame carrying a matching `reqId` and resolves a single oneshot, so a chunked reply sharing one `reqId` would deliver chunk 1 to the HTTP caller and **leak chunks 2..N onto the broadcast feed**. A whole-world snapshot would not fit in one frame inside the 10 s reply timeout either. So it is a diff sweep on the broadcast stream. The per-account `vendor.snapshot` RPC is untouched; the player portal keeps using it. ### The one genuinely new pattern: an amortized round-robin Worth a reviewer's attention because it is unlike every other sweep here. `BridgeHousing`, `BridgePoints`, `BridgeSocial` and the rest walk their **whole** collection each tick — fine for tens of houses or a fixed set of ~25 point systems, and not fine for a world of shops whose inventories recurse into containers. `BridgeMarket` inventories at most `MarketSweepBatch` (25) vendors per tick from a persistent cursor, so **per-tick cost is bounded by the batch rather than by world size**. Full coverage takes `ceil(vendors / batch) × MarketSweepSeconds`. The subtlety the batching creates is the **removal** pass: `_last` holds every vendor seen in any previous tick, so "not in this tick's window" does *not* mean gone. Removals are decided against the current vendor list, which is a cheap pass over serials rather than a second inventory walk. ### `VendorSearch.GetItemName` is never called It builds an `ObjectPropertyList`, calls `GetProperties`, serialises it and then **byte-parses the resulting packet — per item**. Across a full pass that is a multi-hundred-millisecond stall on the Core thread. The frame carries `itemId`, `hue`, `amount`, `price`, the plain `item.Name` field (null for most items) and `item.LabelNumber`; the website resolves names against its own cliloc table, exactly as `char.profile.equipment` already does. It would not work anyway: every current client ships its cliloc files **compressed**, ServUO's bundled `Ultima.StringList` reads only the older plain layout, so `VendorSearch.StringList` is null and `GetItemName` returns `item.Name` regardless — **the in-game gump has the same gap.** ### Design notes worth a look - **`location` is ONE nested object, not flat `map`/`x`/`y`/`region`.** The website's visibility projection matches literal JSON keys, so nesting is what lets a single `market.location` rule hide a vendor's whereabouts on both the live frame and the stored read model. Flat keys would need five rules that could drift apart — and Part A had *already* pre-wired a `location` rule that flat keys would have made inert, exactly like the `characterName` miss one part earlier. - **Owner is flat `ownerSerial`/`ownerName`, never `BridgeJson.Actor`**, which would add `acct` and `webId`. Same argument `points.board` makes: this is the widest-audience surface the bridge has. - **`pv.VendorSearch` is honoured** — ServUO's own per-vendor opt-out, which `DoSearch` filters on. A shop hidden in game is hidden on the site, and the seen-set removal then emits `vendor.listing.remove`, so *revoking* consent takes effect rather than merely stopping refreshes. - **Container-priced items carry `child: true`**, exactly as `DoSearch` reports them. ServUO prices a container as a unit; a site that printed the container's price against each item inside it would be lying about the shard. - **The diff signature is the full listing set**, not §8.3's proposed `count | Σ(serial ^ price)` — that hash collides on two items swapping prices, which is exactly what re-pricing a shop looks like. - **The item walk uses the public `VendorSearch.GetItems(Container, List<Item>)`** rather than a hand-rolled recursion, so ServUO's rule about which containers are sold whole (quivers, seed boxes, jewelry boxes, …) stays ServUO's to define — the predicate that decides it is private, and a copy here would silently diverge. ## How it was tested Deployed to the local ServUO tree and **compiled offline first** (`dotnet build Scripts.csproj -c Release` → `0 Error(s)`), because a boot-time rebuild failure silently reloads the previous `Scripts.dll`. Boot confirmed the compile banner, not merely the absence of errors. Run against the **real shard** (209,116 items / 43,011 mobiles / 27 player vendors) with the Rust sidecar listening: | | | |---|---| | Vendors / listings published | **27 / 1,040** | | First tick — 25 vendors, cold | **15.4 ms** | | Second tick — the remaining 2 | **3.4 ms** | | Steady state — nothing changed | **0.3 ms** | Every frame checked for shape: nested `location` with real facets, regions and house-sign names (26 of 27 in a house — the null path exercised), flat owner fields, `count`/`total`/`truncated`, and 1,040 items carrying `cliloc` with `name: null`, which is the normal case. ### Sweep cost is now reported, because the batch cap is a *claim* about it `[bridge status` gained `lastMs`/`maxMs`, and a tick over 50 ms prints a rate-limited warning naming the knob. An operator tuning `MarketSweepBatch` was otherwise tuning blind. Note the arithmetic the warning exists for: 25 **full** shops at the 250-listing cap is 6,250 items ≈ 95 ms, over budget. Real shops hold tens, which is why 25 is the default — but a shard of commodity resellers should lower the batch, and now it will be told to. ### Two small things the live run caught - **`BridgeJson.Escape` takes a NON-NULL string** (it dereferences `value.Length`) and `BridgeJson.Str` writes its own `,"key":` prefix, so neither serves a value inside a hand-built object. Nearly everything this frame writes is legitimately null. `BridgeMarket.Text()` is the two-line writer that was missing. - **The ServUO console writes in the OS code page**, so an em dash in a `Console.WriteLine` renders as `???` in the log an operator would paste into an issue. Fixed to ASCII. **Not covered:** `vendor.listing.remove` from the *plugin* side needs a vendor dismissed or opted out in game, which needs a client. The removal path was verified at the sidecar and website layers instead (synthetic insert-then-remove over the loopback socket, and a routing test). **CI note:** `pr-checks.yml` only triggers on PRs into `main`, so this PR (targeting `edge`) runs no CI — local verification above is the gate. ## Checklist - [x] I have read [CONTRIBUTING.md](CONTRIBUTING.md). - [x] The change builds and existing tests/checks pass locally. - [x] I have added or updated tests/docs where it makes sense. - [x] My commits are reasonably scoped with clear messages. ## AI-assisted contributions (required) - [ ] No AI tools were used to produce this contribution. - [x] AI tools were used. Tool(s): `Claude Code (Opus 5)`. I have reviewed and understand every change, and take responsibility for it. AI-authored commits are marked with a `Co-Authored-By` trailer. ## License - [x] I agree that my contribution is licensed under this project's license (**GNU GPL v3.0 or later**), and I have the right to contribute it.

wtclaude added 1 commit 2026-07-29 14:54:12 +00:00

feat(bridge): publish the player-vendor market index as vendor.listing 48d57e6278

Protocol 3.0 §8. Every player vendor's shop name, owner, location and priced
inventory, so the website can offer the search the in-game Vendor Search gump
offers — from outside the game, and honouring the same per-player opt-out.

It cannot be an RPC. rpc.rs correlates a reply on the FIRST frame carrying a
matching reqId, so a chunked reply sharing one reqId would deliver chunk 1 to the
HTTP caller and leak chunks 2..N onto the broadcast feed; a whole-world snapshot
would not fit in one frame inside the 10 s timeout either. So it is a diff sweep
on the broadcast stream, one authoritative frame per vendor.

The one genuinely new pattern here is an amortized round-robin: every other sweep
walks its whole collection per tick, which is fine for tens of houses and is not
fine for a world of shops whose inventories recurse into containers.
MarketSweepBatch (25) vendors are inventoried per tick from a persistent cursor,
so per-tick cost is bounded by the batch rather than by world size.

VendorSearch.GetItemName is never called: it builds an ObjectPropertyList,
serialises it and byte-parses the packet per item. The frame carries itemId, hue,
amount, price, the plain item.Name field and item.LabelNumber; the website
resolves names against its own cliloc table. (It would not work anyway — every
current client ships its cliloc files compressed and ServUO's Ultima.StringList
cannot read them, so the in-game gump has the same gap.)

Measured on the live shard (27 vendors x 40 listings, 209k items / 43k mobiles):
15.4 ms for the first cold tick of 25 vendors, 3.4 ms for the next, 0.3 ms in
steady state. `[bridge status` now reports lastMs/maxMs and a tick over 50 ms
warns, naming the knob — the batch cap is a claim about that number and an
operator tuning it was otherwise tuning blind.

- location is ONE nested object, not flat map/x/y/region, so the website's single
  market.location visibility rule can hide a vendor's whereabouts on both the
  live frame and the stored read model. Flat keys would need five rules.
- Owner is flat ownerSerial/ownerName, never BridgeJson.Actor, which would add
  acct and webId. Same argument points.board makes.
- pv.VendorSearch is honoured, so a shop hidden in game is hidden on the site;
  the seen-set removal then emits vendor.listing.remove.
- Container-priced items carry child:true, exactly as DoSearch reports them.
- Over MarketMaxListings (250) the frame says truncated and carries the real
  total, so the site shows "250 of 3,104" rather than a partial shop as complete.

Co-Authored-By: Claude <noreply@anthropic.com>

whitlocktech merged commit 7215ae5fe1 into edge

2026-07-29 20:06:45 +00:00

whitlocktech deleted branch feat/vendor-listing

2026-07-29 20:06:46 +00:00

whitlocktech referenced this issue from a commit

2026-07-29 20:06:46 +00:00

Merge pull request 'feat(bridge): publish the player-vendor market index as vendor.listing' (#5) from feat/vendor-listing into edge

wtclaude referenced this pull request

2026-07-29 23:06:38 +00:00

feat(bridge)!: Protocol 3.0 cutover — world.ruleset, points.board, vendor.listing #6

Sign in to join this conversation.