wtclaude
6ce60a82c3
Documents order 5b across the four repos, and records what building it changed about §8 as designed. - NEW website/MARKETPLACE.md — the operator guide: what the pages must say out loud and why, the privacy contract (the player's in-game Vendor Search toggle wins, and no admin setting overrides it), the Bridge.cfg knobs and how they trade against each other, and the measured sweep costs. - INTEGRATION.md — catalog entry for vendor.listing / vendor.listing.remove with its six consumer gotchas, and the GET /market REST section (the sidecar's only paged read, and why it orders by serial rather than shop name). - BACKEND_DESIGN.md — shard_vendors / shard_vendor_items, the routes, and the marketplace search as the only rate-limited public read. - SHARD_VISIBILITY.md — why the market's fields default to Everyone (the in-game gump already shows exactly that set), why location is one setting covering four things, and why hiding the owner name without the owner id achieves nothing. - PLAN.md — the amortized round-robin as the one sweep pattern the bridge did not previously have, and an update to §7's cliloc note: pushing name resolution to the plugin was never an option, because ServUO cannot read a modern client's compressed cliloc files either. - v3.md §8.8 — the four things the build settled differently, chief among them that §8.1's FLAT location payload would have made Part A's pre-wired market.location rule inert, exactly like the characterName miss one part earlier. Co-Authored-By: Claude <noreply@anthropic.com>
168 lines
8.9 KiB
Markdown
168 lines
8.9 KiB
Markdown
# Marketplace — the player-vendor index
|
||
|
||
**Status:** On `edge` — servuo-plugins [#5](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/5), link [#19](https://gitea.whitlocktech.com/RunicGateway/link/pulls/19), website [#116](https://gitea.whitlocktech.com/RunicGateway/website/pulls/116).
|
||
**Design:** [`docs/link/v3.md` §8](../link/v3.md) — Protocol 3.0 Part B/3.
|
||
**Depends on:** [`CLILOCS.md`](CLILOCS.md) — without a cliloc table, listings render as item ids.
|
||
|
||
The marketplace is a searchable index of every player vendor on the shard: what
|
||
each shop is selling, for how much, and where it is standing. It is the same set
|
||
the in-game **Vendor Search** gump reads, offered from outside the game — so a
|
||
player can find the vanquishing kryss they want before logging in, and someone
|
||
who does not play at all can see that the economy exists.
|
||
|
||
Page: `/site/market`, plus `/site/market/vendors/:serial` for one shop.
|
||
|
||
## Three things the pages must say out loud
|
||
|
||
Everything below follows from how the data is gathered, and each has a visible
|
||
consequence the UI is required to surface.
|
||
|
||
**1. The prices are not live.** The shard sweeps vendors **round-robin** — at
|
||
most `Bridge.MarketSweepBatch` shops per tick — so a given shop can be a full
|
||
cycle behind. The page carries a *"prices last refreshed N minutes ago"* banner
|
||
driven by the **oldest** vendor row, not the newest: the one stale shop is the
|
||
one that wastes somebody's trip.
|
||
|
||
**2. A shop can be truncated.** `Bridge.MarketMaxListings` (250 by default) caps
|
||
how many listings one frame carries. A commodity reseller with thousands of
|
||
stacked resources is a real thing, and an uncapped frame for one is measured in
|
||
megabytes. Over the cap the shop reports `truncated`, and the vendor page says
|
||
*"showing 250 of 3,104 — this shop holds more than the shard publishes"* rather
|
||
than presenting a partial shop as complete.
|
||
|
||
**3. An item may have no name.** Items on the wire carry a cliloc id, not a name.
|
||
On a shard whose operator has not converted a cliloc table
|
||
([`CLILOCS.md`](CLILOCS.md)) the honest render is the item id — never an invented
|
||
label, which would be indistinguishable from a real one.
|
||
|
||
## Privacy: the player's own toggle wins
|
||
|
||
Only vendors whose owner left the in-game **Vendor Search** flag ON are ever sent
|
||
to the site. A player who hides their shop in game is hidden here too, and no
|
||
admin setting overrides that. When they hide one that was already indexed, the
|
||
shard emits `vendor.listing.remove` and the row is deleted — so revoking consent
|
||
takes effect, it does not merely stop refreshing.
|
||
|
||
Shop name, owner character name and location default to **Everyone**, because the
|
||
stock Vendor Search gump already shows exactly that set to any player in game.
|
||
They remain admin-configurable; see [`SHARD_VISIBILITY.md`](SHARD_VISIBILITY.md).
|
||
Account names and website user ids never cross the wire at all.
|
||
|
||
## How it is put together
|
||
|
||
```
|
||
ServUO uo-link sidecar website
|
||
────── ─────────────── ───────
|
||
BridgeMarket.cs vendors table shard_vendors
|
||
round-robin sweep ──────► (whole frame blob) ──────► shard_vendor_items
|
||
per-vendor diff GET /market (paged) + display_name
|
||
vendor.listing resolved at ingest
|
||
vendor.listing.remove
|
||
```
|
||
|
||
**The shard side** walks at most `MarketSweepBatch` vendors per tick from a
|
||
persistent cursor, diffs each against what it last published, and emits a whole
|
||
frame for any shop that moved. Per-tick cost is therefore bounded by the batch,
|
||
not by how many vendors the world holds — full coverage takes
|
||
`ceil(vendors / batch) × MarketSweepSeconds`.
|
||
|
||
**The sidecar** stores each frame whole and serves `GET /market`, its only paged
|
||
read. It normalizes nothing and defines no audiences: it is a dumb forwarder, and
|
||
search is the website's job.
|
||
|
||
**The website** splits each frame into a vendor row and its listings, replacing
|
||
that vendor's whole listing set inside one transaction (the frame is
|
||
authoritative for that vendor, never a delta). Item names are resolved against
|
||
the cliloc table **on the way in** and stored denormalized, which is what makes
|
||
search-by-name possible and keeps the cliloc table off the hot path.
|
||
|
||
## Operating it
|
||
|
||
Everything is in `Config/Bridge.cfg` on the shard. There is nothing to configure
|
||
on the website.
|
||
|
||
| Setting | Default | What it does |
|
||
|---|---|---|
|
||
| `MarketEnabled` | `true` | Master switch. Off publishes nothing; the page shows an empty index. |
|
||
| `MarketSweepSeconds` | `60` | Tick interval. |
|
||
| `MarketSweepBatch` | `25` | Vendors inventoried per tick. Clamped 1..500. |
|
||
| `MarketMaxListings` | `250` | Per-shop listing cap, after which `truncated`. Clamped 1..5000. |
|
||
|
||
**Faster coverage vs. per-tick cost.** Lowering `MarketSweepSeconds` or raising
|
||
`MarketSweepBatch` both refresh the index sooner and both cost more per tick.
|
||
The expensive part is the item walk, which recurses into every container a vendor
|
||
is selling — so a shard of big shops should raise the interval rather than the
|
||
batch.
|
||
|
||
`[bridge status` reports the sweep, including `lastMs` and `maxMs`:
|
||
|
||
```
|
||
market(enabled=True sweeps=42 scanned=108 emitted=27 removed=0 skipped=0
|
||
truncated=0 tracked=27 vendors=27 cursor=2 batch=25 lastMs=0.31 maxMs=15.40)
|
||
```
|
||
|
||
A tick over **50 ms** prints a rate-limited warning naming the knob:
|
||
|
||
```
|
||
[Bridge] market sweep took 82.4 ms (budget 50 ms) - lower Bridge.MarketSweepBatch (now 25) if this persists
|
||
```
|
||
|
||
Measured on a shard with 27 vendors × 40 listings (209k items, 43k mobiles):
|
||
**15.4 ms** for the first cold tick of 25 vendors, **0.3 ms** in steady state —
|
||
the diff is what makes an unchanged world nearly free. Note the arithmetic: 25
|
||
*full* shops at the 250-listing cap is 6,250 items ≈ 95 ms, over budget. Real
|
||
shops hold tens, which is why 25 is the default and why the warning exists.
|
||
|
||
`[bridge sweepnow` runs one tick immediately; `[bridge reload` re-reads the
|
||
settings above without a restart.
|
||
|
||
## Names arriving late
|
||
|
||
Item names come from the cliloc table, and the market sweep will **not** re-send
|
||
an unchanged shop just because the site learned what its items are called. So a
|
||
cliloc import triggers a bulk re-resolution of every stored listing — otherwise
|
||
an operator who configures clilocs after the first sweep would see item ids until
|
||
every shop happened to change on its own. It runs after a boot import and after
|
||
an admin import, takes ~50 ms per thousand listings, and never throws: a failure
|
||
leaves names exactly as they were.
|
||
|
||
## API
|
||
|
||
All under `/api/v1/public/shard`, gated by the `market` feature and
|
||
**rate-limited** — these are the first genuinely expensive public reads on the
|
||
site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*`
|
||
table, reachable with no session).
|
||
|
||
| Route | What |
|
||
|---|---|
|
||
| `GET /market` | Search. Returns **listings**, not vendors — "who sells X and for how much" is the question. `?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=`, `sort ∈ {price_asc, price_desc, recent}`. |
|
||
| `GET /market/meta` | Index size, staleness, and which facets and regions actually hold vendors — so a client builds its filters without running a search it will discard. |
|
||
| `GET /market/vendors/:serial` | One shop and its listings. **404** for a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer. |
|
||
|
||
`q` matches the resolved display name **or** the item's own literal name, because
|
||
an item with a player-set name (most of what is worth searching for on a
|
||
player-run shard) may carry a generic cliloc. `%` and `_` in a query are escaped:
|
||
they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let
|
||
a search for `%` match every listing on the shard.
|
||
|
||
Full schemas are in the OpenAPI spec (`ShardMarketPage`, `ShardMarketVendor`,
|
||
`ShardMarketMeta`, `ShardMarketListing`, `ShardMarketLocation`).
|
||
|
||
## Tables
|
||
|
||
`shard_vendors` (one row per shop) and `shard_vendor_items` (one row per priced
|
||
listing). Both are ingest-owned; nothing else writes to them. No foreign keys,
|
||
in keeping with every other `shard_*` table — the ingest transaction is what
|
||
keeps them consistent, and an FK would turn a malformed frame into a failed write
|
||
rather than a dropped row.
|
||
|
||
There is deliberately **no `payload` column** on `shard_vendors`, unlike the
|
||
points board next door. The board's top-N is a fixed-size list read whole, so it
|
||
lives in JSON; here the items *are* the searchable rows, so they are normalized
|
||
and there is nothing left worth duplicating. The sidecar keeps the whole blob,
|
||
because outage resilience is its job.
|
||
|
||
`shard_vendor_items.display_name` is denormalized and indexed (alone, and
|
||
composite with `price` for "cheapest matching X"). See "Names arriving late"
|
||
above for how it is kept current.
|