feat(shard): the player-vendor marketplace
Protocol 3.0 §8, the website half. Ingests vendor.listing / vendor.listing.remove
into shard_vendors + shard_vendor_items, serves a searchable public API over
them, and ships /site/market and /site/market/vendors/:serial.
Three things the pages have to say out loud, all consequences of how the data is
gathered:
- The prices are NOT live. The shard sweeps vendors round-robin, so a shop can be
a full cycle behind. The banner is driven by the OLDEST vendor row, not the
newest — the one stale shop is the one that wastes somebody's trip.
- A shop can be truncated. `total` exceeding `count` means the shop holds more
than the shard publishes per frame; the vendor page says "showing 250 of 3,104"
rather than presenting a partial shop as complete.
- An item may have no name. On a shard with no cliloc table the honest render is
the item id, never an invented label.
## The pre-wired visibility rules, re-checked
Part A pre-wired market.ownerName and market.location before the frame existed,
and the sibling rule it pre-wired for leaderboards (`characterName`) turned out
to be INERT because projectValue matches literal JSON keys. Both market rules
were checked against the real frame this time:
- `ownerName` is a real key. Kept.
- `location` is a real key ONLY because the frame nests it. Flat map/x/y/region
would have made the rule match nothing — the same failure, one part later. It
is nested on the wire and on the read model so one rule hides the facet, the
coordinates, the region and the house together; five flat keys would be five
rules that drift apart.
- `ownerSerial` was ADDED. An admin who hides the owner's name and leaves a
serial that the leaderboards and guild boards resolve back to that same name
has not hidden anything.
Tests assert all three bite, on the stored read model AND on the raw frame —
the market's SSE stream is off by default but an admin can turn it on, and a rule
that worked on only one path is exactly the leak §3.6.1 records.
## Notable
- **No payload column on shard_vendors**, unlike shard_points_boards next door.
The board's top-N is a fixed-size list read whole; here the items ARE the
searchable rows, so they are normalized and nothing is left worth duplicating.
- **display_name is denormalized at ingest** (literal name preferred over the
cliloc — a player set it, so it is more specific). Resolving at query time
would put the cliloc table on the hot path and make search-by-name impossible.
Because the shard's diff sweep will not re-send an unchanged shop just because
the site learned what its items are called, a cliloc import now triggers a bulk
re-resolution — 50 ms per thousand rows, never throws.
- **updated_at is written explicitly** on every upsert. MariaDB does not fire ON
UPDATE CURRENT_TIMESTAMP when every column is written back unchanged, and a
shop re-published identically is still freshly confirmed — without this the
staleness banner would age a perfectly current shop forever.
- **LIKE wildcards in `q` are escaped.** `%` and `_` are LIKE metacharacters, not
SQL ones, so parameterization does not neutralize them: `?q=%` would otherwise
match every listing on the shard.
- **Rate-limited** (60/min/IP), the only limited public read. Every other public
GET is an indexed lookup of bounded size; this is a LIKE scan plus a COUNT over
the largest shard_* table, anonymous by default.
- Reconnect backfill pages /market, bounded by MARKET_SNAPSHOT_MAX = 5000 and
stopping on a short page as well as on `total`, so a concurrent sweep shrinking
the index cannot spin the walk.
## How it was tested
673 server tests pass (27 new). Client builds clean; swagger-output.json,
routes.manifest.json and routes.guards.json regenerated.
Verified full-stack against the live MariaDB and a real shard, not only units:
- 27 real vendors / 1,040 listings swept off the ServUO tree, through the Rust
sidecar, into the site — names resolving through the cliloc table ("longsword",
"katana"), real facets and regions in the filters.
- `?q=sword` 682, `?q=%` and `?q=_` **0** (the escape), map/region/price/sort
filters, paging, and the vendor detail route.
- Visibility live: fields gated to staff vanish for an anonymous caller while
shopName and price survive; audience=player 403s; enabled=0 404s; and
/shard/features correctly drops `market` so the nav hides it.
- Re-publishing a shop smaller leaves no orphan items; an identical re-publish
moves updated_at.
- The limiter fires (38x200 then 32x429 on a 70-request burst).
Not covered by an automated test: the two React pages are presentational and this
repo's client suite covers pure-logic modules only. They were driven against the
live API above, but not rendered in a DOM harness.
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
114
server/test/shardIngest.market.test.js
Normal file
114
server/test/shardIngest.market.test.js
Normal file
@@ -0,0 +1,114 @@
|
||||
const { test, beforeEach } = require('node:test')
|
||||
const assert = require('node:assert/strict')
|
||||
|
||||
const shardIngest = require('../src/utils/shardIngest')
|
||||
|
||||
// Protocol 3.0 vendor.listing / vendor.listing.remove routing. Same shape as
|
||||
// shardIngest.points.test.js: stubbed deps, asserting where the dispatcher sends
|
||||
// the frame and whether it is appended to the event log.
|
||||
function makeDeps() {
|
||||
const calls = { upserts: [], removes: [], appended: [], broadcast: [] }
|
||||
const noop = async () => {}
|
||||
return {
|
||||
calls,
|
||||
shardEvents: { append: async (row) => { calls.appended.push(row); return true } },
|
||||
shardState: {
|
||||
// Present so any stray routing is a harmless no-op rather than a crash.
|
||||
clearOnline: noop, upsertOnline: noop, setOffline: noop, upsertHouse: noop,
|
||||
addEconomySample: noop, setRuleset: noop, upsertPointsBoard: noop,
|
||||
},
|
||||
shardMarket: {
|
||||
upsertVendor: async (ev) => { calls.upserts.push(ev) },
|
||||
removeVendor: async (serial) => { calls.removes.push(serial) },
|
||||
},
|
||||
shardLinks: { removeByAccount: noop },
|
||||
uoLinkConfig: { recordStatus: noop },
|
||||
broadcast: (ev) => { calls.broadcast.push(ev) },
|
||||
pushDispatch: async () => {},
|
||||
log: { warn() {}, info() {}, error() {} },
|
||||
}
|
||||
}
|
||||
|
||||
const FRAME = {
|
||||
kind: 'vendor.listing',
|
||||
t: 1000,
|
||||
serial: '0x40001234',
|
||||
shopName: "Darrow's Bargains",
|
||||
ownerSerial: '0x1A2B',
|
||||
ownerName: 'Darrow',
|
||||
location: { map: 'Trammel', x: 1421, y: 1699, z: 0, region: 'Britain', house: "Darrow's Villa" },
|
||||
count: 2,
|
||||
total: 2,
|
||||
truncated: false,
|
||||
items: [
|
||||
{ serial: '0x40012ABC', itemId: 3922, hue: 0, amount: 1, price: 25000, name: null, cliloc: 1023721 },
|
||||
{ serial: '0x40012ABD', itemId: 7026, hue: 1157, amount: 3, price: 500, name: 'a shard sigil', cliloc: 1041243 },
|
||||
],
|
||||
}
|
||||
|
||||
beforeEach(() => shardIngest.reset())
|
||||
|
||||
test('vendor.listing routes to the market model with the whole frame', async () => {
|
||||
const deps = makeDeps()
|
||||
await shardIngest.ingest(FRAME, deps)
|
||||
assert.equal(deps.calls.upserts.length, 1)
|
||||
const stored = deps.calls.upserts[0]
|
||||
assert.equal(stored.serial, '0x40001234')
|
||||
assert.equal(stored.location.region, 'Britain')
|
||||
assert.equal(stored.items.length, 2)
|
||||
})
|
||||
|
||||
test('vendor.listing.remove routes to removeVendor with the serial', async () => {
|
||||
const deps = makeDeps()
|
||||
await shardIngest.ingest({ kind: 'vendor.listing.remove', t: 2000, serial: '0x40001234' }, deps)
|
||||
assert.deepEqual(deps.calls.removes, ['0x40001234'])
|
||||
assert.equal(deps.calls.upserts.length, 0)
|
||||
})
|
||||
|
||||
// The market IS the state. One frame carries up to 250 listings and the sweep
|
||||
// re-emits a shop on any price change, so logging would turn shard_events into a
|
||||
// price history nobody reads — the strongest case of the three v3 kinds.
|
||||
test('neither market kind is appended to the event log', async () => {
|
||||
const deps = makeDeps()
|
||||
const a = await shardIngest.ingest(FRAME, deps)
|
||||
const b = await shardIngest.ingest({ kind: 'vendor.listing.remove', serial: '0x40001234' }, deps)
|
||||
assert.equal(a.logged, false)
|
||||
assert.equal(b.logged, false)
|
||||
assert.equal(deps.calls.appended.length, 0)
|
||||
assert.equal(shardIngest.LOGGED_KINDS.has('vendor.listing'), false)
|
||||
assert.equal(shardIngest.LOGGED_KINDS.has('vendor.listing.remove'), false)
|
||||
})
|
||||
|
||||
// Broadcast is unconditional at this layer — whether it actually reaches anyone
|
||||
// is shardBroadcast's call, and the market feature ships with its stream off.
|
||||
test('vendor.listing is handed to the broadcaster', async () => {
|
||||
const deps = makeDeps()
|
||||
await shardIngest.ingest(FRAME, deps)
|
||||
assert.equal(deps.calls.broadcast.length, 1)
|
||||
assert.equal(deps.calls.broadcast[0].kind, 'vendor.listing')
|
||||
})
|
||||
|
||||
test('a backfilled vendor.listing still stores but does not broadcast', async () => {
|
||||
const deps = makeDeps()
|
||||
await shardIngest.ingest(FRAME, { ...deps, fromBackfill: true })
|
||||
assert.equal(deps.calls.upserts.length, 1)
|
||||
assert.equal(deps.calls.broadcast.length, 0)
|
||||
})
|
||||
|
||||
// The reconnect backfill replays the whole index through this path, so a single
|
||||
// bad vendor must not abort it.
|
||||
test('an upsertVendor failure does not throw or stop the broadcast', async () => {
|
||||
const deps = makeDeps()
|
||||
deps.shardMarket.upsertVendor = async () => { throw new Error('db down') }
|
||||
const r = await shardIngest.ingest(FRAME, deps)
|
||||
assert.equal(r.logged, false)
|
||||
assert.equal(deps.calls.broadcast.length, 1)
|
||||
})
|
||||
|
||||
// Each vendor is its own row; the frame is authoritative for that vendor only.
|
||||
test('two vendors are stored independently', async () => {
|
||||
const deps = makeDeps()
|
||||
await shardIngest.ingest(FRAME, deps)
|
||||
await shardIngest.ingest({ ...FRAME, serial: '0x40009999', shopName: 'Second Shop' }, deps)
|
||||
assert.deepEqual(deps.calls.upserts.map((v) => v.serial), ['0x40001234', '0x40009999'])
|
||||
})
|
||||
Reference in New Issue
Block a user