Files
website/server/test/shardIngest.market.test.js
wtclaude 8771a1cf6c feat(shard): the player-vendor marketplace
Protocol 3.0 §8, the website half. Ingests vendor.listing / vendor.listing.remove
into shard_vendors + shard_vendor_items, serves a searchable public API over
them, and ships /site/market and /site/market/vendors/:serial.

Three things the pages have to say out loud, all consequences of how the data is
gathered:

- The prices are NOT live. The shard sweeps vendors round-robin, so a shop can be
  a full cycle behind. The banner is driven by the OLDEST vendor row, not the
  newest — the one stale shop is the one that wastes somebody's trip.
- A shop can be truncated. `total` exceeding `count` means the shop holds more
  than the shard publishes per frame; the vendor page says "showing 250 of 3,104"
  rather than presenting a partial shop as complete.
- An item may have no name. On a shard with no cliloc table the honest render is
  the item id, never an invented label.

## The pre-wired visibility rules, re-checked

Part A pre-wired market.ownerName and market.location before the frame existed,
and the sibling rule it pre-wired for leaderboards (`characterName`) turned out
to be INERT because projectValue matches literal JSON keys. Both market rules
were checked against the real frame this time:

- `ownerName` is a real key. Kept.
- `location` is a real key ONLY because the frame nests it. Flat map/x/y/region
  would have made the rule match nothing — the same failure, one part later. It
  is nested on the wire and on the read model so one rule hides the facet, the
  coordinates, the region and the house together; five flat keys would be five
  rules that drift apart.
- `ownerSerial` was ADDED. An admin who hides the owner's name and leaves a
  serial that the leaderboards and guild boards resolve back to that same name
  has not hidden anything.

Tests assert all three bite, on the stored read model AND on the raw frame —
the market's SSE stream is off by default but an admin can turn it on, and a rule
that worked on only one path is exactly the leak §3.6.1 records.

## Notable

- **No payload column on shard_vendors**, unlike shard_points_boards next door.
  The board's top-N is a fixed-size list read whole; here the items ARE the
  searchable rows, so they are normalized and nothing is left worth duplicating.
- **display_name is denormalized at ingest** (literal name preferred over the
  cliloc — a player set it, so it is more specific). Resolving at query time
  would put the cliloc table on the hot path and make search-by-name impossible.
  Because the shard's diff sweep will not re-send an unchanged shop just because
  the site learned what its items are called, a cliloc import now triggers a bulk
  re-resolution — 50 ms per thousand rows, never throws.
- **updated_at is written explicitly** on every upsert. MariaDB does not fire ON
  UPDATE CURRENT_TIMESTAMP when every column is written back unchanged, and a
  shop re-published identically is still freshly confirmed — without this the
  staleness banner would age a perfectly current shop forever.
- **LIKE wildcards in `q` are escaped.** `%` and `_` are LIKE metacharacters, not
  SQL ones, so parameterization does not neutralize them: `?q=%` would otherwise
  match every listing on the shard.
- **Rate-limited** (60/min/IP), the only limited public read. Every other public
  GET is an indexed lookup of bounded size; this is a LIKE scan plus a COUNT over
  the largest shard_* table, anonymous by default.
- Reconnect backfill pages /market, bounded by MARKET_SNAPSHOT_MAX = 5000 and
  stopping on a short page as well as on `total`, so a concurrent sweep shrinking
  the index cannot spin the walk.

## How it was tested

673 server tests pass (27 new). Client builds clean; swagger-output.json,
routes.manifest.json and routes.guards.json regenerated.

Verified full-stack against the live MariaDB and a real shard, not only units:

- 27 real vendors / 1,040 listings swept off the ServUO tree, through the Rust
  sidecar, into the site — names resolving through the cliloc table ("longsword",
  "katana"), real facets and regions in the filters.
- `?q=sword` 682, `?q=%` and `?q=_` **0** (the escape), map/region/price/sort
  filters, paging, and the vendor detail route.
- Visibility live: fields gated to staff vanish for an anonymous caller while
  shopName and price survive; audience=player 403s; enabled=0 404s; and
  /shard/features correctly drops `market` so the nav hides it.
- Re-publishing a shop smaller leaves no orphan items; an identical re-publish
  moves updated_at.
- The limiter fires (38x200 then 32x429 on a 70-request burst).

Not covered by an automated test: the two React pages are presentational and this
repo's client suite covers pure-logic modules only. They were driven against the
live API above, but not rendered in a DOM harness.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-29 09:51:50 -05:00

115 lines
4.7 KiB
JavaScript

const { test, beforeEach } = require('node:test')
const assert = require('node:assert/strict')
const shardIngest = require('../src/utils/shardIngest')
// Protocol 3.0 vendor.listing / vendor.listing.remove routing. Same shape as
// shardIngest.points.test.js: stubbed deps, asserting where the dispatcher sends
// the frame and whether it is appended to the event log.
function makeDeps() {
const calls = { upserts: [], removes: [], appended: [], broadcast: [] }
const noop = async () => {}
return {
calls,
shardEvents: { append: async (row) => { calls.appended.push(row); return true } },
shardState: {
// Present so any stray routing is a harmless no-op rather than a crash.
clearOnline: noop, upsertOnline: noop, setOffline: noop, upsertHouse: noop,
addEconomySample: noop, setRuleset: noop, upsertPointsBoard: noop,
},
shardMarket: {
upsertVendor: async (ev) => { calls.upserts.push(ev) },
removeVendor: async (serial) => { calls.removes.push(serial) },
},
shardLinks: { removeByAccount: noop },
uoLinkConfig: { recordStatus: noop },
broadcast: (ev) => { calls.broadcast.push(ev) },
pushDispatch: async () => {},
log: { warn() {}, info() {}, error() {} },
}
}
const FRAME = {
kind: 'vendor.listing',
t: 1000,
serial: '0x40001234',
shopName: "Darrow's Bargains",
ownerSerial: '0x1A2B',
ownerName: 'Darrow',
location: { map: 'Trammel', x: 1421, y: 1699, z: 0, region: 'Britain', house: "Darrow's Villa" },
count: 2,
total: 2,
truncated: false,
items: [
{ serial: '0x40012ABC', itemId: 3922, hue: 0, amount: 1, price: 25000, name: null, cliloc: 1023721 },
{ serial: '0x40012ABD', itemId: 7026, hue: 1157, amount: 3, price: 500, name: 'a shard sigil', cliloc: 1041243 },
],
}
beforeEach(() => shardIngest.reset())
test('vendor.listing routes to the market model with the whole frame', async () => {
const deps = makeDeps()
await shardIngest.ingest(FRAME, deps)
assert.equal(deps.calls.upserts.length, 1)
const stored = deps.calls.upserts[0]
assert.equal(stored.serial, '0x40001234')
assert.equal(stored.location.region, 'Britain')
assert.equal(stored.items.length, 2)
})
test('vendor.listing.remove routes to removeVendor with the serial', async () => {
const deps = makeDeps()
await shardIngest.ingest({ kind: 'vendor.listing.remove', t: 2000, serial: '0x40001234' }, deps)
assert.deepEqual(deps.calls.removes, ['0x40001234'])
assert.equal(deps.calls.upserts.length, 0)
})
// The market IS the state. One frame carries up to 250 listings and the sweep
// re-emits a shop on any price change, so logging would turn shard_events into a
// price history nobody reads — the strongest case of the three v3 kinds.
test('neither market kind is appended to the event log', async () => {
const deps = makeDeps()
const a = await shardIngest.ingest(FRAME, deps)
const b = await shardIngest.ingest({ kind: 'vendor.listing.remove', serial: '0x40001234' }, deps)
assert.equal(a.logged, false)
assert.equal(b.logged, false)
assert.equal(deps.calls.appended.length, 0)
assert.equal(shardIngest.LOGGED_KINDS.has('vendor.listing'), false)
assert.equal(shardIngest.LOGGED_KINDS.has('vendor.listing.remove'), false)
})
// Broadcast is unconditional at this layer — whether it actually reaches anyone
// is shardBroadcast's call, and the market feature ships with its stream off.
test('vendor.listing is handed to the broadcaster', async () => {
const deps = makeDeps()
await shardIngest.ingest(FRAME, deps)
assert.equal(deps.calls.broadcast.length, 1)
assert.equal(deps.calls.broadcast[0].kind, 'vendor.listing')
})
test('a backfilled vendor.listing still stores but does not broadcast', async () => {
const deps = makeDeps()
await shardIngest.ingest(FRAME, { ...deps, fromBackfill: true })
assert.equal(deps.calls.upserts.length, 1)
assert.equal(deps.calls.broadcast.length, 0)
})
// The reconnect backfill replays the whole index through this path, so a single
// bad vendor must not abort it.
test('an upsertVendor failure does not throw or stop the broadcast', async () => {
const deps = makeDeps()
deps.shardMarket.upsertVendor = async () => { throw new Error('db down') }
const r = await shardIngest.ingest(FRAME, deps)
assert.equal(r.logged, false)
assert.equal(deps.calls.broadcast.length, 1)
})
// Each vendor is its own row; the frame is authoritative for that vendor only.
test('two vendors are stored independently', async () => {
const deps = makeDeps()
await shardIngest.ingest(FRAME, deps)
await shardIngest.ingest({ ...FRAME, serial: '0x40009999', shopName: 'Second Shop' }, deps)
assert.deepEqual(deps.calls.upserts.map((v) => v.serial), ['0x40001234', '0x40009999'])
})