Files
website/server/src/router/v1/public/shard.controller.js
wtclaude 8771a1cf6c feat(shard): the player-vendor marketplace
Protocol 3.0 §8, the website half. Ingests vendor.listing / vendor.listing.remove
into shard_vendors + shard_vendor_items, serves a searchable public API over
them, and ships /site/market and /site/market/vendors/:serial.

Three things the pages have to say out loud, all consequences of how the data is
gathered:

- The prices are NOT live. The shard sweeps vendors round-robin, so a shop can be
  a full cycle behind. The banner is driven by the OLDEST vendor row, not the
  newest — the one stale shop is the one that wastes somebody's trip.
- A shop can be truncated. `total` exceeding `count` means the shop holds more
  than the shard publishes per frame; the vendor page says "showing 250 of 3,104"
  rather than presenting a partial shop as complete.
- An item may have no name. On a shard with no cliloc table the honest render is
  the item id, never an invented label.

## The pre-wired visibility rules, re-checked

Part A pre-wired market.ownerName and market.location before the frame existed,
and the sibling rule it pre-wired for leaderboards (`characterName`) turned out
to be INERT because projectValue matches literal JSON keys. Both market rules
were checked against the real frame this time:

- `ownerName` is a real key. Kept.
- `location` is a real key ONLY because the frame nests it. Flat map/x/y/region
  would have made the rule match nothing — the same failure, one part later. It
  is nested on the wire and on the read model so one rule hides the facet, the
  coordinates, the region and the house together; five flat keys would be five
  rules that drift apart.
- `ownerSerial` was ADDED. An admin who hides the owner's name and leaves a
  serial that the leaderboards and guild boards resolve back to that same name
  has not hidden anything.

Tests assert all three bite, on the stored read model AND on the raw frame —
the market's SSE stream is off by default but an admin can turn it on, and a rule
that worked on only one path is exactly the leak §3.6.1 records.

## Notable

- **No payload column on shard_vendors**, unlike shard_points_boards next door.
  The board's top-N is a fixed-size list read whole; here the items ARE the
  searchable rows, so they are normalized and nothing is left worth duplicating.
- **display_name is denormalized at ingest** (literal name preferred over the
  cliloc — a player set it, so it is more specific). Resolving at query time
  would put the cliloc table on the hot path and make search-by-name impossible.
  Because the shard's diff sweep will not re-send an unchanged shop just because
  the site learned what its items are called, a cliloc import now triggers a bulk
  re-resolution — 50 ms per thousand rows, never throws.
- **updated_at is written explicitly** on every upsert. MariaDB does not fire ON
  UPDATE CURRENT_TIMESTAMP when every column is written back unchanged, and a
  shop re-published identically is still freshly confirmed — without this the
  staleness banner would age a perfectly current shop forever.
- **LIKE wildcards in `q` are escaped.** `%` and `_` are LIKE metacharacters, not
  SQL ones, so parameterization does not neutralize them: `?q=%` would otherwise
  match every listing on the shard.
- **Rate-limited** (60/min/IP), the only limited public read. Every other public
  GET is an indexed lookup of bounded size; this is a LIKE scan plus a COUNT over
  the largest shard_* table, anonymous by default.
- Reconnect backfill pages /market, bounded by MARKET_SNAPSHOT_MAX = 5000 and
  stopping on a short page as well as on `total`, so a concurrent sweep shrinking
  the index cannot spin the walk.

## How it was tested

673 server tests pass (27 new). Client builds clean; swagger-output.json,
routes.manifest.json and routes.guards.json regenerated.

Verified full-stack against the live MariaDB and a real shard, not only units:

- 27 real vendors / 1,040 listings swept off the ServUO tree, through the Rust
  sidecar, into the site — names resolving through the cliloc table ("longsword",
  "katana"), real facets and regions in the filters.
- `?q=sword` 682, `?q=%` and `?q=_` **0** (the escape), map/region/price/sort
  filters, paging, and the vendor detail route.
- Visibility live: fields gated to staff vanish for an anonymous caller while
  shopName and price survive; audience=player 403s; enabled=0 404s; and
  /shard/features correctly drops `market` so the nav hides it.
- Re-publishing a shop smaller leaves no orphan items; an identical re-publish
  moves updated_at.
- The limiter fires (38x200 then 32x429 on a 70-request burst).

Not covered by an automated test: the two React pages are presentational and this
repo's client suite covers pure-logic modules only. They were driven against the
live API above, but not rendered in a DOM harness.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-29 09:51:50 -05:00

423 lines
17 KiB
JavaScript

// ── Public: shard live data ────────────────────────────────────────────────
//
// Same-origin, token-free read endpoints backed by the data the WS ingest
// pipeline persists (shard_online / shard_events / shard_economy / shard_houses)
// plus a live character round-trip to the sidecar. The browser never sees the
// sidecar URL or token — every sidecar call is server-side (uoLinkClient).
//
// The stored-data endpoints are cheap DB reads. The live /char endpoint hits the
// running shard, so it is briefly cached and degrades gracefully: a 503 (shard
// restarting) surfaces as a retry-able banner rather than an error.
const shardEvents = require('../../../model/shardEvents/shardEvents.model')
const shardState = require('../../../model/shardState/shardState.model')
const shardMarket = require('../../../model/shardMarket/shardMarket.model')
const uoLinkConfig = require('../../../model/uoLinkConfig/uoLinkConfig.model')
const broadcast = require('../../../utils/shardBroadcast')
const visibility = require('../../../utils/shardVisibility')
const log = require('../../../utils/logger')('public-shard')
// GET /public/shard/status — connection state + online count + latest economy.
async function getStatus(req, res) {
try {
const config = await uoLinkConfig.getSafe()
const [online, economy] = await Promise.all([
shardState.onlineCount(),
shardState.latestEconomy(),
])
return res.json({
enabled: config.enabled,
status: config.status,
pluginConnected: config.pluginConnected,
lastEventAt: config.lastEventAt,
onlineCount: online,
economy,
})
} catch (err) {
log.error('shard.getStatus', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/feed?kind=&limit= — recent notable events from the log.
//
// This is the stored-history twin of the SSE stream, and it must reach the same
// verdict the stream does about the same event. Two things are therefore resolved
// against the LIVE config rather than the compiled defaults:
//
// • which kinds this viewer may read at all — `visibleKinds`, not the static
// PUBLIC_KINDS set (which is fixed at module load, so an admin moving
// `guilds` to `staff` would gate /guilds while /feed kept serving
// guild.join to anonymous callers), and
// • the payload itself, projected per event against ITS OWN kind's feature —
// the rows are a mix of features, and without this the stored frames were
// returned verbatim, `acct`/`webId` and all, on an anonymous endpoint.
async function getFeed(req, res) {
try {
const config = await visibility.getConfig()
const level = req.viewerLevel || (await visibility.viewerLevel(req))
const allowed = new Set(visibility.visibleKinds(level, config))
const { kind, limit } = req.query
// No readable kinds ⇒ nothing to serve. Returning early also keeps us clear
// of `list({ kinds: [] })`, which means "no filter", not "match nothing".
if (allowed.size === 0) return res.json([])
let events
if (kind) {
if (!allowed.has(kind)) return res.json([])
events = await shardEvents.list({ kind, limit })
} else {
events = await shardEvents.list({ kinds: [...allowed], limit })
}
return res.json(
events.map((ev) => ({
...ev,
payload: visibility.projectFeature(
visibility.KIND_FEATURE.get(ev.kind),
ev.payload,
level,
config,
),
})),
)
} catch (err) {
log.error('shard.getFeed', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/economy — gold-supply series, oldest → newest.
async function getEconomy(req, res) {
try {
return res.json(await shardState.listEconomy(req.query.limit))
} catch (err) {
log.error('shard.getEconomy', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/online — players online now whose account is linked to a
// STAFF website user (admin/editor/moderator). Everyone sees that a staff member
// is online (name + serial); their in-game location (map + coordinates) is gated
// on the `presence` feature's `location` field rule, which defaults to `staff`
// — the same admin/moderator set this used to hardcode. Non-staff players are
// never listed.
async function canSeeStaffLocation(req) {
const config = await visibility.getConfig()
const required = config.presence?.fields?.location || 'staff'
const level = req.viewerLevel || (await visibility.viewerLevel(req))
return visibility.meets(level, required)
}
async function getOnline(req, res) {
try {
const rows = await shardState.listOnlineLinked()
const showLocation = await canSeeStaffLocation(req)
return res.json(
rows.map((r) => {
const entry = { serial: r.serial, name: r.name }
if (showLocation) {
entry.map = r.map
entry.x = r.x
entry.y = r.y
entry.z = r.z
}
return entry
}),
)
} catch (err) {
log.error('shard.getOnline', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/idoc — houses currently in danger (stage IDOC).
//
// Projected: shapeHouse flattens the owner actor into `ownerSerial`/`ownerAcct`/
// `ownerName`, so this endpoint used to hand an anonymous caller the house
// owner's GAME ACCOUNT NAME. The public IDOC board only ever needed name, region
// and location — which is all that survives projection below `staff`.
async function getIdoc(req, res) {
try {
return res.json(await visibility.project('houses', await shardState.listIdoc(), req))
} catch (err) {
log.error('shard.getIdoc', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/champs — the current champion-spawn board (all categories).
// Served from our own store; live deltas (champ.update / champ.remove) arrive on
// the public SSE stream so the page can update in place.
async function getChamps(req, res) {
try {
return res.json(await visibility.project('champs', await shardState.listChamps(), req))
} catch (err) {
log.error('shard.getChamps', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/guilds — the current guild board. Served from our store;
// live via guild.update / guild.remove / guild.join on the public SSE stream.
//
// Projected: the stored payload is the raw guild.update frame, whose `leader`
// actor carries `acct` and `webId`. Those are admin-only and were previously
// returned verbatim to anonymous callers.
async function getGuilds(req, res) {
try {
return res.json(await visibility.project('guilds', await shardState.listGuilds(), req))
} catch (err) {
log.error('shard.getGuilds', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/governors — the current town-governor board (empty on shards
// without City Loyalty). Live via city.update on the public SSE stream. Projected
// for the same reason as getGuilds: `governor` / `governorElect` are actors.
async function getGovernors(req, res) {
try {
return res.json(await visibility.project('governors', await shardState.listGovernors(), req))
} catch (err) {
log.error('shard.getGovernors', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/governors/:city/history — the term ledger for one city
// (look-back: "who were all the governors of Britain?"), newest first.
async function getGovernorHistory(req, res) {
try {
const terms = await shardState.listGovernorHistory(req.params.city, req.query.limit)
return res.json(await visibility.project('governors', terms, req))
} catch (err) {
log.error('shard.getGovernorHistory', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/presence — the online-population aggregate (count + per-facet
// + per-region). Live via presence.online on the public SSE stream.
async function getPresence(req, res) {
try {
return res.json(await visibility.project('presence', await shardState.latestPresence(), req))
} catch (err) {
log.error('shard.getPresence', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/houses — PUBLIC view: only houses in danger (IDOC), and only
// their location (name + region + map/coords). Owner, price, co-owners and decay
// detail are staff-only (see admin GET /admin/shard/houses). Live via house.decay
// on the public SSE stream. This is the "where are the falling houses" board.
async function getHouses(req, res) {
try {
const idoc = await shardState.listIdoc()
const publicHouses = idoc.map((h) => ({
serial: h.serial,
name: h.name,
region: h.region,
map: h.map,
x: h.x,
y: h.y,
z: h.z,
isIdoc: true,
}))
// Already a hand-picked safe subset; projected anyway so an admin who
// tightens a `houses` field rule sees it honoured on every houses surface
// rather than on some of them.
return res.json(await visibility.project('houses', publicHouses, req))
} catch (err) {
log.error('shard.getHouses', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/ruleset — the shard's published ruleset (Protocol 3.0):
// expansion, which optional systems are on, skill/stat caps, account and house
// limits, champion scroll rules, the save/restart schedule. Served from our own
// store, so it renders while the shard is down; live via world.ruleset on the
// public SSE stream.
//
// `null` means the shard has never published one (an old plugin, or
// Bridge.RulesetEnabled=false) — a real answer, distinct from a published
// ruleset, and the page says so rather than rendering an empty one.
//
// Projected like every other shard read (§3.6.1's rule: a read path that returns
// shard data and does not call projectFeature is a bug). The `connect` string is
// the one configurable field — an operator who published a connect address may
// still want it behind a login.
async function getRuleset(req, res) {
try {
const ruleset = await shardState.getRuleset()
if (!ruleset) return res.json(null)
return res.json(await visibility.project('ruleset', ruleset, req))
} catch (err) {
log.error('shard.getRuleset', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// The shard keys boards by its own PointsType enum name (QueensLoyalty,
// CleanUpBritannia, …). Constrain the path param to that shape before it reaches
// the model: the column is VARCHAR(48), and an unbounded string here is a needless
// query on a value that can only ever be an identifier.
const SYSTEM_RE = /^[A-Za-z][A-Za-z0-9_]{0,47}$/
// GET /public/shard/points — every points/loyalty leaderboard the shard publishes.
// Served from our own store, so the page renders while the shard is down — which
// matters more here than for live state: these are standings accumulated over
// months, and blanking them during a restart would look like a data loss.
async function getPointsBoards(req, res) {
try {
const boards = await shardState.listPointsBoards()
return res.json(await visibility.project('leaderboards', boards, req))
} catch (err) {
log.error('shard.getPointsBoards', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/points/:system — one system's board.
//
// 404 for a system the shard has never published, matching the sidecar: "no such
// board" and "a board nobody is on yet" are different answers.
async function getPointsBoard(req, res) {
const { system } = req.params
if (!SYSTEM_RE.test(system)) return res.status(400).json({ message: 'Invalid points system.' })
try {
const board = await shardState.getPointsBoard(system)
if (!board) return res.status(404).json({ message: 'Unknown points system.' })
return res.json(await visibility.project('leaderboards', board, req))
} catch (err) {
log.error('shard.getPointsBoard', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// ── Marketplace (Protocol 3.0 vendor.listing) ──────────────────────────────
//
// The shard-wide player-vendor index. Served entirely from our own tables — the
// sidecar is never touched on this path — so shops stay browsable while the shard
// is down, labelled with how stale they may be.
//
// The staleness label is not decoration. The shard sweeps vendors round-robin, so
// a shop can legitimately be a full cycle behind; a page that implied live prices
// would send people to a vendor whose item sold twenty minutes ago.
// The serial spelling the bridge uses everywhere: "0x" and hex. Constrained
// before it reaches the model, like SYSTEM_RE above.
const SERIAL_RE = /^0x[0-9A-Fa-f]{1,16}$/
const intParam = (value) => {
const n = Number.parseInt(value, 10)
return Number.isFinite(n) ? n : undefined
}
// GET /public/shard/market — search the index.
//
// Returns LISTINGS, not vendors: "who sells a vanquishing kryss and for how much"
// is the question, and a vendor-shaped result would make every caller flatten the
// shops back out.
async function getMarket(req, res) {
try {
const page = await shardMarket.search({
q: typeof req.query.q === 'string' ? req.query.q : '',
minPrice: intParam(req.query.minPrice),
maxPrice: intParam(req.query.maxPrice),
itemId: intParam(req.query.itemId),
map: typeof req.query.map === 'string' ? req.query.map : '',
region: typeof req.query.region === 'string' ? req.query.region : '',
sort: typeof req.query.sort === 'string' ? req.query.sort : 'price_asc',
limit: intParam(req.query.limit) ?? 50,
offset: intParam(req.query.offset) ?? 0,
})
return res.json(await visibility.project('market', page, req))
} catch (err) {
log.error('shard.getMarket', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/market/meta — index size, staleness, and the filter options
// (which facets and regions actually hold vendors). Separate from the search so
// the page can build its filters without running a query it will throw away.
async function getMarketMeta(req, res) {
try {
return res.json(await visibility.project('market', await shardMarket.meta(), req))
} catch (err) {
log.error('shard.getMarketMeta', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/market/vendors/:serial — one shop and its listings.
//
// 404 for a serial the index has never seen, which also covers a vendor that has
// since been dismissed or hidden: to an anonymous caller "no such shop" is the
// only honest answer, and distinguishing the two would leak that a vendor exists
// but was hidden.
async function getMarketVendor(req, res) {
const { serial } = req.params
if (!SERIAL_RE.test(serial)) return res.status(400).json({ message: 'Invalid vendor serial.' })
try {
const vendor = await shardMarket.getVendor(serial, {
limit: intParam(req.query.limit) ?? 250,
offset: intParam(req.query.offset) ?? 0,
})
if (!vendor) return res.status(404).json({ message: 'Unknown vendor.' })
return res.json(await visibility.project('market', vendor, req))
} catch (err) {
log.error('shard.getMarketVendor', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/features — the shard features THIS caller can actually see,
// so the SPA (and the Android client) can hide nav entries instead of rendering
// links that 403. Deliberately reports only what the viewer may reach: the list
// itself must not disclose the existence of a feature they're gated out of.
async function getFeatures(req, res) {
try {
const config = await visibility.getConfig()
const level = await visibility.viewerLevel(req)
return res.json({ level, features: visibility.visibleFeatures(level, config) })
} catch (err) {
log.error('shard.getFeatures', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// GET /public/shard/stream — live-event SSE channel. What arrives depends on the
// caller's audience rung, resolved once at subscribe time; see shardBroadcast.js.
function stream(req, res) {
return broadcast.subscribe(req, res, 'public')
}
module.exports = {
getStatus,
getFeed,
getEconomy,
getOnline,
getIdoc,
getChamps,
getGuilds,
getGovernors,
getGovernorHistory,
getPresence,
getHouses,
getRuleset,
getPointsBoards,
getPointsBoard,
getMarket,
getMarketMeta,
getMarketVendor,
getFeatures,
stream,
}