Protocol 3.0 §8, the website half. Ingests vendor.listing / vendor.listing.remove
into shard_vendors + shard_vendor_items, serves a searchable public API over
them, and ships /site/market and /site/market/vendors/:serial.
Three things the pages have to say out loud, all consequences of how the data is
gathered:
- The prices are NOT live. The shard sweeps vendors round-robin, so a shop can be
a full cycle behind. The banner is driven by the OLDEST vendor row, not the
newest — the one stale shop is the one that wastes somebody's trip.
- A shop can be truncated. `total` exceeding `count` means the shop holds more
than the shard publishes per frame; the vendor page says "showing 250 of 3,104"
rather than presenting a partial shop as complete.
- An item may have no name. On a shard with no cliloc table the honest render is
the item id, never an invented label.
## The pre-wired visibility rules, re-checked
Part A pre-wired market.ownerName and market.location before the frame existed,
and the sibling rule it pre-wired for leaderboards (`characterName`) turned out
to be INERT because projectValue matches literal JSON keys. Both market rules
were checked against the real frame this time:
- `ownerName` is a real key. Kept.
- `location` is a real key ONLY because the frame nests it. Flat map/x/y/region
would have made the rule match nothing — the same failure, one part later. It
is nested on the wire and on the read model so one rule hides the facet, the
coordinates, the region and the house together; five flat keys would be five
rules that drift apart.
- `ownerSerial` was ADDED. An admin who hides the owner's name and leaves a
serial that the leaderboards and guild boards resolve back to that same name
has not hidden anything.
Tests assert all three bite, on the stored read model AND on the raw frame —
the market's SSE stream is off by default but an admin can turn it on, and a rule
that worked on only one path is exactly the leak §3.6.1 records.
## Notable
- **No payload column on shard_vendors**, unlike shard_points_boards next door.
The board's top-N is a fixed-size list read whole; here the items ARE the
searchable rows, so they are normalized and nothing is left worth duplicating.
- **display_name is denormalized at ingest** (literal name preferred over the
cliloc — a player set it, so it is more specific). Resolving at query time
would put the cliloc table on the hot path and make search-by-name impossible.
Because the shard's diff sweep will not re-send an unchanged shop just because
the site learned what its items are called, a cliloc import now triggers a bulk
re-resolution — 50 ms per thousand rows, never throws.
- **updated_at is written explicitly** on every upsert. MariaDB does not fire ON
UPDATE CURRENT_TIMESTAMP when every column is written back unchanged, and a
shop re-published identically is still freshly confirmed — without this the
staleness banner would age a perfectly current shop forever.
- **LIKE wildcards in `q` are escaped.** `%` and `_` are LIKE metacharacters, not
SQL ones, so parameterization does not neutralize them: `?q=%` would otherwise
match every listing on the shard.
- **Rate-limited** (60/min/IP), the only limited public read. Every other public
GET is an indexed lookup of bounded size; this is a LIKE scan plus a COUNT over
the largest shard_* table, anonymous by default.
- Reconnect backfill pages /market, bounded by MARKET_SNAPSHOT_MAX = 5000 and
stopping on a short page as well as on `total`, so a concurrent sweep shrinking
the index cannot spin the walk.
## How it was tested
673 server tests pass (27 new). Client builds clean; swagger-output.json,
routes.manifest.json and routes.guards.json regenerated.
Verified full-stack against the live MariaDB and a real shard, not only units:
- 27 real vendors / 1,040 listings swept off the ServUO tree, through the Rust
sidecar, into the site — names resolving through the cliloc table ("longsword",
"katana"), real facets and regions in the filters.
- `?q=sword` 682, `?q=%` and `?q=_` **0** (the escape), map/region/price/sort
filters, paging, and the vendor detail route.
- Visibility live: fields gated to staff vanish for an anonymous caller while
shopName and price survive; audience=player 403s; enabled=0 404s; and
/shard/features correctly drops `market` so the nav hides it.
- Re-publishing a shop smaller leaves no orphan items; an identical re-publish
moves updated_at.
- The limiter fires (38x200 then 32x429 on a 70-request burst).
Not covered by an automated test: the two React pages are presentational and this
repo's client suite covers pure-logic modules only. They were driven against the
live API above, but not rendered in a DOM harness.
Co-Authored-By: Claude <noreply@anthropic.com>
160 lines
5.8 KiB
JavaScript
160 lines
5.8 KiB
JavaScript
const rateLimit = require('express-rate-limit')
|
|
|
|
const log = require('../utils/logger')('ratelimit')
|
|
|
|
function makeLimiter({ windowMs, max, label, message, keyGenerator, validate }) {
|
|
return rateLimit({
|
|
windowMs,
|
|
max,
|
|
standardHeaders: true,
|
|
legacyHeaders: false,
|
|
message: { message },
|
|
// Default key is the client IP; callers can widen it (e.g. IP + provider).
|
|
...(keyGenerator ? { keyGenerator } : {}),
|
|
// Custom keyGenerators that fold in req.ip trip v7's IPv6 fallback validator;
|
|
// callers pass `validate` to scope that off just for their limiter.
|
|
...(validate !== undefined ? { validate } : {}),
|
|
handler: (req, res, next, options) => {
|
|
log.warn(`${label} rate limit exceeded`, { ip: req.ip, path: req.originalUrl })
|
|
res.status(options.statusCode).json(options.message)
|
|
},
|
|
})
|
|
}
|
|
|
|
// Brute-force protection on login.
|
|
const loginLimiter = makeLimiter({
|
|
windowMs: 15 * 60 * 1000,
|
|
max: 10,
|
|
label: 'login',
|
|
message: 'Too many login attempts. Please try again later.',
|
|
})
|
|
|
|
// Public self-registration. Mirrors the login cap: a handful of legitimate
|
|
// attempts per window, a flood is abuse. The global botScore guard + honeypot
|
|
// cover the rest.
|
|
const registerLimiter = makeLimiter({
|
|
windowMs: 15 * 60 * 1000,
|
|
max: 10,
|
|
label: 'register',
|
|
message: 'Too many registration attempts. Please try again later.',
|
|
})
|
|
|
|
// Authenticated self-service credential changes (username / password). Tighter
|
|
// than login — a signed-in player rarely changes these, and the wrong-current-
|
|
// password path also feeds the shared login backoff (see the controller).
|
|
const accountChangeLimiter = makeLimiter({
|
|
windowMs: 15 * 60 * 1000,
|
|
max: 10,
|
|
label: 'account-change',
|
|
message: 'Too many changes. Please try again later.',
|
|
})
|
|
|
|
// Throttle the public contact form.
|
|
const contactLimiter = makeLimiter({
|
|
windowMs: 60 * 60 * 1000,
|
|
max: 5,
|
|
label: 'contact',
|
|
message: 'Too many messages sent. Please try again later.',
|
|
})
|
|
|
|
// Cap mobile refresh-token exchanges per IP. Legitimate apps refresh at most a
|
|
// handful of times per window; a flood is either a bug or an attempt to brute
|
|
// the refresh endpoint.
|
|
const mobileRefreshLimiter = makeLimiter({
|
|
windowMs: 15 * 60 * 1000,
|
|
max: 30,
|
|
label: 'mobile-refresh',
|
|
message: 'Too many refresh attempts. Please try again later.',
|
|
})
|
|
|
|
// Throttle SSO redirect starts per IP — cheap to trigger, and a flood is either a
|
|
// bug or an attempt to spin the OAuth flow. Generous enough for real users.
|
|
const ssoStartLimiter = makeLimiter({
|
|
windowMs: 15 * 60 * 1000,
|
|
max: 30,
|
|
label: 'sso-start',
|
|
message: 'Too many sign-in attempts. Please try again later.',
|
|
})
|
|
|
|
// Mobile SSO bridge — throttle /start per IP AND per provider: each call spawns a
|
|
// mobile_auth_sessions row, so without a per-provider dimension /start is a cheap
|
|
// way to spam rows for one provider from many-but-few IPs. Generous for real users
|
|
// (a login is a handful of taps). `validate:{ip:false}` scopes off v7's IPv6
|
|
// fallback check, which fires only because our key folds in req.ip.
|
|
const mobileSsoStartLimiter = makeLimiter({
|
|
windowMs: 15 * 60 * 1000,
|
|
max: 20,
|
|
label: 'mobile-sso-start',
|
|
message: 'Too many sign-in attempts. Please try again later.',
|
|
keyGenerator: (req) => `${req.ip}:${req.query && req.query.provider ? req.query.provider : ''}`,
|
|
validate: { ip: false },
|
|
})
|
|
|
|
// Mobile SSO bridge — throttle /exchange per IP. The code is single-use, PKCE-bound
|
|
// and short-lived, but cap redemption attempts anyway to blunt guessing.
|
|
const mobileSsoExchangeLimiter = makeLimiter({
|
|
windowMs: 15 * 60 * 1000,
|
|
max: 30,
|
|
label: 'mobile-sso-exchange',
|
|
message: 'Too many attempts. Please try again later.',
|
|
})
|
|
|
|
// Password-reset requests per IP. Each one can send email, so cap tighter than
|
|
// login to blunt email-bombing and enumeration timing probes. The endpoint always
|
|
// returns a generic success regardless of match, so honest users never see this.
|
|
const passwordResetRequestLimiter = makeLimiter({
|
|
windowMs: 60 * 60 * 1000,
|
|
max: 5,
|
|
label: 'password-reset-request',
|
|
message: 'Too many reset requests. Please try again later.',
|
|
})
|
|
|
|
// Reset confirmations (token + new password) per IP. A wrong/expired token is a
|
|
// guessing surface; the token itself is 256-bit random, but cap anyway.
|
|
const passwordResetConfirmLimiter = makeLimiter({
|
|
windowMs: 15 * 60 * 1000,
|
|
max: 15,
|
|
label: 'password-reset-confirm',
|
|
message: 'Too many attempts. Please try again later.',
|
|
})
|
|
|
|
// The player-vendor market search. The first genuinely expensive PUBLIC endpoint
|
|
// on the site: every call is a LIKE scan plus a COUNT over the listings table,
|
|
// which on a large shard is the biggest table there is, and it is anonymous by
|
|
// default. Generous for a human browsing shops (a typed search is debounced to
|
|
// one request, and paging is a click), tight enough that it cannot be used as a
|
|
// cheap way to load the database.
|
|
const marketLimiter = makeLimiter({
|
|
windowMs: 60 * 1000,
|
|
max: 60,
|
|
label: 'market',
|
|
message: 'Too many searches. Please slow down.',
|
|
})
|
|
|
|
// CSP violation reports. Unauthenticated by necessity (browsers send them with no
|
|
// session), and every accepted report writes a log line — so an attacker who can get
|
|
// a victim to load a page could otherwise use it as a log-flood amplifier. Generous
|
|
// enough for the real case: a genuinely broken directive fires a handful of times per
|
|
// page load, and browsers already de-duplicate identical violations per document.
|
|
const cspReportLimiter = makeLimiter({
|
|
windowMs: 5 * 60 * 1000,
|
|
max: 60,
|
|
label: 'csp-report',
|
|
message: 'Too many reports.',
|
|
})
|
|
|
|
module.exports = {
|
|
loginLimiter,
|
|
registerLimiter,
|
|
accountChangeLimiter,
|
|
contactLimiter,
|
|
mobileRefreshLimiter,
|
|
ssoStartLimiter,
|
|
mobileSsoStartLimiter,
|
|
mobileSsoExchangeLimiter,
|
|
passwordResetRequestLimiter,
|
|
passwordResetConfirmLimiter,
|
|
marketLimiter,
|
|
cspReportLimiter,
|
|
}
|