feat(modules): ctx additions and the post-hook registry (API 1.1.0)
Everything the extraction needed from core that ctx did not already offer. Additions only, so minor. ctx.activity.log, because an admin action a module performs has to land in core's one audit log or the trail has a hole exactly where a module operates the game -- a module keeping its own log would be a second place to look, which in practice means a place nobody looks. Write-only; reading the log is the admin panel's job and it spans every actor. ctx.users.getById, one function for one caller: the admin.users.detail slot router needs the user its prefix names. ctx.site.baseUrl, because a module has to build absolute links and §2.7 forbids it reading core's APP_BASE_URL -- a getter, not a captured string, so it cannot go stale against the env. ctx.middleware.rateLimit is core's makeLimiter, plus accountChangeLimiter handed over whole. The split is deliberate: a module states its own window and cap because it knows what its endpoints cost, and takes the plumbing from core so there is one express-rate-limit in the process and one place a breach is logged. accountChangeLimiter is shared policy -- core's /auth/me and /player/account sit behind the same counter -- so a module's account-change route has to land IN it rather than beside it. marketLimiter was UO policy living in core's file and leaves with the route it guards. registerPostHook is the fourth registry, and the last thing binding core to the module. Core's post controller called newsGump.syncPost directly: core's CMS naming a UO file. It now publishes what it already knows and a subscriber decides what to do with it. Not folded into registerAnnounceLeg, which fires on the same transition, because a leg is a one-shot DELIVERY with retry and classification while a post hook maintains idempotent STATE, runs on delete as well as save, and refreshes silently on an edit. Also fixes a real loader defect the extraction exposed: schema table names were matched against the RAW file, so a fragment whose header says "every CREATE TABLE carries IF NOT EXISTS" was rejected for a prefix violation on a table called `carries`. module-uo's fragment hit exactly that. Both scans now read split statements, which strip comments -- the same class of bug as a boundary check failing on its own documentation. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -1,287 +0,0 @@
|
||||
// Cliloc parsing — the pure half.
|
||||
//
|
||||
// A "cliloc" is UO's localization table: an integer id mapped to a display
|
||||
// string. Items carry a `LabelNumber` rather than a name, so without this table
|
||||
// the site can only render `id 1023721` where the game shows "quarter staff".
|
||||
// The shard already sends the id on every equipment entry (`char.profile`'s
|
||||
// `cliloc` field) and will send one per marketplace listing — the *number* was
|
||||
// never the missing piece, the *table* was.
|
||||
//
|
||||
// This module is fs-free on purpose, exactly like `spawnAtlasParse.js`: the
|
||||
// suite runs in CI where there is no UO client, so every parser here is driven
|
||||
// from inline fixtures. `clilocSource.js` is the only thing that touches disk.
|
||||
//
|
||||
// ── Two input formats, and why ─────────────────────────────────────────────
|
||||
//
|
||||
// The client's own `Cliloc.enu` is COMPRESSED (Mythic format) on any modern
|
||||
// client, and decompressing it is a bit-level port of an inverse-BWT coder that
|
||||
// nothing in this stack needs at runtime. ServUO's own bundled `Ultima.StringList`
|
||||
// cannot read it either — which is why `VendorSearch.GetItemName` is already inert
|
||||
// on such a shard and the plugin could not supply names even if we asked it to.
|
||||
//
|
||||
// So the operator converts once, from their own client, and points the site at
|
||||
// the result (see docs/website/CLILOCS.md). Two shapes are accepted because
|
||||
// different tools produce different things:
|
||||
//
|
||||
// • PLAIN BINARY — the pre-compression cliloc layout: a 6-byte header, then
|
||||
// records of {int32 number, byte flag, uint16 length, UTF-8 bytes}.
|
||||
// • DELIMITED TEXT — `number<TAB|,|;>text` per line, which is what the common
|
||||
// GUI exports emit. Quoted CSV fields and a header row are tolerated.
|
||||
//
|
||||
// Nothing derived from the client is ever committed: the converted file lives at
|
||||
// an operator-supplied path and is gitignored, the same rule the spawn atlas art
|
||||
// map already follows.
|
||||
|
||||
/** Raised for a file we can identify but deliberately refuse to guess at. */
|
||||
class ClilocFormatError extends Error {
|
||||
constructor(message, code) {
|
||||
super(message)
|
||||
this.name = 'ClilocFormatError'
|
||||
this.code = code
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Bumped when this parser produces DIFFERENT data from an IDENTICAL source file.
|
||||
*
|
||||
* Stored beside the source hash so the boot path can tell "same file, but the
|
||||
* parser moved on" from "same file, nothing to do". Without it a corrected parse
|
||||
* would ship and never reach an install whose cliloc file never changes — the
|
||||
* trap `spawnAtlasSource.PARSER_VERSION` documents.
|
||||
*/
|
||||
const PARSER_VERSION = 1
|
||||
|
||||
// The plain layout's header is `02 00 00 00 01 00` — a 4-byte version and a
|
||||
// 2-byte language marker. Only the size matters for parsing; the values are
|
||||
// checked to sniff the format, not to validate it.
|
||||
const HEADER_BYTES = 6
|
||||
const RECORD_HEADER_BYTES = 7 // int32 number + byte flag + uint16 length
|
||||
|
||||
// Every compressed cliloc file the client ships begins with a DWORD whose high
|
||||
// byte is 0x8E (the XOR key UOFiddler calls `HeaderXorKey`, 0x8E2C9A3D). That is
|
||||
// the single cheapest way to tell an operator they exported the wrong file —
|
||||
// without it, the plain parser happily reads compressed bytes as ~19k records of
|
||||
// negative ids and 60 KB "strings" before dying somewhere in the middle, and the
|
||||
// resulting error names the wrong problem.
|
||||
const MYTHIC_HIGH_BYTE = 0x8e
|
||||
|
||||
/** True when `buffer` is a Mythic-compressed cliloc rather than the plain layout. */
|
||||
function isCompressedCliloc(buffer) {
|
||||
return buffer.length >= 4 && buffer[3] === MYTHIC_HIGH_BYTE
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the plain binary cliloc layout.
|
||||
*
|
||||
* Strict about truncation, and that strictness is load-bearing: a half-copied or
|
||||
* partly-written file is the realistic failure here, and it must fail loudly
|
||||
* rather than import a silently short table that then renders half the world as
|
||||
* `id 1023721`. A record that runs past the end of the buffer throws.
|
||||
*/
|
||||
function parseClilocBinary(buffer) {
|
||||
if (!Buffer.isBuffer(buffer)) throw new ClilocFormatError('Not a buffer', 'NOT_BUFFER')
|
||||
if (isCompressedCliloc(buffer)) {
|
||||
throw new ClilocFormatError(
|
||||
'This is a compressed (Mythic-format) cliloc file, which the site cannot read. ' +
|
||||
'Convert it to the plain format first — see docs/website/CLILOCS.md.',
|
||||
'COMPRESSED',
|
||||
)
|
||||
}
|
||||
if (buffer.length < HEADER_BYTES) {
|
||||
throw new ClilocFormatError('File is shorter than a cliloc header', 'TRUNCATED')
|
||||
}
|
||||
|
||||
const entries = []
|
||||
let offset = HEADER_BYTES
|
||||
|
||||
while (offset < buffer.length) {
|
||||
if (offset + RECORD_HEADER_BYTES > buffer.length) {
|
||||
throw new ClilocFormatError(
|
||||
`Truncated record header at byte ${offset} (${entries.length} entries read)`,
|
||||
'TRUNCATED',
|
||||
)
|
||||
}
|
||||
const number = buffer.readInt32LE(offset)
|
||||
const flag = buffer.readUInt8(offset + 4)
|
||||
// The length is written by the client as an unsigned 16-bit value. Reading it
|
||||
// signed (as ServUO's own SDK does) turns any string over 32 KB into a
|
||||
// negative length; real tables top out around 12 KB, so this has no effect on
|
||||
// current data and costs nothing to get right.
|
||||
const length = buffer.readUInt16LE(offset + 5)
|
||||
offset += RECORD_HEADER_BYTES
|
||||
|
||||
if (offset + length > buffer.length) {
|
||||
throw new ClilocFormatError(
|
||||
`Truncated record body at byte ${offset} (${entries.length} entries read)`,
|
||||
'TRUNCATED',
|
||||
)
|
||||
}
|
||||
entries.push({ number, flag, text: buffer.toString('utf8', offset, offset + length) })
|
||||
offset += length
|
||||
}
|
||||
|
||||
return entries
|
||||
}
|
||||
|
||||
// A delimited line splits on the FIRST separator only: cliloc text is full of
|
||||
// commas ("a scroll of magery, unfinished") and splitting on all of them would
|
||||
// truncate every such entry at its first comma.
|
||||
const TEXT_SEPARATORS = ['\t', ',', ';']
|
||||
|
||||
/** Unwrap one CSV field: strip surrounding quotes and unescape doubled quotes. */
|
||||
function unquote(value) {
|
||||
const trimmed = value.trim()
|
||||
if (trimmed.length >= 2 && trimmed.startsWith('"') && trimmed.endsWith('"')) {
|
||||
return trimmed.slice(1, -1).replace(/""/g, '"')
|
||||
}
|
||||
return trimmed
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a delimited text export: `number<sep>text` per line.
|
||||
*
|
||||
* Tolerant by design — this is whatever an operator's GUI tool produced, not a
|
||||
* format we control. A header row, blank lines, `#` comments and a trailing
|
||||
* flags column are all ignored. A line whose first field is not an integer is
|
||||
* skipped rather than fatal, because that is exactly what a header row is.
|
||||
*
|
||||
* The one thing it will NOT do is return an empty table quietly: a file that
|
||||
* yields no entries at all is a wrong file, not an empty one.
|
||||
*/
|
||||
function parseClilocText(text) {
|
||||
const entries = []
|
||||
for (const line of String(text).split(/\r?\n/)) {
|
||||
// The line is deliberately NOT trimmed before the separator search. Roughly
|
||||
// half of a real cliloc table is empty strings (unused ids), which export as
|
||||
// `1005008<TAB>` — and trimming eats that trailing separator, leaving a bare
|
||||
// number that then looks like a header row and is skipped. That silently
|
||||
// dropped 55,994 of 123,490 entries. Individual FIELDS are trimmed instead,
|
||||
// by `unquote`.
|
||||
if (line.trim() === '' || line.trimStart().startsWith('#')) continue
|
||||
|
||||
// Pick the separator that actually appears first, so a tab-delimited line
|
||||
// whose text contains a comma still splits on the tab.
|
||||
let cut = -1
|
||||
for (const sep of TEXT_SEPARATORS) {
|
||||
const at = line.indexOf(sep)
|
||||
if (at !== -1 && (cut === -1 || at < cut)) cut = at
|
||||
}
|
||||
if (cut === -1) continue
|
||||
|
||||
// An EMPTY first field must not become id 0: `Number('')` is 0, not NaN, so
|
||||
// a line that merely starts with a separator would otherwise import as a
|
||||
// bogus cliloc 0 instead of being skipped.
|
||||
const head = unquote(line.slice(0, cut))
|
||||
if (head === '') continue
|
||||
const number = Number(head)
|
||||
if (!Number.isInteger(number)) continue // header row, or a wrapped line
|
||||
|
||||
let rest = line.slice(cut + 1)
|
||||
// Some exports carry `number,flag,text`. A bare integer in the second field
|
||||
// is a flag; anything else is the text itself (and a text field that IS just
|
||||
// a number is indistinguishable, so it stays as the text — the safer miss).
|
||||
let flag = 0
|
||||
for (const sep of TEXT_SEPARATORS) {
|
||||
const at = rest.indexOf(sep)
|
||||
if (at === -1) continue
|
||||
const head = unquote(rest.slice(0, at))
|
||||
if (/^\d{1,3}$/.test(head) && rest.slice(at + 1).trim() !== '') {
|
||||
flag = Number(head)
|
||||
rest = rest.slice(at + 1)
|
||||
}
|
||||
break
|
||||
}
|
||||
|
||||
entries.push({ number, flag, text: unquote(rest) })
|
||||
}
|
||||
|
||||
if (entries.length === 0) {
|
||||
throw new ClilocFormatError('No cliloc entries found in the text export', 'EMPTY')
|
||||
}
|
||||
return entries
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse either supported shape, sniffing which one this is.
|
||||
*
|
||||
* The sniff is on the binary header rather than the file extension: operators
|
||||
* name these things whatever they like, and an `.enu` that is really a TSV (or a
|
||||
* `.txt` that is really binary) should still import.
|
||||
*/
|
||||
function parseCliloc(buffer) {
|
||||
const buf = Buffer.isBuffer(buffer) ? buffer : Buffer.from(buffer)
|
||||
|
||||
if (isCompressedCliloc(buf)) {
|
||||
throw new ClilocFormatError(
|
||||
'This is a compressed (Mythic-format) cliloc file, which the site cannot read. ' +
|
||||
'Convert it to the plain format first — see docs/website/CLILOCS.md.',
|
||||
'COMPRESSED',
|
||||
)
|
||||
}
|
||||
|
||||
// The plain layout always opens with version 2 / language 1. Anything else is
|
||||
// treated as text, which is the recoverable guess: a mis-sniffed text file
|
||||
// yields "no entries found", while a mis-sniffed binary yields nonsense.
|
||||
if (buf.length >= HEADER_BYTES && buf.readInt32LE(0) === 2 && buf.readUInt16LE(4) === 1) {
|
||||
return parseClilocBinary(buf)
|
||||
}
|
||||
return parseClilocText(buf.toString('utf8'))
|
||||
}
|
||||
|
||||
// ── Display ────────────────────────────────────────────────────────────────
|
||||
|
||||
// Cliloc strings interpolate arguments the client supplies out of an item's
|
||||
// property list: `~1_val~`, `~2_NAME~`, `~1_ITEM~`. We never have those — the
|
||||
// bridge sends the id, not the packet — so a name carrying them must be reduced
|
||||
// to what is actually knowable rather than shown with the raw tokens in it.
|
||||
const PLACEHOLDER_RE = /~\d+_[^~]*~/g
|
||||
|
||||
/**
|
||||
* Reduce a raw cliloc string to something displayable.
|
||||
*
|
||||
* Placeholders are dropped and the leftover punctuation tidied, so
|
||||
* `"[~1_stuff~]"` becomes `""` (correctly nothing — the whole string was the
|
||||
* argument) and `"cold damage ~1_val~%"` becomes `"cold damage"`.
|
||||
*
|
||||
* **Punctuation is only tidied when a placeholder was actually removed.** The
|
||||
* trailing `%` above is the unit belonging to the number we never had, and the
|
||||
* brackets in `[~1_stuff~]` only ever wrapped the argument — but a string with
|
||||
* no placeholder has no such debris, and trimming it anyway corrupts real names.
|
||||
* A shard's `"Runic Gateway Sigil (v2)"` came back as `"(v2"` while this was
|
||||
* unconditional.
|
||||
*
|
||||
* Returns `''` when nothing survives, which callers treat as "no name" and fall
|
||||
* back to the item id — better than showing a bracket.
|
||||
*/
|
||||
const DEBRIS = /^[\s\-–—,.;:%[\]()]+|[\s\-–—,.;:%[\]()]+$/g
|
||||
|
||||
function displayText(raw) {
|
||||
if (raw == null) return ''
|
||||
const source = String(raw)
|
||||
const hadPlaceholder = PLACEHOLDER_RE.test(source)
|
||||
PLACEHOLDER_RE.lastIndex = 0 // the regex is global; `test` advances it
|
||||
|
||||
if (!hadPlaceholder) return source.replace(/\s+/g, ' ').trim()
|
||||
|
||||
return source
|
||||
.replace(PLACEHOLDER_RE, ' ')
|
||||
.replace(/\s+/g, ' ')
|
||||
.replace(/\s+([,.;:!?])/g, '$1')
|
||||
.replace(DEBRIS, '')
|
||||
.trim()
|
||||
}
|
||||
|
||||
/** True when a raw cliloc string is nothing but interpolated arguments. */
|
||||
const isPlaceholderOnly = (raw) => raw != null && String(raw).trim() !== '' && displayText(raw) === ''
|
||||
|
||||
module.exports = {
|
||||
ClilocFormatError,
|
||||
PARSER_VERSION,
|
||||
HEADER_BYTES,
|
||||
isCompressedCliloc,
|
||||
parseCliloc,
|
||||
parseClilocBinary,
|
||||
parseClilocText,
|
||||
displayText,
|
||||
isPlaceholderOnly,
|
||||
}
|
||||
@@ -1,316 +0,0 @@
|
||||
// Cliloc table — the filesystem layer.
|
||||
//
|
||||
// `clilocParse.js` holds the pure parsers; this module is the only thing that
|
||||
// touches cliloc files on disk, and it is shared by both callers:
|
||||
//
|
||||
// - the server, which refreshes the table on boot (`shardClilocs.model.js`)
|
||||
// - the admin panel, which can force a reimport without a restart
|
||||
//
|
||||
// The files are the OPERATOR'S (see docs/website/CLILOCS.md). Nothing derived
|
||||
// from them is committed: the repo holds no string table, exactly as it holds no
|
||||
// map snapshot and no artwork. That rule is why this module reads a configured
|
||||
// path instead of a path inside the repo.
|
||||
//
|
||||
// ── Why this reads a SET of files, not one ────────────────────────────────
|
||||
//
|
||||
// Shards edit items and add new ones. Those carry cliloc ids that a stock client
|
||||
// table does not have — and forcing a 5 MB client re-export every time an
|
||||
// operator adds one item would be miserable enough that the table would simply
|
||||
// go stale, which is the exact failure the spawn atlas was redesigned to avoid.
|
||||
//
|
||||
// So this mirrors `spawnAtlasSource.readSources()`: a BASE table (the converted
|
||||
// client file) plus every operator-maintained OVERLAY beside it, all re-read on
|
||||
// every boot and hash-gated as a SET. Adding, editing or removing any overlay
|
||||
// counts as drift and re-imports. Later sources win, so an overlay both adds new
|
||||
// ids and overrides stock ones.
|
||||
//
|
||||
// Measured on a real shard: the script tree references 16,434 cliloc ids and only
|
||||
// 37 are absent from the stock client table. Tens of entries against a 67k base
|
||||
// is what makes the overlay the right shape rather than a second full table.
|
||||
//
|
||||
// Reading and hashing ~5 MB costs a few milliseconds and a full parse ~50 ms, so
|
||||
// the boot path hashes first and only parses when something actually changed.
|
||||
|
||||
const crypto = require('crypto')
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
|
||||
const { ClilocFormatError, PARSER_VERSION, parseCliloc, isCompressedCliloc } = require('./clilocParse')
|
||||
|
||||
/**
|
||||
* Filenames looked for as the BASE table when the configured path is a directory.
|
||||
*
|
||||
* Ordered by how specific they are: an explicitly converted file wins over
|
||||
* something that merely sits in a client folder, so an operator who dropped a
|
||||
* `cliloc.plain.enu` next to the original compressed `cliloc.enu` gets the one
|
||||
* they made rather than the one that will be rejected.
|
||||
*
|
||||
* Matching is case-insensitive against the real directory listing, because the
|
||||
* client ships `Cliloc.enu` on Windows and the site usually runs on Linux, where
|
||||
* a hardcoded lowercase open would simply miss.
|
||||
*/
|
||||
const CANDIDATE_NAMES = [
|
||||
'clilocs.tsv',
|
||||
'clilocs.csv',
|
||||
'clilocs.plain',
|
||||
'cliloc.plain',
|
||||
'cliloc.plain.enu',
|
||||
'cliloc.enu.plain',
|
||||
'clilocs.txt',
|
||||
'cliloc.enu',
|
||||
]
|
||||
|
||||
/**
|
||||
* Where shard-specific additions and overrides live: a `custom/` directory
|
||||
* beside the base table.
|
||||
*
|
||||
* ServUO has **no server-side convention** for custom clilocs — they live in the
|
||||
* patched client file a shard distributes to its players, and nothing in the
|
||||
* tree declares them. There is therefore nothing to discover, and this is the
|
||||
* one place in the cliloc pipeline that is a convention we chose rather than one
|
||||
* the shard already has. It is a directory rather than a single file so an
|
||||
* operator can keep additions grouped however they like (per system, per patch)
|
||||
* without the site caring.
|
||||
*/
|
||||
const CUSTOM_DIR = 'custom'
|
||||
const CUSTOM_EXTENSIONS = ['.tsv', '.csv', '.txt', '.enu', '.plain']
|
||||
|
||||
class ClilocSourceError extends Error {
|
||||
constructor(message, code) {
|
||||
super(message)
|
||||
this.name = 'ClilocSourceError'
|
||||
this.code = code
|
||||
}
|
||||
}
|
||||
|
||||
function sha256(buffer) {
|
||||
return crypto.createHash('sha256').update(buffer).digest('hex')
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the configured path to `{ root, base }`.
|
||||
*
|
||||
* Accepts either a direct file path or a directory to search, because operators
|
||||
* reasonably supply both — "here is the file" and "here is the folder I put it
|
||||
* in" are equally natural answers to the admin panel's prompt. When it is a
|
||||
* file, `root` is the directory CONTAINING it, so overlays work either way: an
|
||||
* operator who pointed at a file should not have to re-point at its folder just
|
||||
* to add a `custom/` directory next to it.
|
||||
*/
|
||||
function resolveBase(configured) {
|
||||
if (!configured || String(configured).trim() === '') {
|
||||
throw new ClilocSourceError('No cliloc path configured', 'NO_PATH')
|
||||
}
|
||||
const target = String(configured).trim()
|
||||
|
||||
let stat
|
||||
try {
|
||||
stat = fs.statSync(target)
|
||||
} catch {
|
||||
throw new ClilocSourceError(`Cliloc path does not exist: ${target}`, 'NOT_FOUND')
|
||||
}
|
||||
|
||||
if (stat.isFile()) return { root: path.dirname(target), base: target }
|
||||
|
||||
if (!stat.isDirectory()) {
|
||||
throw new ClilocSourceError(`Cliloc path is neither a file nor a directory: ${target}`, 'NOT_FOUND')
|
||||
}
|
||||
|
||||
let listing
|
||||
try {
|
||||
listing = fs.readdirSync(target)
|
||||
} catch {
|
||||
throw new ClilocSourceError(`Cliloc directory is not readable: ${target}`, 'NOT_FOUND')
|
||||
}
|
||||
|
||||
const byLower = new Map(listing.map((name) => [name.toLowerCase(), name]))
|
||||
for (const candidate of CANDIDATE_NAMES) {
|
||||
const actual = byLower.get(candidate)
|
||||
if (actual) return { root: target, base: path.join(target, actual) }
|
||||
}
|
||||
|
||||
throw new ClilocSourceError(
|
||||
`No cliloc file found in ${target} (looked for ${CANDIDATE_NAMES.join(', ')})`,
|
||||
'NO_FILE',
|
||||
)
|
||||
}
|
||||
|
||||
/** Overlay files under `<root>/custom/`, sorted so precedence is deterministic. */
|
||||
function listCustom(root) {
|
||||
const dir = path.join(root, CUSTOM_DIR)
|
||||
let listing
|
||||
try {
|
||||
listing = fs.readdirSync(dir, { withFileTypes: true })
|
||||
} catch (err) {
|
||||
// No overlay directory is the normal case, not an error.
|
||||
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') return []
|
||||
throw new ClilocSourceError(`Cliloc overlay directory is not readable: ${dir}`, 'UNREADABLE')
|
||||
}
|
||||
return listing
|
||||
.filter((e) => e.isFile() && CUSTOM_EXTENSIONS.includes(path.extname(e.name).toLowerCase()))
|
||||
.map((e) => e.name)
|
||||
.sort()
|
||||
.map((name) => path.join(dir, name))
|
||||
}
|
||||
|
||||
function readFileOrThrow(file) {
|
||||
try {
|
||||
return fs.readFileSync(file)
|
||||
} catch {
|
||||
throw new ClilocSourceError(`Cliloc file is not readable: ${file}`, 'UNREADABLE')
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read every cliloc source under the configured path.
|
||||
*
|
||||
* Returns `{ root, files: [{ label, kind, file, buffer, sha256, bytes, compressed }] }`
|
||||
* with the base first and overlays after, in the order they must be merged.
|
||||
*
|
||||
* Labels are root-relative and forward-slashed so a hash map compares equal
|
||||
* across platforms — the same directory read on Windows and Linux must produce
|
||||
* the same fingerprint, or every boot would look like a change. (The same
|
||||
* reasoning, and the same bug, as `spawnAtlasSource.readSources`.)
|
||||
*/
|
||||
function readSources(configured) {
|
||||
const { root, base } = resolveBase(configured)
|
||||
|
||||
const describe = (file, kind) => {
|
||||
const buffer = readFileOrThrow(file)
|
||||
return {
|
||||
label: path.relative(root, file).split(path.sep).join('/'),
|
||||
kind,
|
||||
file,
|
||||
buffer,
|
||||
sha256: sha256(buffer),
|
||||
bytes: buffer.length,
|
||||
compressed: isCompressedCliloc(buffer),
|
||||
}
|
||||
}
|
||||
|
||||
const files = [describe(base, 'base')]
|
||||
for (const overlay of listCustom(root)) files.push(describe(overlay, 'custom'))
|
||||
|
||||
return { root, files }
|
||||
}
|
||||
|
||||
/**
|
||||
* A fingerprint of every source: `{ "<label>": "<sha256>" }`, plus the base's
|
||||
* details for the admin panel.
|
||||
*
|
||||
* `compressed` is reported here rather than left to the parse because the admin
|
||||
* panel calls this and NOT `readCliloc` (parsing 5 MB on every status poll would
|
||||
* be wasteful). Without it, pointing the setting at an unconverted client
|
||||
* directory reports a perfectly readable file with pending drift — "ready to
|
||||
* import" — and the operator only learns otherwise when the import fails. The
|
||||
* check is four bytes of a buffer already in hand.
|
||||
*/
|
||||
function hashSources(configured) {
|
||||
const { root, files } = readSources(configured)
|
||||
const hashes = {}
|
||||
for (const file of files) hashes[file.label] = file.sha256
|
||||
const base = files[0]
|
||||
return {
|
||||
root,
|
||||
hashes,
|
||||
file: base.file,
|
||||
bytes: base.bytes,
|
||||
compressed: files.some((f) => f.compressed),
|
||||
customCount: files.length - 1,
|
||||
}
|
||||
}
|
||||
|
||||
/** True when two source fingerprints describe the same set of files. */
|
||||
function sameSources(a, b) {
|
||||
if (!a || !b) return false
|
||||
const aKeys = Object.keys(a).sort()
|
||||
const bKeys = Object.keys(b).sort()
|
||||
if (aKeys.length !== bKeys.length) return false
|
||||
return aKeys.every((key, i) => key === bKeys[i] && a[key] === b[key])
|
||||
}
|
||||
|
||||
/**
|
||||
* Labels present in `loaded` that are absent from `current`.
|
||||
*
|
||||
* This is the multi-source hazard that a single file did not have. One corrupt
|
||||
* file fails the parse loudly, but a source that has simply VANISHED — an
|
||||
* unmounted volume, a half-copied deploy — parses perfectly and imports a table
|
||||
* quietly missing everything that file contributed. That is the same ambiguity
|
||||
* the spawn atlas escalates for a disappearing facet, so it is escalated here
|
||||
* too rather than applied.
|
||||
*/
|
||||
function missingSources(current, loaded) {
|
||||
if (!loaded) return []
|
||||
return Object.keys(loaded).filter((label) => !Object.hasOwn(current, label))
|
||||
}
|
||||
|
||||
/**
|
||||
* Read and parse every source, merged into one entry list.
|
||||
*
|
||||
* Later sources win: the base client table first, then each overlay in sorted
|
||||
* order, so an overlay both ADDS ids the client never had and OVERRIDES stock
|
||||
* ones the shard has re-purposed.
|
||||
*
|
||||
* Returns `{ entries, source }`. Throws `ClilocSourceError` for anything about
|
||||
* the paths and `ClilocFormatError` for anything about the contents — different
|
||||
* problems for an operator (wrong place vs wrong file), and the admin panel says
|
||||
* which. A format error names the file it came from, because "which of my six
|
||||
* overlay files is malformed" is otherwise a guessing game.
|
||||
*/
|
||||
function readCliloc(configured) {
|
||||
const { root, files } = readSources(configured)
|
||||
|
||||
const merged = new Map()
|
||||
const perSource = []
|
||||
|
||||
for (const file of files) {
|
||||
let entries
|
||||
try {
|
||||
entries = parseCliloc(file.buffer)
|
||||
} catch (err) {
|
||||
if (err instanceof ClilocFormatError) {
|
||||
throw new ClilocFormatError(`${file.label}: ${err.message}`, err.code)
|
||||
}
|
||||
throw err
|
||||
}
|
||||
|
||||
let added = 0
|
||||
let overrode = 0
|
||||
for (const entry of entries) {
|
||||
if (!Number.isInteger(entry.number)) continue
|
||||
if (merged.has(entry.number)) overrode++
|
||||
else added++
|
||||
merged.set(entry.number, entry)
|
||||
}
|
||||
perSource.push({ label: file.label, kind: file.kind, entries: entries.length, added, overrode })
|
||||
}
|
||||
|
||||
return {
|
||||
entries: [...merged.values()],
|
||||
source: {
|
||||
root,
|
||||
file: files[0].file,
|
||||
sha256: files[0].sha256,
|
||||
bytes: files[0].bytes,
|
||||
hashes: Object.fromEntries(files.map((f) => [f.label, f.sha256])),
|
||||
parserVersion: PARSER_VERSION,
|
||||
sources: perSource,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
ClilocFormatError,
|
||||
ClilocSourceError,
|
||||
PARSER_VERSION,
|
||||
CANDIDATE_NAMES,
|
||||
CUSTOM_DIR,
|
||||
CUSTOM_EXTENSIONS,
|
||||
resolveBase,
|
||||
listCustom,
|
||||
readSources,
|
||||
hashSources,
|
||||
sameSources,
|
||||
missingSources,
|
||||
readCliloc,
|
||||
}
|
||||
@@ -1,124 +0,0 @@
|
||||
// ── Town Cryer News gump sync (Protocol 2.1) ───────────────────────────────
|
||||
//
|
||||
// Keeps the in-game Town Cryer *News* gump in sync with the site's published
|
||||
// news posts. Distinct from the scrolling town-crier lines (that's a one-shot
|
||||
// announce leg in announceWorker); this is a STATE SYNC — an article stays in the
|
||||
// gump while its post is published news, and is pulled when the post is
|
||||
// unpublished/deleted/re-categorised.
|
||||
//
|
||||
// The website is the source of truth. POST /news is idempotent (re-post replaces),
|
||||
// so a refresh or a reconnect re-assert is safe. Every call is best-effort and
|
||||
// never throws — a sidecar/shard hiccup must never break saving or deleting a
|
||||
// post. Reliability comes from reassertAll() on every WS (re)connect
|
||||
// (uoLinkSocket.backfill), which re-pushes the current published set silently and
|
||||
// closes the gap if an earlier live push failed.
|
||||
|
||||
const posts = require('../model/posts/posts.model')
|
||||
const uoLinkClient = require('./uoLinkClient')
|
||||
const settings = require('../model/settings/settings.model')
|
||||
const { deriveExcerpt } = require('./sanitizeHtml')
|
||||
const log = require('./logger')('news-gump')
|
||||
|
||||
const MAX_TITLE = 120
|
||||
const MAX_BODY = 900
|
||||
|
||||
function baseUrl() {
|
||||
return (process.env.APP_BASE_URL || 'http://localhost:5173').replace(/\/+$/, '')
|
||||
}
|
||||
|
||||
function clamp(value, max) {
|
||||
const s = String(value == null ? '' : value).replace(/\s+/g, ' ').trim()
|
||||
return s.length <= max ? s : `${s.slice(0, max - 1).trimEnd()}…`
|
||||
}
|
||||
|
||||
// A post belongs in the gump exactly when it is published AND in the news category.
|
||||
function inGump(post) {
|
||||
return Boolean(post && post.published && post.category === 'news')
|
||||
}
|
||||
|
||||
// Optional UO gump image id for news articles (a shard art id), from the
|
||||
// `news_gump_image` setting. Omitted → the sidecar uses a neutral scroll.
|
||||
async function gumpImage() {
|
||||
try {
|
||||
const raw = await settings.get('news_gump_image')
|
||||
const n = Number(raw)
|
||||
return Number.isInteger(n) && n > 0 ? n : undefined
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
|
||||
// Build the in-game News article from a post. Body is a compact gump-HTML block
|
||||
// (title centred + a plain-text excerpt) rather than the post's full rich HTML —
|
||||
// the UO gump only supports a small HTML subset, so we keep it predictable. The
|
||||
// "more info" URL is the public news list (news posts have no per-post route).
|
||||
async function buildArticle(post, { announce = true } = {}) {
|
||||
const title = clamp(post.title, MAX_TITLE)
|
||||
const excerpt = clamp(post.excerpt || deriveExcerpt(post.body, MAX_BODY) || '', MAX_BODY)
|
||||
const body = excerpt ? `<CENTER>${title}</CENTER><BR><BR>${excerpt}` : `<CENTER>${title}</CENTER>`
|
||||
return {
|
||||
id: String(post.id),
|
||||
title,
|
||||
body,
|
||||
image: await gumpImage(),
|
||||
url: `${baseUrl()}/site/news`,
|
||||
announce,
|
||||
}
|
||||
}
|
||||
|
||||
// Push a post to the gump (only if it belongs there). announce=true has the criers
|
||||
// proclaim the title; false is a silent refresh/re-assert.
|
||||
async function pushPost(post, { announce = true } = {}) {
|
||||
if (!inGump(post)) return { ok: false, skipped: true }
|
||||
const res = await uoLinkClient.postNews(await buildArticle(post, { announce }))
|
||||
if (!res.ok) log.warn('news gump push failed', { id: post.id, status: res.status, error: res.error })
|
||||
return res
|
||||
}
|
||||
|
||||
// Remove a post from the gump. A 404 (not present) is not an error worth noting.
|
||||
async function removePost(id) {
|
||||
const res = await uoLinkClient.deleteNews(String(id))
|
||||
if (!res.ok && res.status !== 404) {
|
||||
log.warn('news gump remove failed', { id, status: res.status, error: res.error })
|
||||
}
|
||||
return res
|
||||
}
|
||||
|
||||
// Reconcile the gump after a post create/update/publish. `transition`
|
||||
// ({ wasPublished, wasNews }) tells a fresh publish (announce) from an in-place
|
||||
// edit (silent refresh) and catches a post leaving published-news (pull it).
|
||||
async function syncPost(post, transition = {}) {
|
||||
try {
|
||||
if (inGump(post)) {
|
||||
const wasInGump = Boolean(transition.wasPublished && transition.wasNews)
|
||||
await pushPost(post, { announce: !wasInGump })
|
||||
} else if (transition.wasPublished && transition.wasNews) {
|
||||
await removePost(post.id)
|
||||
}
|
||||
} catch (err) {
|
||||
log.warn('news gump sync failed', { id: post && post.id, message: err.message })
|
||||
}
|
||||
}
|
||||
|
||||
// Re-push every currently-published news post, silently — run on each WS
|
||||
// (re)connect to reconcile the gump to our source of truth (also recovers any
|
||||
// article whose original live push failed). Best-effort; never throws.
|
||||
async function reassertAll() {
|
||||
try {
|
||||
const list = await posts.listAll('news')
|
||||
const published = (list || []).filter((p) => p.published)
|
||||
let pushed = 0
|
||||
for (const p of published) {
|
||||
const full = await posts.getById(p.id) // list projection may omit the body
|
||||
if (full) {
|
||||
await pushPost(full, { announce: false })
|
||||
pushed += 1
|
||||
}
|
||||
}
|
||||
if (pushed) log.info('re-asserted news gump articles', { count: pushed })
|
||||
} catch (err) {
|
||||
log.warn('news gump reassert failed', { message: err.message })
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { inGump, buildArticle, pushPost, removePost, syncPost, reassertAll }
|
||||
@@ -1,77 +0,0 @@
|
||||
// ── The in-game town-crier announce leg ────────────────────────────────────
|
||||
//
|
||||
// MODULE-UO CONTENT, still living in core. MODULE_SYSTEM.md §1.8's third
|
||||
// entangled file: utils/announceWorker.js is core's news dispatcher, but one of
|
||||
// its two delivery legs goes to the shard through uoLinkClient.postTownCrier.
|
||||
// PR 4 turned the legs into registrations, and this file is what module-uo will
|
||||
// register in Phase 3 — it moves whole, with `'core'` becoming `'uo'` and the
|
||||
// leg id staying `towncrier` (grandfathered in registries.js: the id is a stored
|
||||
// value in announce_job_legs.leg).
|
||||
|
||||
const uoLinkClient = require('./uoLinkClient')
|
||||
const { deriveExcerpt } = require('./sanitizeHtml')
|
||||
const { articleUrl, baseUrl, legError } = require('../model/announceJobs/announceJobs.logic')
|
||||
|
||||
const TOWNCRIER_DURATION_SEC = Number(process.env.TOWNCRIER_DURATION_SEC) || 3600
|
||||
|
||||
// Sidecar town-crier caps, mirrored from the admin route validation
|
||||
// (admin/uoLink.router.js: lines isArray({ max: 8 }), lines.* isLength({ max: 200 })).
|
||||
// We pre-truncate to these so a published post never bounces with an error.
|
||||
const MAX_LINES = 8
|
||||
const MAX_LINE_LEN = 200
|
||||
|
||||
// Trim to a hard length, appending an ellipsis only when something was cut.
|
||||
function clamp(value, max) {
|
||||
const s = String(value == null ? '' : value)
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim()
|
||||
if (s.length <= max) return s
|
||||
return `${s.slice(0, max - 1).trimEnd()}…`
|
||||
}
|
||||
|
||||
// Build the town-crier lines: title, a one-line excerpt, then the URL. Each line
|
||||
// is clamped to the sidecar's per-line cap and the whole thing to the line-count
|
||||
// cap. Falls back to a stripped body excerpt when the post has no excerpt.
|
||||
function buildTownCrierText(post, { baseUrl: base } = {}) {
|
||||
const title = clamp(post.title, MAX_LINE_LEN)
|
||||
const excerptSource = post.excerpt || deriveExcerpt(post.body, MAX_LINE_LEN) || ''
|
||||
const lines = [title]
|
||||
const excerpt = clamp(excerptSource, MAX_LINE_LEN)
|
||||
if (excerpt) lines.push(excerpt)
|
||||
const url = clamp(articleUrl(base), MAX_LINE_LEN)
|
||||
if (url) lines.push(url)
|
||||
return lines.filter(Boolean).slice(0, MAX_LINES)
|
||||
}
|
||||
|
||||
async function dispatch(post) {
|
||||
const lines = buildTownCrierText(post, { baseUrl: baseUrl() })
|
||||
// Stable id: re-posting `post-<id>` REPLACES the prior town-crier entry rather
|
||||
// than stacking a duplicate, so a retry after a partial failure is safe.
|
||||
return uoLinkClient.postTownCrier({
|
||||
id: `post-${post.id}`,
|
||||
lines,
|
||||
durationSec: TOWNCRIER_DURATION_SEC,
|
||||
})
|
||||
}
|
||||
|
||||
function classify(result) {
|
||||
if (result && result.ok) return { outcome: 'done' }
|
||||
const status = result ? result.status : 0
|
||||
// 400 = over the line/duration caps (a data problem — do NOT retry).
|
||||
// 401 = token mismatch, 409 = protocol mismatch (both config problems).
|
||||
if (status === 400 || status === 401 || status === 409) {
|
||||
return { outcome: 'terminal', error: legError(result) }
|
||||
}
|
||||
// 503 (shard not connected), 504 (shard timeout), 0 (network/timeout / not
|
||||
// configured yet), and any other 5xx are transient — retry.
|
||||
return { outcome: 'retry', error: legError(result) }
|
||||
}
|
||||
|
||||
const leg = {
|
||||
leg: 'towncrier',
|
||||
label: 'In-game town crier',
|
||||
dispatch,
|
||||
classify,
|
||||
}
|
||||
|
||||
module.exports = { leg, dispatch, classify, buildTownCrierText, MAX_LINES, MAX_LINE_LEN }
|
||||
@@ -1,166 +0,0 @@
|
||||
// ── Shard live-feed SSE broadcaster ────────────────────────────────────────
|
||||
//
|
||||
// The browser can't talk to the sidecar's WebSocket directly (the token must
|
||||
// never reach it, and the WS may be on another host). Instead the server ingests
|
||||
// the WS feed and re-broadcasts events to browsers over Server-Sent Events
|
||||
// (plain HTTP — works through any reverse proxy).
|
||||
//
|
||||
// Since Protocol 3.0 the split is no longer "one public channel with a static
|
||||
// allowlist plus one admin channel". Each subscriber carries the audience rung
|
||||
// it resolved to at subscribe time, and every frame is
|
||||
//
|
||||
// 1. mapped kind → feature (an UNMAPPED kind reaches nobody below admin —
|
||||
// fail closed; see utils/shardVisibility.js rule 2),
|
||||
// 2. gated on that feature being enabled, streamed, and within the viewer's
|
||||
// rung, and
|
||||
// 3. passed through field projection, so `acct` / `webId` and any field an
|
||||
// admin has re-gated are stripped per viewer.
|
||||
//
|
||||
// **This is the security boundary.** It used to be the PUBLIC_KINDS set in this
|
||||
// file; it is now the kind map plus the visibility config. PUBLIC_KINDS still
|
||||
// exists and is still exported, but it is now DERIVED from the kind map (see
|
||||
// shardVisibility.js) so the two can no longer drift.
|
||||
//
|
||||
// shardIngest calls broadcast(event) for each ingested event; the public/admin
|
||||
// SSE route handlers call subscribe(req, res, channel).
|
||||
|
||||
const visibility = require('./shardVisibility')
|
||||
const log = require('./logger')('shard-broadcast')
|
||||
|
||||
// Re-exported for back-compat: shardEvents `/feed` filtering and
|
||||
// config/notificationStreams.js both ask "is this kind public-safe?".
|
||||
const { PUBLIC_KINDS } = visibility
|
||||
|
||||
// Open streams. Each entry is { res, level }. The admin bucket is kept separate
|
||||
// because it is unconditional and must not depend on a config read.
|
||||
const clients = { public: new Set(), admin: new Set() }
|
||||
|
||||
const KEEPALIVE_MS = 25000
|
||||
|
||||
// Register an SSE stream on a channel. Sets the SSE headers, sends an initial
|
||||
// comment, keeps the connection warm with periodic pings, and cleans up on close.
|
||||
//
|
||||
// The viewer's rung is resolved ONCE, here, and frozen for the life of the
|
||||
// connection — a long-lived stream must not silently gain privilege because the
|
||||
// caller's session changed underneath it. (Config changes, by contrast, DO take
|
||||
// effect live: the config is read per broadcast, cached ~5s.)
|
||||
async function subscribe(req, res, channel) {
|
||||
const bucket = clients[channel]
|
||||
if (!bucket) {
|
||||
res.status(400).end()
|
||||
return
|
||||
}
|
||||
|
||||
let level = 'admin'
|
||||
if (channel === 'public') {
|
||||
try {
|
||||
level = await visibility.viewerLevel(req)
|
||||
} catch (err) {
|
||||
// Fail closed: an unresolvable viewer is anonymous, not privileged.
|
||||
log.warn('viewerLevel failed on subscribe; treating as anonymous', { message: err.message })
|
||||
level = 'anonymous'
|
||||
}
|
||||
}
|
||||
|
||||
res.writeHead(200, {
|
||||
'Content-Type': 'text/event-stream',
|
||||
'Cache-Control': 'no-cache, no-transform',
|
||||
Connection: 'keep-alive',
|
||||
'X-Accel-Buffering': 'no', // disable proxy buffering so events flush immediately
|
||||
})
|
||||
res.write('retry: 5000\n\n') // tell EventSource to reconnect after 5s if dropped
|
||||
res.write(': connected\n\n')
|
||||
|
||||
const client = { res, level, ping: null }
|
||||
bucket.add(client)
|
||||
|
||||
client.ping = setInterval(() => {
|
||||
try {
|
||||
res.write(': ping\n\n')
|
||||
} catch {
|
||||
/* write after close — cleanup below handles it */
|
||||
}
|
||||
}, KEEPALIVE_MS)
|
||||
|
||||
const cleanup = () => drop(bucket, client)
|
||||
req.on('close', cleanup)
|
||||
res.on('error', cleanup)
|
||||
}
|
||||
|
||||
// The ONLY way a client leaves a bucket. Clearing the keepalive here (rather
|
||||
// than only in the close handler) matters: a client dropped because its write
|
||||
// threw never fires `req.close`, so its interval would otherwise keep firing on
|
||||
// a dead socket for the life of the process.
|
||||
function drop(bucket, client) {
|
||||
clearInterval(client.ping)
|
||||
bucket.delete(client)
|
||||
}
|
||||
|
||||
function writeTo(bucket, client, payload) {
|
||||
try {
|
||||
client.res.write(payload)
|
||||
} catch (err) {
|
||||
log.warn('sse write failed; dropping client', { message: err.message })
|
||||
drop(bucket, client)
|
||||
}
|
||||
}
|
||||
|
||||
// Fan an ingested event out. The admin channel gets it verbatim, always. Public
|
||||
// subscribers are filtered and projected per their own rung — so two viewers on
|
||||
// the same channel can legitimately receive different versions of one frame, or
|
||||
// one of them nothing at all.
|
||||
async function broadcast(event) {
|
||||
if (!event || !event.kind) return
|
||||
|
||||
if (clients.admin.size) {
|
||||
const frame = `data: ${JSON.stringify(event)}\n\n`
|
||||
for (const client of [...clients.admin]) writeTo(clients.admin, client, frame)
|
||||
}
|
||||
|
||||
if (!clients.public.size) return
|
||||
|
||||
let config
|
||||
try {
|
||||
config = await visibility.getConfig()
|
||||
} catch (err) {
|
||||
// Fail closed: without a config we cannot prove a frame is safe to send.
|
||||
log.error('visibility config unavailable; withholding public frame', err)
|
||||
return
|
||||
}
|
||||
|
||||
// Most frames land on one rung set, so cache the serialised payload per level
|
||||
// instead of re-projecting and re-stringifying for every subscriber.
|
||||
const byLevel = new Map()
|
||||
for (const client of [...clients.public]) {
|
||||
let frame = byLevel.get(client.level)
|
||||
if (frame === undefined) {
|
||||
frame = visibility.kindVisibleTo(event.kind, client.level, config)
|
||||
? `data: ${JSON.stringify(visibility.projectFeature(visibility.KIND_FEATURE.get(event.kind), event, client.level, config))}\n\n`
|
||||
: null
|
||||
byLevel.set(client.level, frame)
|
||||
}
|
||||
if (frame) writeTo(clients.public, client, frame)
|
||||
}
|
||||
}
|
||||
|
||||
// Close every open stream (graceful shutdown). Clears each keepalive timer too —
|
||||
// without that the intervals keep the event loop alive after the streams are
|
||||
// gone, and the process won't exit.
|
||||
function closeAll() {
|
||||
for (const bucket of Object.values(clients)) {
|
||||
for (const client of [...bucket]) {
|
||||
drop(bucket, client)
|
||||
try {
|
||||
client.res.end()
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function stats() {
|
||||
return { publicClients: clients.public.size, adminClients: clients.admin.size }
|
||||
}
|
||||
|
||||
module.exports = { subscribe, broadcast, closeAll, stats, PUBLIC_KINDS }
|
||||
@@ -1,314 +0,0 @@
|
||||
// ── Shard event ingest dispatcher ──────────────────────────────────────────
|
||||
//
|
||||
// The single entry point for every event that arrives on the uo-link WebSocket
|
||||
// feed (and for backfilled /history events on reconnect). It routes by kind:
|
||||
// • state-changing kinds update shard_online / shard_economy / shard_houses,
|
||||
// • notable kinds are appended to the append-only shard_events log,
|
||||
// • every kind is fanned out to the SSE broadcaster (which decides public vs
|
||||
// admin visibility).
|
||||
// High-frequency kinds (char.vitals, economy.supply) are deliberately NOT logged
|
||||
// to shard_events — they only update state — keeping the event log lean.
|
||||
//
|
||||
// Dependencies are injected (defaulting to the real models) so the routing can
|
||||
// be unit-tested with mocked writes.
|
||||
|
||||
const shardEventsModel = require('../model/shardEvents/shardEvents.model')
|
||||
const shardStateModel = require('../model/shardState/shardState.model')
|
||||
const shardLinksModel = require('../model/shardLinks/shardLinks.model')
|
||||
const shardMarketModel = require('../model/shardMarket/shardMarket.model')
|
||||
const uoLinkConfigModel = require('../model/uoLinkConfig/uoLinkConfig.model')
|
||||
const settingsModel = require('../model/settings/settings.model')
|
||||
const broadcaster = require('./shardBroadcast')
|
||||
const shardPush = require('./shardPush')
|
||||
const defaultLog = require('./logger')('shard-ingest')
|
||||
|
||||
// Notable kinds appended to the shard_events log. High-frequency/session kinds
|
||||
// (char.vitals, economy.supply, mob.login/logout, account.login.attempt,
|
||||
// gold.change, vendor.buy/sell) are excluded on purpose. house.decay is handled
|
||||
// specially — logged only on the transition INTO IDOC.
|
||||
const LOGGED_KINDS = new Set([
|
||||
'vendor.sale',
|
||||
'player.death',
|
||||
'player.murdered',
|
||||
'mob.killed',
|
||||
'quest.complete',
|
||||
'skill.gain',
|
||||
'fame.change',
|
||||
'karma.change',
|
||||
'audit.set',
|
||||
'audit.command',
|
||||
'admin.audit',
|
||||
'cheat.fastwalk',
|
||||
'link.request',
|
||||
'server.hello',
|
||||
'server.shutdown',
|
||||
'server.crashed',
|
||||
// Protocol 2.0: a real-time guild join (the board itself is state, not logged).
|
||||
'guild.join',
|
||||
// Protocol 2.0 provisioning audit (admin channel only — not in PUBLIC_KINDS).
|
||||
'account.audit',
|
||||
'account.unlinked',
|
||||
])
|
||||
|
||||
// Tracks the current shard boot id so a restart (changed bootId on server.hello)
|
||||
// can be detected and stale online state dropped. Module-level so it survives
|
||||
// across events within a process; reset() is exposed for tests.
|
||||
const state = { bootId: null }
|
||||
function reset() {
|
||||
state.bootId = null
|
||||
}
|
||||
|
||||
// Should this event be written to the append-only log?
|
||||
function shouldLog(event) {
|
||||
if (event.kind === 'house.decay') return String(event.to).toUpperCase() === 'IDOC'
|
||||
return LOGGED_KINDS.has(event.kind)
|
||||
}
|
||||
|
||||
// ServUO's stock Server.cfg name. An operator who never set one publishes this
|
||||
// verbatim, so it carries no more information than a blank — matched
|
||||
// case-insensitively and trim-tolerantly, but ONLY as an exact whole value: a
|
||||
// shard genuinely called "My Shard Reborn" keeps its name.
|
||||
const STOCK_SHARD_NAME = 'my shard'
|
||||
|
||||
/**
|
||||
* The name to publish for the shard: its own, or this instance's when it has
|
||||
* effectively not given one.
|
||||
*
|
||||
* Deliberately not a general "blank means brand" rule applied across the wire —
|
||||
* it is scoped to this one field, where the two names denote the same thing.
|
||||
*/
|
||||
async function resolveShardName(shard, deps) {
|
||||
const given = String(shard ?? '').trim()
|
||||
if (given !== '' && given.toLowerCase() !== STOCK_SHARD_NAME) return given
|
||||
try {
|
||||
return (await deps.settings.getInstanceName()) || given
|
||||
} catch {
|
||||
// A ruleset that publishes the stock name is still better than one that
|
||||
// fails to store because the settings read hiccuped.
|
||||
return given
|
||||
}
|
||||
}
|
||||
|
||||
// Apply the state-change side effect for a kind (if any). Returns a promise.
|
||||
async function applyStateChange(event, deps) {
|
||||
const { shardState, uoLinkConfig, log } = deps
|
||||
switch (event.kind) {
|
||||
case 'server.hello': {
|
||||
const incoming = event.bootId || null
|
||||
if (incoming && state.bootId && incoming !== state.bootId) {
|
||||
log.warn('shard restarted (bootId changed) — clearing online roster', {
|
||||
from: state.bootId,
|
||||
to: incoming,
|
||||
})
|
||||
await shardState.clearOnline()
|
||||
}
|
||||
if (incoming) state.bootId = incoming
|
||||
await uoLinkConfig.recordStatus({ pluginConnected: true, bootId: incoming, lastEventAt: event.t })
|
||||
return
|
||||
}
|
||||
case 'server.shutdown':
|
||||
case 'server.crashed':
|
||||
// Shard is going away — nobody is online anymore.
|
||||
await shardState.clearOnline()
|
||||
await uoLinkConfig.recordStatus({ pluginConnected: false })
|
||||
return
|
||||
case 'mob.login': {
|
||||
const who = event.who || {}
|
||||
await shardState.upsertOnline({
|
||||
serial: who.serial,
|
||||
name: who.name,
|
||||
acct: who.acct,
|
||||
webId: event.webId,
|
||||
map: event.map,
|
||||
x: event.x,
|
||||
y: event.y,
|
||||
z: event.z,
|
||||
})
|
||||
return
|
||||
}
|
||||
case 'mob.logout': {
|
||||
const who = event.who || {}
|
||||
if (who.serial) await shardState.setOffline(who.serial)
|
||||
return
|
||||
}
|
||||
case 'char.vitals':
|
||||
await shardState.upsertOnline({
|
||||
serial: event.serial,
|
||||
hits: event.hits,
|
||||
hitsMax: event.hitsMax,
|
||||
mana: event.mana,
|
||||
manaMax: event.manaMax,
|
||||
stam: event.stam,
|
||||
stamMax: event.stamMax,
|
||||
str: event.str,
|
||||
dex: event.dex,
|
||||
int: event.int,
|
||||
map: event.map,
|
||||
x: event.x,
|
||||
y: event.y,
|
||||
})
|
||||
return
|
||||
case 'economy.supply':
|
||||
await shardState.addEconomySample({ accounts: event.accounts, gold: event.gold, t: event.t })
|
||||
return
|
||||
case 'house.decay':
|
||||
await shardState.upsertHouse({
|
||||
serial: event.serial,
|
||||
stage: event.to,
|
||||
map: event.map,
|
||||
x: event.x,
|
||||
y: event.y,
|
||||
z: event.z,
|
||||
region: event.region,
|
||||
name: event.name,
|
||||
ownerSerial: event.ownerSerial,
|
||||
ownerAcct: event.ownerAcct,
|
||||
builtOn: event.builtOn,
|
||||
lastRefreshed: event.lastRefreshed,
|
||||
})
|
||||
return
|
||||
case 'champ.update':
|
||||
await shardState.upsertChamp(event)
|
||||
return
|
||||
case 'champ.remove':
|
||||
await shardState.removeChamp(event.serial)
|
||||
return
|
||||
case 'page.new':
|
||||
case 'page.updated':
|
||||
await shardState.upsertPage(event)
|
||||
return
|
||||
case 'page.closed':
|
||||
await shardState.removePage(event.pageId)
|
||||
return
|
||||
// ── Protocol 2.0 boards ──────────────────────────────────────────────
|
||||
case 'guild.update':
|
||||
await shardState.upsertGuild(event)
|
||||
return
|
||||
case 'guild.remove':
|
||||
await shardState.removeGuild(event.id)
|
||||
return
|
||||
case 'city.update':
|
||||
// Upserts the board AND captures term history (idempotent).
|
||||
await shardState.upsertGovernor(event)
|
||||
return
|
||||
case 'presence.online':
|
||||
await shardState.setPresence(event)
|
||||
return
|
||||
case 'house.update':
|
||||
await shardState.upsertHouseRegistry(event)
|
||||
return
|
||||
case 'house.remove':
|
||||
await shardState.removeHouse(event.serial)
|
||||
return
|
||||
// ── Protocol 3.0 ─────────────────────────────────────────────────────
|
||||
// The shard re-emits its whole ruleset on every sidecar connect, so this is
|
||||
// an overwrite, not an append — and deliberately NOT in LOGGED_KINDS: it
|
||||
// would put a duplicate row in the event log on every reconnect, and
|
||||
// server.hello already marks each of those.
|
||||
case 'world.ruleset':
|
||||
// A shard whose operator never edited Server.cfg publishes ServUO's stock
|
||||
// "My Shard". That is the shard saying *unnamed*, not a name, so the site
|
||||
// answers with its own — the rules page reading "My Shard" under a header
|
||||
// reading UOMysticmoon is the shard failing to introduce itself.
|
||||
//
|
||||
// Normalized HERE rather than on read because the ruleset is also live: the
|
||||
// same `event` object is handed to the SSE broadcast a few lines below, and
|
||||
// a read-time fix would be undone by the next reconnect's frame.
|
||||
event.shard = await resolveShardName(event.shard, deps)
|
||||
await shardState.setRuleset(event)
|
||||
return
|
||||
// Board state, like guild.update — the newest frame for a system replaces the
|
||||
// previous one, so it is NOT in LOGGED_KINDS. Logging would append a row every
|
||||
// time anyone's score moved the top ten, which is a board, not an event.
|
||||
case 'points.board':
|
||||
await shardState.upsertPointsBoard(event)
|
||||
return
|
||||
// Player-vendor market index. Each frame is authoritative for one shop, so
|
||||
// the model replaces that vendor's whole listing set rather than merging.
|
||||
//
|
||||
// NOT in LOGGED_KINDS, and this is the strongest case of the three v3 kinds:
|
||||
// one frame carries up to 250 listings, the sweep re-emits a shop on any
|
||||
// price change, and appending each of those to the event log would make
|
||||
// shard_events mostly a price history nobody reads. The market IS the state.
|
||||
case 'vendor.listing':
|
||||
await deps.shardMarket.upsertVendor(event)
|
||||
return
|
||||
case 'vendor.listing.remove':
|
||||
await deps.shardMarket.removeVendor(event.serial)
|
||||
return
|
||||
case 'account.unlinked':
|
||||
// A player ran [unlink in game (or a site-side unlink echoed back) — drop
|
||||
// our local link mirror so attribution stops immediately.
|
||||
if (event.account) await deps.shardLinks.removeByAccount(event.account)
|
||||
return
|
||||
// guild.join / account.audit → logged; region.enter → broadcast-only.
|
||||
default:
|
||||
// No state side effect (e.g. vendor.sale, audit.*, cheat.*) — logging and
|
||||
// broadcasting still happen in ingest().
|
||||
}
|
||||
}
|
||||
|
||||
// Ingest one event. Returns { logged, stored } for tests/stats. `fromBackfill`
|
||||
// suppresses the SSE broadcast (a reconnect replay shouldn't re-animate the
|
||||
// live ticker). Never throws — a bad single event must not kill the feed.
|
||||
// Resolve the injectable dependencies to their live defaults (tests override a
|
||||
// subset). Split out so ingest() isn't penalised for the fan of `|| default`s.
|
||||
function resolveDeps(deps) {
|
||||
return {
|
||||
shardEvents: deps.shardEvents || shardEventsModel,
|
||||
shardState: deps.shardState || shardStateModel,
|
||||
shardLinks: deps.shardLinks || shardLinksModel,
|
||||
shardMarket: deps.shardMarket || shardMarketModel,
|
||||
uoLinkConfig: deps.uoLinkConfig || uoLinkConfigModel,
|
||||
settings: deps.settings || settingsModel,
|
||||
broadcast: deps.broadcast || broadcaster.broadcast,
|
||||
pushDispatch: deps.pushDispatch || shardPush.fromShardEvent,
|
||||
log: deps.log || defaultLog,
|
||||
}
|
||||
}
|
||||
|
||||
async function ingest(event, deps = {}) {
|
||||
const d = resolveDeps(deps)
|
||||
|
||||
if (!event || typeof event.kind !== 'string') return { logged: false, stored: false }
|
||||
// ws.hello / pong are transport frames, not game events.
|
||||
if (event.kind === 'ws.hello' || event.kind === 'pong') return { logged: false, stored: false }
|
||||
|
||||
const t = Number.isFinite(event.t) ? event.t : Date.now()
|
||||
let stored = false
|
||||
let logged = false
|
||||
|
||||
try {
|
||||
await applyStateChange(event, d)
|
||||
} catch (err) {
|
||||
d.log.warn('state-change write failed', { kind: event.kind, message: err.message })
|
||||
}
|
||||
|
||||
if (shouldLog(event)) {
|
||||
logged = true
|
||||
try {
|
||||
stored = await d.shardEvents.append({ kind: event.kind, t, bootId: state.bootId, payload: event })
|
||||
} catch (err) {
|
||||
d.log.warn('event log write failed', { kind: event.kind, message: err.message })
|
||||
}
|
||||
}
|
||||
|
||||
if (!deps.fromBackfill) {
|
||||
// Broadcast is async since v3 (it reads the visibility config to decide what
|
||||
// each subscriber may see). Fire-and-forget, like the push fan-out below: a
|
||||
// slow config read must never delay or fail ingest.
|
||||
Promise.resolve(d.broadcast(event)).catch((err) =>
|
||||
d.log.warn('broadcast failed', { kind: event.kind, message: err.message }),
|
||||
)
|
||||
// Opt-in push fan-out, off the same event source as the SSE broadcast.
|
||||
// Fire-and-forget (a slow/dead ntfy relay must never delay or fail ingest);
|
||||
// fromShardEvent is self-guarding, but .catch() covers any lookup rejection.
|
||||
Promise.resolve(d.pushDispatch(event, { shardLinks: d.shardLinks })).catch((err) =>
|
||||
d.log.warn('push dispatch failed', { kind: event.kind, message: err.message }),
|
||||
)
|
||||
}
|
||||
|
||||
return { logged, stored }
|
||||
}
|
||||
|
||||
module.exports = { ingest, shouldLog, reset, LOGGED_KINDS, state }
|
||||
@@ -1,47 +0,0 @@
|
||||
// ── Shard event → push fan-out ─────────────────────────────────────────────
|
||||
//
|
||||
// MODULE-UO CONTENT, still living in core — the inverted half of
|
||||
// MODULE_SYSTEM.md §1.8's second entangled file. `utils/pushDispatch.js` is core
|
||||
// infrastructure, but its `fromShardEvent()` required the shardLinks model and
|
||||
// the shard event mapper, which is a core file importing content. PR 4 inverted
|
||||
// it: `publish()` stays core, and this — the thing that knows what a shard event
|
||||
// is — moved out to call it. Phase 3 moves this file to module-uo whole, where it
|
||||
// will reach `publish` through `ctx.push.publish` instead of a require.
|
||||
//
|
||||
// Owner resolution is the reason this cannot just be a mapper: a personal
|
||||
// (owner-keyed) target names a GAME account, and turning that into a website user
|
||||
// needs the shardLinks model. An unlinked account is simply nobody to notify.
|
||||
|
||||
const shardLinks = require('../model/shardLinks/shardLinks.model')
|
||||
const { mapShardEvent } = require('../config/shardStreams')
|
||||
const { publish } = require('./pushDispatch')
|
||||
const log = require('./logger')('shard-push')
|
||||
|
||||
// Fan a shard event out to push. Resolves personal (owner-keyed) targets to the
|
||||
// owning website user via shardLinks (an unlinked account → nobody to notify).
|
||||
// Never throws — a dead relay must never affect ingest.
|
||||
async function fromShardEvent(event, deps = {}) {
|
||||
const links = deps.shardLinks || shardLinks
|
||||
const doPublish = deps.publish || publish
|
||||
const targets = mapShardEvent(event, deps.tracker)
|
||||
for (const t of targets) {
|
||||
try {
|
||||
if (t.ownerAccount) {
|
||||
let owner = null
|
||||
try {
|
||||
owner = await links.getByAccount(t.ownerAccount)
|
||||
} catch {
|
||||
owner = null
|
||||
}
|
||||
if (!owner || owner.userId == null) continue
|
||||
await doPublish(t.streamId, { ref: t.ref, ownerUserId: owner.userId }, deps)
|
||||
} else {
|
||||
await doPublish(t.streamId, { ref: t.ref }, deps)
|
||||
}
|
||||
} catch (err) {
|
||||
log.warn('push dispatch target failed', { streamId: t.streamId, message: err.message })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { fromShardEvent }
|
||||
@@ -1,26 +0,0 @@
|
||||
// Recent player-vendor sales for a set of game accounts. Shared by the player
|
||||
// self endpoint (the caller's linked accounts) and the admin user-detail
|
||||
// endpoint (a target user's linked accounts). Reads the site's own ingested
|
||||
// event log — no sidecar round-trip — and filters to sales whose owning account
|
||||
// is in the set. Newest 50, already newest-first from shardEvents.list.
|
||||
|
||||
const shardEvents = require('../model/shardEvents/shardEvents.model')
|
||||
|
||||
async function salesForAccounts(accounts) {
|
||||
const set = accounts instanceof Set ? accounts : new Set(accounts)
|
||||
if (set.size === 0) return []
|
||||
const events = await shardEvents.list({ kind: 'vendor.sale', limit: 500 })
|
||||
return events
|
||||
.filter((e) => e.payload && set.has(e.payload.ownerAcct))
|
||||
.slice(0, 50)
|
||||
.map((e) => ({
|
||||
t: e.t,
|
||||
itemType: e.payload.itemType,
|
||||
amount: e.payload.amount,
|
||||
price: e.payload.price,
|
||||
commission: e.payload.commission,
|
||||
ownerAcct: e.payload.ownerAcct,
|
||||
}))
|
||||
}
|
||||
|
||||
module.exports = { salesForAccounts }
|
||||
@@ -1,435 +0,0 @@
|
||||
// ── Shard feature visibility ───────────────────────────────────────────────
|
||||
//
|
||||
// Admin-configurable, per-feature and per-field audience control over every
|
||||
// shard-derived surface on the site. Replaces the hardcoded split that used to
|
||||
// live in two places (the PUBLIC_KINDS allowlist in shardBroadcast.js, and the
|
||||
// ad-hoc `canSeeStaffLocation` style checks in the public controllers).
|
||||
//
|
||||
// Design rules (docs/link/v3.md §3):
|
||||
//
|
||||
// • Visibility lives HERE, on the website — never in the sidecar. The sidecar
|
||||
// is a dumb forwarder: it accepts frames, stores them, forwards them
|
||||
// verbatim, and serves store-backed reads. It defines no audiences.
|
||||
// • Every default reproduces the behavior that shipped before this module, so
|
||||
// installing it changes nothing until an admin edits the config.
|
||||
// • Two rules an admin CANNOT override:
|
||||
// 1. `acct` / `webId` are admin-only, always. They are not in-game
|
||||
// visible (unlike a character name) and are not configurable fields.
|
||||
// 2. A kind absent from KIND_FEATURE is never broadcast below `admin`.
|
||||
// Fail closed — this is what keeps the kind map a security boundary
|
||||
// rather than a convenience filter.
|
||||
//
|
||||
// The audience ladder is ordered; each rung implies the ones below it.
|
||||
|
||||
const db = require('../model/shardVisibility/shardVisibility.model')
|
||||
const shardLinks = require('../model/shardLinks/shardLinks.model')
|
||||
const auth = require('./auth')
|
||||
const log = require('./logger')('shard-visibility')
|
||||
|
||||
// ── The ladder ─────────────────────────────────────────────────────────────
|
||||
|
||||
const LADDER = ['anonymous', 'logged_in', 'player', 'staff', 'admin']
|
||||
const RANK = new Map(LADDER.map((level, i) => [level, i]))
|
||||
|
||||
const isLevel = (level) => RANK.has(level)
|
||||
|
||||
// The two fallbacks are deliberately ASYMMETRIC, and the asymmetry is the whole
|
||||
// point: an unrecognised value must always lose. A single shared fallback cannot
|
||||
// do that — whichever direction it picks, it fails open on one side. So:
|
||||
//
|
||||
// • an unknown VIEWER level floors to the bottom rung (grants nothing), and
|
||||
// • an unknown REQUIREMENT ceils to the top rung (satisfied by nobody but admin).
|
||||
//
|
||||
// With one `rank()` defaulting to admin, a viewer level that fell through (a
|
||||
// typo, a future rung this build doesn't know, a value from a caller that
|
||||
// skipped viewerLevel) would have been treated as an ADMIN and passed every gate.
|
||||
const viewerRank = (level) => RANK.get(level) ?? 0
|
||||
const requiredRank = (level) => RANK.get(level) ?? RANK.get('admin')
|
||||
|
||||
// True when a viewer at `viewer` satisfies a requirement of `required`.
|
||||
const meets = (viewer, required) => viewerRank(viewer) >= requiredRank(required)
|
||||
|
||||
// Exported for tests/diagnostics; `meets` is what callers should use.
|
||||
const rank = viewerRank
|
||||
|
||||
// ── Features ───────────────────────────────────────────────────────────────
|
||||
//
|
||||
// All ten shard surfaces: the six that shipped before v3 plus the four v3 adds.
|
||||
// `fields` lists only the SENSITIVE fields — those an admin may re-gate. A field
|
||||
// not listed here is visible whenever the feature itself is.
|
||||
//
|
||||
// LOCKED_FIELDS are exempt from configuration entirely (rule 1 above).
|
||||
|
||||
const LOCKED_FIELDS = { acct: 'admin', webId: 'admin' }
|
||||
|
||||
// Rule 1 matches on the FIELD'S MEANING, not on one exact spelling. The wire
|
||||
// frames nest actors (`leader.acct`), but several read models flatten them
|
||||
// instead (`shapeHouse` emits `ownerAcct`, `shapeGuild`'s fallback emits
|
||||
// `leaderAcct`/`leaderWebId`), and an exact-key check silently missed every
|
||||
// flattened one — which is how `GET /public/shard/idoc` served `ownerAcct` to
|
||||
// anonymous callers while the same account name was correctly stripped from the
|
||||
// live `house.decay` frame.
|
||||
//
|
||||
// So a key is locked when it IS `acct`/`webId` or ENDS in one, case-insensitively
|
||||
// (`ownerAcct`, `leaderWebId`, `governorAcct`). Suffix matching is what makes this
|
||||
// fail closed for shapes nobody has written yet.
|
||||
const LOCKED_SUFFIXES = ['acct', 'webid']
|
||||
const isLockedField = (key) => {
|
||||
const k = String(key).toLowerCase()
|
||||
return LOCKED_SUFFIXES.some((suffix) => k === suffix || k.endsWith(suffix))
|
||||
}
|
||||
|
||||
const FEATURES = {
|
||||
// ── Shipped before v3. Defaults reproduce the previous hardcoded behavior. ──
|
||||
status: { audience: 'anonymous', fields: {} },
|
||||
activity: { audience: 'anonymous', fields: {} },
|
||||
champs: { audience: 'anonymous', fields: {} },
|
||||
guilds: { audience: 'anonymous', fields: {} },
|
||||
governors: { audience: 'anonymous', fields: {} },
|
||||
// The public Houses page showed IDOC location only; owner/price were staff.
|
||||
// `owner` is the actor object on the house.decay/house.update frames;
|
||||
// `ownerName`/`ownerSerial` are the flattened spellings shapeHouse emits on the
|
||||
// REST read models. Both are listed so one rule covers the wire and the read
|
||||
// model — the flattened `ownerAcct` needs no entry, being locked by rule 1.
|
||||
houses: {
|
||||
audience: 'anonymous',
|
||||
fields: { owner: 'staff', ownerName: 'staff', ownerSerial: 'staff', price: 'staff' },
|
||||
},
|
||||
// /public/shard/online listed linked staff to everyone but gated location to
|
||||
// admin+moderator — which is exactly the `staff` rung.
|
||||
presence: { audience: 'anonymous', fields: { location: 'staff' } },
|
||||
|
||||
// ── New in v3. ──
|
||||
ruleset: { audience: 'anonymous', fields: { connect: 'anonymous' } },
|
||||
atlas: { audience: 'anonymous', fields: {} },
|
||||
// `name` is the ranked character's name inside points.board's `top` entries, and
|
||||
// it is spelled the way the WIRE spells it, not the way v3.md §7.4 describes it
|
||||
// ("characterName"). projectValue matches on the literal JSON key, so a rule
|
||||
// named for the field's meaning rather than its key silently does nothing — the
|
||||
// same failure §3.6.1 records for the flattened `ownerAcct` spelling. Within a
|
||||
// leaderboards payload `name` can only be a character name: the board's own
|
||||
// display name arrives as `nameString`/`nameNumber`.
|
||||
leaderboards: { audience: 'anonymous', fields: { name: 'anonymous' } },
|
||||
// Shop name, owner character name and vendor location are already globally
|
||||
// visible in-game via the stock Vendor Search gump, so publishing them is not
|
||||
// a new disclosure — but they stay configurable so an admin can tighten them.
|
||||
//
|
||||
// `ownerName` and `location` were pre-wired here by Part A, before the frame
|
||||
// existed; both were re-checked against the real `vendor.listing` and both are
|
||||
// genuine keys on it (unlike leaderboards' `characterName`, which was inert).
|
||||
// `location` is a NESTED object on the wire and on the read model precisely so
|
||||
// that one rule hides map, coordinates, region and house together — five flat
|
||||
// keys would be five rules that drift apart.
|
||||
//
|
||||
// `ownerSerial` is listed alongside `ownerName` for the same reason `houses`
|
||||
// lists both: an admin who hides the owner's name and is left with a serial
|
||||
// that every other board resolves back to that name has not hidden anything.
|
||||
market: {
|
||||
audience: 'anonymous',
|
||||
fields: { ownerName: 'anonymous', ownerSerial: 'anonymous', location: 'anonymous' },
|
||||
},
|
||||
}
|
||||
|
||||
const FEATURE_NAMES = Object.keys(FEATURES)
|
||||
const isFeature = (name) => Object.hasOwn(FEATURES, name)
|
||||
|
||||
// ── Kind → feature ─────────────────────────────────────────────────────────
|
||||
//
|
||||
// Every event kind that may ever leave the admin channel must appear here.
|
||||
// Anything else is admin-only by omission (rule 2). This map is seeded from
|
||||
// what PUBLIC_KINDS listed before v3, so the public stream carries exactly the
|
||||
// same kinds it did — now attributed to a feature that an admin can re-gate.
|
||||
|
||||
const KIND_FEATURE = new Map(
|
||||
Object.entries({
|
||||
// status / lifecycle
|
||||
'server.hello': 'status',
|
||||
'server.shutdown': 'status',
|
||||
'server.crashed': 'status',
|
||||
'economy.supply': 'status',
|
||||
// activity feed
|
||||
'player.death': 'activity',
|
||||
'player.murdered': 'activity',
|
||||
'mob.killed': 'activity',
|
||||
'quest.complete': 'activity',
|
||||
'skill.gain': 'activity',
|
||||
'fame.change': 'activity',
|
||||
'karma.change': 'activity',
|
||||
'mob.login': 'activity',
|
||||
'mob.logout': 'activity',
|
||||
// boards
|
||||
'champ.update': 'champs',
|
||||
'champ.remove': 'champs',
|
||||
'guild.update': 'guilds',
|
||||
'guild.remove': 'guilds',
|
||||
'guild.join': 'guilds',
|
||||
'city.update': 'governors',
|
||||
'presence.online': 'presence',
|
||||
'region.enter': 'presence',
|
||||
// house.decay is the IDOC signal the public Houses page renders. The full
|
||||
// registry (house.update / house.remove — owner, price, co-owners) stays
|
||||
// off the map deliberately, so it remains admin-only exactly as before.
|
||||
'house.decay': 'houses',
|
||||
// v3
|
||||
'world.ruleset': 'ruleset',
|
||||
'points.board': 'leaderboards',
|
||||
// vendor.listing IS mapped, but the market feature ships with its stream
|
||||
// disabled (see DEFAULT_STREAM_OFF): a live firehose of full vendor
|
||||
// inventories would be the site's biggest bandwidth consumer and no page
|
||||
// needs it live. An admin can turn it on.
|
||||
'vendor.listing': 'market',
|
||||
'vendor.listing.remove': 'market',
|
||||
}),
|
||||
)
|
||||
|
||||
// Features whose SSE fan-out is off unless an admin enables it. The REST reads
|
||||
// are unaffected; only the live stream is suppressed.
|
||||
const DEFAULT_STREAM_OFF = new Set(['market'])
|
||||
|
||||
// Back-compat: the set of kinds that reach an anonymous viewer under the default
|
||||
// config. shardEvents `/feed` filtering and notificationStreams.js both consume
|
||||
// this. Derived from the map above rather than hand-maintained, so the two can
|
||||
// no longer drift.
|
||||
const PUBLIC_KINDS = new Set(
|
||||
[...KIND_FEATURE.entries()]
|
||||
.filter(([, feature]) => {
|
||||
if (DEFAULT_STREAM_OFF.has(feature)) return false
|
||||
return FEATURES[feature].audience === 'anonymous'
|
||||
})
|
||||
.map(([kind]) => kind),
|
||||
)
|
||||
|
||||
// ── Config (DB-backed, cached) ─────────────────────────────────────────────
|
||||
|
||||
const CONFIG_TTL_MS = 5000
|
||||
let cache = null
|
||||
let cachedAt = 0
|
||||
|
||||
// Merge a stored row over its compiled default. Unknown feature names in the DB
|
||||
// are ignored (a stale row from a removed feature must not resurrect it), and an
|
||||
// invalid rung falls back to the default rather than failing open.
|
||||
function applyRow(name, row) {
|
||||
const base = FEATURES[name]
|
||||
const audience = isLevel(row?.audience) ? row.audience : base.audience
|
||||
const fields = { ...base.fields }
|
||||
for (const [field, level] of Object.entries(row?.fieldRules || {})) {
|
||||
if (isLockedField(field)) continue // rule 1: not configurable
|
||||
if (isLevel(level)) fields[field] = level
|
||||
}
|
||||
return {
|
||||
enabled: row ? !!row.enabled : true,
|
||||
audience,
|
||||
fields,
|
||||
stream: row?.stream == null ? !DEFAULT_STREAM_OFF.has(name) : !!row.stream,
|
||||
}
|
||||
}
|
||||
|
||||
function compileDefaults() {
|
||||
const out = {}
|
||||
for (const name of FEATURE_NAMES) out[name] = applyRow(name, null)
|
||||
return out
|
||||
}
|
||||
|
||||
// Read the config, cached briefly. Falls back to compiled defaults if the DB is
|
||||
// unreachable — the defaults reproduce pre-v3 behavior, so a DB blip degrades to
|
||||
// "what the site did before" rather than to "everything is public".
|
||||
async function getConfig() {
|
||||
const now = Date.now()
|
||||
if (cache && now - cachedAt < CONFIG_TTL_MS) return cache
|
||||
try {
|
||||
const rows = await db.listAll()
|
||||
const byName = new Map(rows.map((r) => [r.feature, r]))
|
||||
const out = {}
|
||||
for (const name of FEATURE_NAMES) out[name] = applyRow(name, byName.get(name))
|
||||
cache = out
|
||||
cachedAt = now
|
||||
} catch (err) {
|
||||
log.error('getConfig; falling back to defaults', err)
|
||||
cache = cache || compileDefaults()
|
||||
cachedAt = now
|
||||
}
|
||||
return cache
|
||||
}
|
||||
|
||||
const invalidate = () => {
|
||||
cache = null
|
||||
cachedAt = 0
|
||||
}
|
||||
|
||||
// ── Viewer level ───────────────────────────────────────────────────────────
|
||||
//
|
||||
// anonymous no session
|
||||
// logged_in authenticated, no linked game account
|
||||
// player authenticated with a linked game account
|
||||
// staff admin | moderator — the same set as the existing `modAccess` gate.
|
||||
// `editor` is a CONTENT role with no shard privilege today, so it
|
||||
// resolves by link status like any other member; mapping it to staff
|
||||
// here would silently widen what editors can see.
|
||||
// admin admin
|
||||
//
|
||||
// Staff always satisfy the `player` rung (rank order guarantees it) even without
|
||||
// a linked account, matching the existing rule that /player/* is role-agnostic
|
||||
// self-service.
|
||||
|
||||
// Same TTL as the config cache: this decides a privilege rung, so an unlinked
|
||||
// (or newly relinked) account must not keep the old answer for long. Anonymous,
|
||||
// staff and admin callers short-circuit before this runs, so the lookup only
|
||||
// costs a query on the logged-in-member path.
|
||||
const LINK_TTL_MS = CONFIG_TTL_MS
|
||||
const linkCache = new Map() // userId → { hasLink, at }
|
||||
|
||||
async function hasLinkedAccount(userId) {
|
||||
const hit = linkCache.get(userId)
|
||||
const now = Date.now()
|
||||
if (hit && now - hit.at < LINK_TTL_MS) return hit.hasLink
|
||||
let hasLink = false
|
||||
try {
|
||||
const links = await shardLinks.listForUser(userId)
|
||||
hasLink = Array.isArray(links) && links.length > 0
|
||||
} catch (err) {
|
||||
log.warn('hasLinkedAccount failed; treating as unlinked', { message: err.message })
|
||||
}
|
||||
linkCache.set(userId, { hasLink, at: now })
|
||||
return hasLink
|
||||
}
|
||||
|
||||
// Drop a user's cached link status (called when a link is created or removed so
|
||||
// the rung takes effect immediately rather than up to LINK_TTL_MS later).
|
||||
const forgetUser = (userId) => linkCache.delete(userId)
|
||||
|
||||
async function viewerLevel(req) {
|
||||
const viewer = req.user || auth.getUserFromRequest(req)
|
||||
if (!viewer) return 'anonymous'
|
||||
if (viewer.role === 'admin') return 'admin'
|
||||
if (viewer.role === 'moderator') return 'staff'
|
||||
return (await hasLinkedAccount(viewer.id)) ? 'player' : 'logged_in'
|
||||
}
|
||||
|
||||
// ── Enforcement ────────────────────────────────────────────────────────────
|
||||
|
||||
// Route gate. 404 when the feature is disabled (do not leak that it exists);
|
||||
// 403 when it exists but the viewer sits below its audience. Stashes the
|
||||
// resolved level on the request so controllers can project without re-resolving.
|
||||
function requireFeature(name) {
|
||||
return async (req, res, next) => {
|
||||
try {
|
||||
const config = await getConfig()
|
||||
const feature = config[name]
|
||||
if (!feature || !feature.enabled) return res.status(404).json({ message: 'Not Found' })
|
||||
const level = await viewerLevel(req)
|
||||
req.viewerLevel = level
|
||||
if (!meets(level, feature.audience)) return res.status(403).json({ message: 'Forbidden' })
|
||||
return next()
|
||||
} catch (err) {
|
||||
log.error(`requireFeature(${name})`, err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Strip the fields a viewer at `level` may not see. Applies the locked rules
|
||||
// first (so acct/webId can never survive below admin), then the feature's
|
||||
// configured field rules. Recurses into arrays and nested objects because the
|
||||
// sensitive fields sit inside actor sub-objects (guild.leader, city.governor).
|
||||
// Only ARRAYS and PLAIN objects are walked. A Date, Buffer or other class
|
||||
// instance is a value, not a bag of fields: rebuilding one key-by-key would
|
||||
// return `{}` (a Date has no enumerable own properties), which is how the DB-
|
||||
// backed read models — whose rows carry real Date columns — differ from the
|
||||
// pure-JSON wire frames the projection was first written against.
|
||||
const isPlainObject = (v) => {
|
||||
if (v === null || typeof v !== 'object') return false
|
||||
const proto = Object.getPrototypeOf(v)
|
||||
return proto === Object.prototype || proto === null
|
||||
}
|
||||
|
||||
function projectValue(value, rules, level) {
|
||||
if (Array.isArray(value)) return value.map((v) => projectValue(v, rules, level))
|
||||
if (!isPlainObject(value)) return value
|
||||
const out = {}
|
||||
for (const [key, v] of Object.entries(value)) {
|
||||
// Locked fields are checked by meaning first, so no configured rule (and no
|
||||
// flattened spelling) can widen them past `admin`.
|
||||
const required = isLockedField(key) ? 'admin' : rules[key]
|
||||
if (required && !meets(level, required)) continue
|
||||
out[key] = projectValue(v, rules, level)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Project a payload for one feature. `level` defaults to admin-equivalent only
|
||||
// when explicitly passed; callers should always pass a resolved level.
|
||||
function projectFeature(name, payload, level, config) {
|
||||
const feature = config?.[name]
|
||||
const rules = { ...LOCKED_FIELDS, ...(feature ? feature.fields : {}) }
|
||||
return projectValue(payload, rules, level)
|
||||
}
|
||||
|
||||
// Convenience for controllers: resolve config once, project, return.
|
||||
async function project(name, payload, req) {
|
||||
const config = await getConfig()
|
||||
const level = req.viewerLevel || (await viewerLevel(req))
|
||||
return projectFeature(name, payload, level, config)
|
||||
}
|
||||
|
||||
// Is this event kind allowed to reach a viewer at `level`? Fail closed on an
|
||||
// unmapped kind (rule 2), and honour both the feature gate and its stream flag.
|
||||
function kindVisibleTo(kind, level, config) {
|
||||
if (level === 'admin') return true
|
||||
const name = KIND_FEATURE.get(kind)
|
||||
if (!name) return false // rule 2: unmapped ⇒ admin-only
|
||||
const feature = config?.[name]
|
||||
if (!feature || !feature.enabled || !feature.stream) return false
|
||||
return meets(level, feature.audience)
|
||||
}
|
||||
|
||||
// The event kinds a viewer at `level` may read under the CURRENT config. This is
|
||||
// the live counterpart of PUBLIC_KINDS, which is a module-load constant derived
|
||||
// from the compiled DEFAULTS and therefore cannot answer "may THIS viewer see
|
||||
// this kind, given what the admin has configured?".
|
||||
//
|
||||
// Deliberately ignores the `stream` flag: that governs SSE fan-out only, so a
|
||||
// feature whose live firehose is off (market) is still readable from the stored
|
||||
// history. Unmapped kinds are absent by construction (rule 2).
|
||||
function visibleKinds(level, config) {
|
||||
return [...KIND_FEATURE.entries()]
|
||||
.filter(([, name]) => {
|
||||
const feature = config?.[name]
|
||||
return !!feature && feature.enabled && meets(level, feature.audience)
|
||||
})
|
||||
.map(([kind]) => kind)
|
||||
}
|
||||
|
||||
// The features a viewer at `level` can actually see — drives SPA nav so it never
|
||||
// renders a link that would 403.
|
||||
function visibleFeatures(level, config) {
|
||||
return FEATURE_NAMES.filter((name) => {
|
||||
const feature = config[name]
|
||||
return feature.enabled && meets(level, feature.audience)
|
||||
})
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
LADDER,
|
||||
FEATURES,
|
||||
FEATURE_NAMES,
|
||||
LOCKED_FIELDS,
|
||||
KIND_FEATURE,
|
||||
PUBLIC_KINDS,
|
||||
DEFAULT_STREAM_OFF,
|
||||
isLevel,
|
||||
isFeature,
|
||||
isLockedField,
|
||||
rank,
|
||||
meets,
|
||||
getConfig,
|
||||
invalidate,
|
||||
compileDefaults,
|
||||
viewerLevel,
|
||||
forgetUser,
|
||||
requireFeature,
|
||||
projectFeature,
|
||||
project,
|
||||
kindVisibleTo,
|
||||
visibleKinds,
|
||||
visibleFeatures,
|
||||
}
|
||||
@@ -1,686 +0,0 @@
|
||||
// Spawn atlas parsers — pure functions over strings, no `fs`, no dependencies.
|
||||
//
|
||||
// These back the CLI build script (`scripts/buildSpawnAtlas.js`), which is the
|
||||
// only thing that reads a ServUO tree. Keeping every parser pure and fs-free is
|
||||
// what lets the test suite cover them in CI, where no ServUO tree exists: the
|
||||
// tests hand these functions literal XML strings.
|
||||
//
|
||||
// Four source shapes, two very different parsing strategies:
|
||||
//
|
||||
// Spawns/*.xml ~10.5 MB across 13 files, FLAT <Points> records
|
||||
// → streaming regex, never a DOM. See parsePoints().
|
||||
// Data/Regions.xml 129 KB, genuinely nested <region> inside <region>
|
||||
// Data/Locations/*.xml nested <parent>/<child>
|
||||
// Config/ChampionSpawns.xml 4.8 KB, <spawn>/<location>
|
||||
// → the small recursive tokenizer below.
|
||||
//
|
||||
// The server has zero XML dependencies and this adds none. The tokenizer is
|
||||
// deliberately a *subset* parser: it handles the constructs these four files
|
||||
// actually use (elements, attributes, self-closing tags, comments, the XML
|
||||
// declaration, CDATA, the five predefined entities plus numeric refs) and
|
||||
// nothing else. It is not a general-purpose XML parser and must not be reused
|
||||
// as one — no namespaces, no DTDs, no entity declarations.
|
||||
|
||||
// ── Entities ───────────────────────────────────────────────────────────────
|
||||
|
||||
const NAMED_ENTITIES = {
|
||||
amp: '&',
|
||||
lt: '<',
|
||||
gt: '>',
|
||||
quot: '"',
|
||||
apos: "'",
|
||||
}
|
||||
|
||||
// Region and location names carry apostrophes ("Mondain's Legacy", "Wrong's
|
||||
// Level 3"), so entity decoding is load-bearing here, not decorative.
|
||||
function decodeEntities(text) {
|
||||
if (!text.includes('&')) return text
|
||||
return text.replace(/&(#x?[0-9a-fA-F]+|[a-zA-Z]+);/g, (match, body) => {
|
||||
if (body[0] === '#') {
|
||||
const code =
|
||||
body[1] === 'x' || body[1] === 'X'
|
||||
? Number.parseInt(body.slice(2), 16)
|
||||
: Number.parseInt(body.slice(1), 10)
|
||||
return Number.isFinite(code) ? String.fromCodePoint(code) : match
|
||||
}
|
||||
const named = NAMED_ENTITIES[body.toLowerCase()]
|
||||
return named === undefined ? match : named
|
||||
})
|
||||
}
|
||||
|
||||
// ── The tokenizer ──────────────────────────────────────────────────────────
|
||||
|
||||
const ATTR_RE = /([\w:.-]+)\s*=\s*("([^"]*)"|'([^']*)')/g
|
||||
|
||||
function parseAttrs(source) {
|
||||
const attrs = {}
|
||||
ATTR_RE.lastIndex = 0
|
||||
let match
|
||||
while ((match = ATTR_RE.exec(source)) !== null) {
|
||||
const raw = match[3] !== undefined ? match[3] : match[4]
|
||||
attrs[match[1]] = decodeEntities(raw)
|
||||
}
|
||||
return attrs
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a small nested XML document into `{ name, attrs, children, text }`.
|
||||
*
|
||||
* Intended for Regions.xml / Locations / ChampionSpawns.xml only — never for
|
||||
* the multi-megabyte Spawns files. Returns the root element, or `null` for a
|
||||
* document with no elements.
|
||||
*
|
||||
* Mismatched or stray closing tags are ignored rather than thrown on: these are
|
||||
* hand-maintained shard config files, and one malformed region should degrade
|
||||
* to a missing region, not abort a build that is otherwise fine.
|
||||
*/
|
||||
function parseXml(source) {
|
||||
const text = String(source)
|
||||
const root = { name: '#document', attrs: {}, children: [], text: '' }
|
||||
const stack = [root]
|
||||
let i = 0
|
||||
|
||||
while (i < text.length) {
|
||||
const lt = text.indexOf('<', i)
|
||||
if (lt === -1) {
|
||||
appendText(stack[stack.length - 1], text.slice(i))
|
||||
break
|
||||
}
|
||||
if (lt > i) appendText(stack[stack.length - 1], text.slice(i, lt))
|
||||
|
||||
// Comment, declaration/DOCTYPE, or CDATA — skipped wholesale.
|
||||
if (text.startsWith('<!--', lt)) {
|
||||
const end = text.indexOf('-->', lt + 4)
|
||||
i = end === -1 ? text.length : end + 3
|
||||
continue
|
||||
}
|
||||
if (text.startsWith('<![CDATA[', lt)) {
|
||||
const end = text.indexOf(']]>', lt + 9)
|
||||
const stop = end === -1 ? text.length : end
|
||||
appendRawText(stack[stack.length - 1], text.slice(lt + 9, stop))
|
||||
i = end === -1 ? text.length : end + 3
|
||||
continue
|
||||
}
|
||||
if (text.startsWith('<?', lt)) {
|
||||
const end = text.indexOf('?>', lt + 2)
|
||||
i = end === -1 ? text.length : end + 2
|
||||
continue
|
||||
}
|
||||
if (text.startsWith('<!', lt)) {
|
||||
const end = text.indexOf('>', lt + 2)
|
||||
i = end === -1 ? text.length : end + 1
|
||||
continue
|
||||
}
|
||||
|
||||
const gt = findTagEnd(text, lt)
|
||||
if (gt === -1) {
|
||||
// Unterminated tag: nothing sane is left to read.
|
||||
break
|
||||
}
|
||||
const inner = text.slice(lt + 1, gt)
|
||||
|
||||
if (inner[0] === '/') {
|
||||
const name = inner.slice(1).trim()
|
||||
// Pop to the nearest matching open element. If there is no match the tag
|
||||
// is stray and we drop it rather than unwinding the whole stack.
|
||||
for (let depth = stack.length - 1; depth > 0; depth -= 1) {
|
||||
if (stack[depth].name === name) {
|
||||
stack.length = depth
|
||||
break
|
||||
}
|
||||
}
|
||||
i = gt + 1
|
||||
continue
|
||||
}
|
||||
|
||||
const selfClosing = inner.endsWith('/')
|
||||
const body = selfClosing ? inner.slice(0, -1) : inner
|
||||
const space = body.search(/\s/)
|
||||
const name = (space === -1 ? body : body.slice(0, space)).trim()
|
||||
const node = {
|
||||
name,
|
||||
attrs: space === -1 ? {} : parseAttrs(body.slice(space)),
|
||||
children: [],
|
||||
text: '',
|
||||
}
|
||||
stack[stack.length - 1].children.push(node)
|
||||
if (!selfClosing) stack.push(node)
|
||||
i = gt + 1
|
||||
}
|
||||
|
||||
return root.children.length > 0 ? root.children[0] : null
|
||||
}
|
||||
|
||||
// `>` inside a quoted attribute value must not end the tag.
|
||||
function findTagEnd(text, from) {
|
||||
let quote = null
|
||||
for (let i = from + 1; i < text.length; i += 1) {
|
||||
const ch = text[i]
|
||||
if (quote) {
|
||||
if (ch === quote) quote = null
|
||||
} else if (ch === '"' || ch === "'") {
|
||||
quote = ch
|
||||
} else if (ch === '>') {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
function appendText(node, chunk) {
|
||||
if (chunk.trim() === '') return
|
||||
appendRawText(node, decodeEntities(chunk))
|
||||
}
|
||||
|
||||
function appendRawText(node, chunk) {
|
||||
node.text = node.text ? `${node.text}${chunk}` : chunk
|
||||
}
|
||||
|
||||
function childrenNamed(node, name) {
|
||||
if (!node || !node.children) return []
|
||||
return node.children.filter((child) => child.name === name)
|
||||
}
|
||||
|
||||
// ── Facet names ────────────────────────────────────────────────────────────
|
||||
//
|
||||
// Facets are NOT a fixed list. A shard may add facets, replace them wholesale,
|
||||
// or rename them when its maps are updated, so nothing here may name Felucca,
|
||||
// Trammel or any other stock facet. The facet set is whatever the shard's own
|
||||
// files say it is, discovered at parse time.
|
||||
//
|
||||
// The complication is that the sources disagree about spelling for the SAME
|
||||
// facet and nothing in the files reconciles them: `Spawns/*.xml` `<Map>` and
|
||||
// `Regions.xml` `<Facet name>` say `TerMur`, while `Data/Locations/*.xml` spells
|
||||
// it `Ter Mur` and calls Tokuno `Tokuno Islands`. Left unreconciled this fails
|
||||
// silently — the landmark bucket is keyed differently from the points looking it
|
||||
// up, so the fallback never fires and every unregioned spawn on those facets
|
||||
// reads "Wilderness".
|
||||
//
|
||||
// Reconciliation is therefore done by MATCHING, not by a lookup table:
|
||||
// `facetKey()` collapses spelling differences, and `resolveFacetName()` matches
|
||||
// a loosely-spelled name against the canonical set discovered from the shard's
|
||||
// own data. A facet nobody else mentions keeps its own name rather than being
|
||||
// dropped.
|
||||
|
||||
/**
|
||||
* Collapse a facet name to a comparison key: lowercase, alphanumerics only.
|
||||
* `TerMur`, `Ter Mur` and `ter-mur` all key alike.
|
||||
*/
|
||||
function facetKey(value) {
|
||||
return String(value ?? '')
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, '')
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a key → canonical-spelling lookup from the authoritative facet names.
|
||||
*
|
||||
* The authority is what the spawn records and region definitions actually say,
|
||||
* since those are the names the atlas keys everything on. Later names do not
|
||||
* overwrite earlier ones, so the first source wins consistently.
|
||||
*/
|
||||
function buildFacetIndex(names) {
|
||||
const index = new Map()
|
||||
for (const name of names) {
|
||||
const key = facetKey(name)
|
||||
if (key !== '' && !index.has(key)) index.set(key, String(name).trim())
|
||||
}
|
||||
return index
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a loosely-spelled facet name against the discovered canonical set.
|
||||
*
|
||||
* Tried in order: exact key match (`Ter Mur` → `TerMur`), then a prefix match in
|
||||
* either direction (`Tokuno Islands` → `Tokuno`), longest candidate first so a
|
||||
* more specific facet wins over a shorter one that merely prefixes it.
|
||||
*
|
||||
* A name matching nothing is returned trimmed rather than dropped — on a shard
|
||||
* with a custom facet that is a real facet the atlas simply has no spawns for
|
||||
* yet, and inventing a match would be worse than leaving it alone.
|
||||
*/
|
||||
function resolveFacetName(value, index) {
|
||||
const raw = String(value ?? '').trim()
|
||||
const key = facetKey(raw)
|
||||
if (key === '') return ''
|
||||
if (index.has(key)) return index.get(key)
|
||||
|
||||
let best = null
|
||||
for (const [candidateKey, canonical] of index) {
|
||||
if (!key.startsWith(candidateKey) && !candidateKey.startsWith(key)) continue
|
||||
if (best === null || candidateKey.length > facetKey(best).length) best = canonical
|
||||
}
|
||||
return best ?? raw
|
||||
}
|
||||
|
||||
// ── Small coercions ────────────────────────────────────────────────────────
|
||||
|
||||
function toInt(value, fallback = 0) {
|
||||
const n = Number.parseInt(value, 10)
|
||||
return Number.isFinite(n) ? n : fallback
|
||||
}
|
||||
|
||||
function toBool(value) {
|
||||
return String(value).trim().toLowerCase() === 'true'
|
||||
}
|
||||
|
||||
/**
|
||||
* URL-safe slug used as the creature primary key and in `/atlas/:slug`.
|
||||
* Spawn type tokens are C# class names, so they are already ASCII-ish; this
|
||||
* mainly lowercases and collapses punctuation.
|
||||
*/
|
||||
function slugify(value) {
|
||||
return String(value)
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, '-')
|
||||
.replace(/^-+|-+$/g, '')
|
||||
}
|
||||
|
||||
// ── Objects2 ───────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Parse a `<Objects2>` value into `[{ type, max }]`.
|
||||
*
|
||||
* The format is one or more segments joined by `:OBJ=`, each segment being
|
||||
* `Type:MX=n:SB=0:RT=0:...` — the type is the token before the first `:`, and
|
||||
* every following token is a `KEY=value` pair. Verified against trammel.xml,
|
||||
* where a single point carries six types:
|
||||
*
|
||||
* Giantserpent:MX=1:...:OBJ=Giantspider:MX=1:...:OBJ=Boar:MX=1:...
|
||||
*
|
||||
* Splitting on `:` alone would shred this, which is why the `:OBJ=` split comes
|
||||
* first. `MX` is that type's own max count and is what the atlas displays;
|
||||
* every other flag (spawn/trigger/refractory bookkeeping) is dropped.
|
||||
*
|
||||
* The type token itself may carry XmlSpawner directives appended to the class
|
||||
* name — property assignments after `/` and an amount/argument list after `,`:
|
||||
*
|
||||
* Agralem/Name/Agralem alchemist/z/-50 Fairy,{RND,4,8}
|
||||
* GargishRefugee/hue/34532 greatape,true GargishRouser,1
|
||||
*
|
||||
* Taken literally these produce creatures that do not exist ("alchemist/z/-50")
|
||||
* AND split real ones in two, because `Fairy` and `Fairy,{RND,4,8}` slug apart —
|
||||
* 71 of 845 entries were affected before this was stripped. Only the leading
|
||||
* class name identifies the creature, so everything from the first `/` or `,`
|
||||
* is dropped.
|
||||
*/
|
||||
/** Reduce an XmlSpawner type token to the bare class name. */
|
||||
function stripSpawnerDirectives(token) {
|
||||
const cut = String(token).search(/[/,]/)
|
||||
return (cut === -1 ? String(token) : String(token).slice(0, cut)).trim()
|
||||
}
|
||||
|
||||
function parseObjects2(value) {
|
||||
const source = String(value ?? '').trim()
|
||||
if (source === '') return []
|
||||
|
||||
return source
|
||||
.split(':OBJ=')
|
||||
.map((segment) => {
|
||||
const tokens = segment.split(':')
|
||||
const type = stripSpawnerDirectives(tokens.shift() ?? '')
|
||||
if (type === '') return null
|
||||
let max = 1
|
||||
for (const token of tokens) {
|
||||
const eq = token.indexOf('=')
|
||||
if (eq === -1) continue
|
||||
if (token.slice(0, eq).trim().toUpperCase() === 'MX') {
|
||||
max = toInt(token.slice(eq + 1), 1)
|
||||
}
|
||||
}
|
||||
return { type, max }
|
||||
})
|
||||
.filter((entry) => entry !== null)
|
||||
}
|
||||
|
||||
// ── Spawns/*.xml ───────────────────────────────────────────────────────────
|
||||
|
||||
const POINT_RE = /<Points>([\s\S]*?)<\/Points>/g
|
||||
|
||||
function tagValue(block, name) {
|
||||
const match = block.match(new RegExp(`<${name}>([\\s\\S]*?)</${name}>`))
|
||||
return match ? decodeEntities(match[1]).trim() : ''
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a `Spawns/<facet>.xml` file into spawn point records.
|
||||
*
|
||||
* Deliberately regex/streaming and NOT `parseXml` — these files total ~10.5 MB
|
||||
* and putting them through a DOM builder would allocate a node per element for
|
||||
* ~40 fields on every one of ~6,500 records to keep 14 of them. The records are
|
||||
* flat, so a per-record regex sweep is both correct and cheap.
|
||||
*
|
||||
* Only the fields the site can actually show are kept. Everything to do with
|
||||
* triggering, refractory windows, proximity, sequential spawning, sounds and
|
||||
* `UniqueId` is dropped here rather than downstream — that is what holds the
|
||||
* committed artifact under 1 MB.
|
||||
*
|
||||
* NOTE: the facet comes from each record's own `<Map>`, never from the file
|
||||
* name. `Eodon.xml`, `GravewaterLake.xml` and the other named-area files all
|
||||
* carry TerMur/Trammel points, so there are 13 files but only 6 facets.
|
||||
*/
|
||||
/**
|
||||
* A spawner's respawn window, in seconds.
|
||||
*
|
||||
* `DelayInSec` decides the unit of `MinDelay`/`MaxDelay`; absent (older files)
|
||||
* it is false, which is minutes — the same default XmlSpawner assumes.
|
||||
*/
|
||||
function delaySeconds(block) {
|
||||
const scale = toBool(tagValue(block, 'DelayInSec')) ? 1 : 60
|
||||
return {
|
||||
minDelay: toInt(tagValue(block, 'MinDelay')) * scale,
|
||||
maxDelay: toInt(tagValue(block, 'MaxDelay')) * scale,
|
||||
}
|
||||
}
|
||||
|
||||
function parsePoints(source) {
|
||||
const text = String(source)
|
||||
const points = []
|
||||
POINT_RE.lastIndex = 0
|
||||
let match
|
||||
|
||||
while ((match = POINT_RE.exec(text)) !== null) {
|
||||
const block = match[1]
|
||||
// Reported exactly as written. `<Map>` is the authority the rest of the
|
||||
// atlas keys on, so it is never rewritten.
|
||||
const facet = tagValue(block, 'Map')
|
||||
if (facet === '') continue
|
||||
|
||||
points.push({
|
||||
name: tagValue(block, 'Name'),
|
||||
facet,
|
||||
x: toInt(tagValue(block, 'X')),
|
||||
y: toInt(tagValue(block, 'Y')),
|
||||
width: toInt(tagValue(block, 'Width')),
|
||||
height: toInt(tagValue(block, 'Height')),
|
||||
range: toInt(tagValue(block, 'Range')),
|
||||
maxCount: toInt(tagValue(block, 'MaxCount')),
|
||||
// Normalised to SECONDS here, because the unit is per-record. XmlSpawner
|
||||
// writes minutes by default and switches to seconds only when a spawner's
|
||||
// delay does not divide into whole minutes, flagging that with
|
||||
// `DelayInSec` (XmlSpawner2.cs:7462-7480, read back at :6345-6358). Taken
|
||||
// literally the two are indistinguishable — a `5` means five minutes on
|
||||
// one spawner and five seconds on the next — so a consumer that assumed
|
||||
// either unit would be wrong about the other. Stock ServUO 57.4 has ~30
|
||||
// second-flagged spawners, few enough to look like noise and quietly
|
||||
// mislabel.
|
||||
...delaySeconds(block),
|
||||
// Time-of-day gating: TODMode 0 means "always", in which case the start
|
||||
// and end values are meaningless and the site must not render them.
|
||||
todStart: toInt(tagValue(block, 'TODStart')),
|
||||
todEnd: toInt(tagValue(block, 'TODEnd')),
|
||||
todMode: toInt(tagValue(block, 'TODMode')),
|
||||
// A spawner switched off in-world spawns nothing; the build filters these
|
||||
// out so the atlas describes what actually appears, not what is merely
|
||||
// configured. Parsed here so the decision stays in the build script.
|
||||
running: toBool(tagValue(block, 'IsRunning')),
|
||||
types: parseObjects2(tagValue(block, 'Objects2')),
|
||||
})
|
||||
}
|
||||
|
||||
return points
|
||||
}
|
||||
|
||||
// ── Data/Regions.xml ───────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Flatten `Data/Regions.xml` into `[{ facet, name, type, priority, parent, rects }]`.
|
||||
*
|
||||
* Regions nest: a `<region>` may contain further `<region>` elements, and the
|
||||
* inner ones frequently omit `name` and `priority` (`<region type="CrystalField">`
|
||||
* inside "Prism of Light"). Unnamed regions are skipped — they cannot label a
|
||||
* spawn point — but their children are still walked, and a child that omits
|
||||
* `priority` inherits its parent's rather than defaulting to 0, which would
|
||||
* quietly sort it below every top-level region.
|
||||
*/
|
||||
function parseRegions(source) {
|
||||
const root = parseXml(source)
|
||||
const regions = []
|
||||
if (!root) return regions
|
||||
|
||||
for (const facetNode of childrenNamed(root, 'Facet')) {
|
||||
const facet = (facetNode.attrs.name || '').trim()
|
||||
if (facet === '') continue
|
||||
walkRegions(facetNode, facet, null, 0, regions)
|
||||
}
|
||||
return regions
|
||||
}
|
||||
|
||||
function walkRegions(node, facet, parentName, parentPriority, out) {
|
||||
for (const regionNode of childrenNamed(node, 'region')) {
|
||||
const name = regionNode.attrs.name || ''
|
||||
const priority = Object.hasOwn(regionNode.attrs, 'priority')
|
||||
? toInt(regionNode.attrs.priority, parentPriority)
|
||||
: parentPriority
|
||||
|
||||
if (name !== '') {
|
||||
const rects = childrenNamed(regionNode, 'rect').map((rect) => ({
|
||||
x: toInt(rect.attrs.x),
|
||||
y: toInt(rect.attrs.y),
|
||||
width: toInt(rect.attrs.width),
|
||||
height: toInt(rect.attrs.height),
|
||||
}))
|
||||
// A named region with no rects (some exist purely to carry music or a
|
||||
// `go` point) can never contain anything, so it is not worth indexing.
|
||||
if (rects.length > 0) {
|
||||
out.push({
|
||||
facet,
|
||||
name,
|
||||
type: regionNode.attrs.type || '',
|
||||
priority,
|
||||
parent: parentName,
|
||||
rects,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
walkRegions(regionNode, facet, name === '' ? parentName : name, priority, out)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Data/Locations/*.xml ───────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Flatten a `Data/Locations/<facet>.xml` into landmark points.
|
||||
*
|
||||
* The file nests `<parent>` arbitrarily deep and puts coordinates only on
|
||||
* `<child>`: Trammel → Dungeons → Covetous → "Level 1". The outermost parent is
|
||||
* the facet itself and is dropped from `path`; `group` is the innermost
|
||||
* enclosing parent ("Covetous"), which is the label worth showing — "Covetous"
|
||||
* reads better than "Level 1" when naming where a spawn is.
|
||||
*/
|
||||
function parseLocations(source, facetHint = '') {
|
||||
const root = parseXml(source)
|
||||
const landmarks = []
|
||||
if (!root) return landmarks
|
||||
|
||||
for (const top of childrenNamed(root, 'parent')) {
|
||||
// The file name (`Data/Locations/termur.xml`) is the more reliable signal
|
||||
// and is preferred over the display label inside the file, which is where
|
||||
// the `Ter Mur` / `Tokuno Islands` drift lives. Both are carried so the
|
||||
// build can fall back to matching the label if the file name resolves to
|
||||
// nothing — a shard may well name its files differently from its facets.
|
||||
landmarks.push(
|
||||
...collectLocations(top, facetHint || top.attrs.name || '', top.attrs.name || ''),
|
||||
)
|
||||
}
|
||||
return landmarks
|
||||
}
|
||||
|
||||
function collectLocations(top, facet, label) {
|
||||
const out = []
|
||||
walkLocations(top, facet, [], out)
|
||||
for (const landmark of out) landmark.facetLabel = label
|
||||
return out
|
||||
}
|
||||
|
||||
function walkLocations(node, facet, path, out) {
|
||||
for (const child of childrenNamed(node, 'child')) {
|
||||
const name = child.attrs.name || ''
|
||||
if (name === '') continue
|
||||
out.push({
|
||||
facet,
|
||||
name,
|
||||
group: path.length > 0 ? path[path.length - 1] : name,
|
||||
path: [...path],
|
||||
x: toInt(child.attrs.x),
|
||||
y: toInt(child.attrs.y),
|
||||
z: toInt(child.attrs.z),
|
||||
})
|
||||
}
|
||||
for (const parent of childrenNamed(node, 'parent')) {
|
||||
const name = parent.attrs.name || ''
|
||||
walkLocations(parent, facet, name === '' ? path : [...path, name], out)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Config/ChampionSpawns.xml ──────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Parse `Config/ChampionSpawns.xml` into champion altar records.
|
||||
*
|
||||
* This is the shard's *configured* champion roster — which altars exist, where,
|
||||
* and which type each is pinned to. It is static content and distinct from the
|
||||
* live `champ.update` feed the bridge already carries: this says "there is an
|
||||
* Unholy Terror altar in Deceit", the feed says "it is on level 3 right now".
|
||||
*
|
||||
* A spawn with no `type` is randomised on every activation, which the site must
|
||||
* render as "random" rather than as an empty type.
|
||||
*/
|
||||
function parseChampions(source) {
|
||||
const root = parseXml(source)
|
||||
const champions = []
|
||||
if (!root) return champions
|
||||
|
||||
for (const spawnNode of childrenNamed(root, 'spawn')) {
|
||||
const location = childrenNamed(spawnNode, 'location')[0]
|
||||
const attrs = location ? location.attrs : {}
|
||||
champions.push({
|
||||
name: spawnNode.attrs.name || '',
|
||||
group: spawnNode.attrs.group || '',
|
||||
type: spawnNode.attrs.type || '',
|
||||
randomType: !spawnNode.attrs.type,
|
||||
facet: (attrs.map || '').trim(),
|
||||
x: toInt(attrs.x),
|
||||
y: toInt(attrs.y),
|
||||
z: toInt(attrs.z),
|
||||
radius: toInt(attrs.radius),
|
||||
})
|
||||
}
|
||||
return champions
|
||||
}
|
||||
|
||||
// ── Placement ──────────────────────────────────────────────────────────────
|
||||
|
||||
const DEFAULT_LANDMARK_RADIUS = 200
|
||||
|
||||
function inRect(x, y, rect) {
|
||||
return (
|
||||
x >= rect.x && x < rect.x + rect.width && y >= rect.y && y < rect.y + rect.height
|
||||
)
|
||||
}
|
||||
|
||||
function rectArea(rect) {
|
||||
return Math.max(1, rect.width) * Math.max(1, rect.height)
|
||||
}
|
||||
|
||||
/**
|
||||
* Group parsed regions and landmarks by facet once, so the per-point resolve
|
||||
* below is a scan of one facet instead of the whole world. With ~6,500 points
|
||||
* and a few thousand rects this stays comfortably sub-second; there is no need
|
||||
* for a spatial index and none is worth the complexity.
|
||||
*/
|
||||
function buildPlacementIndex(regions, landmarks) {
|
||||
const byFacet = new Map()
|
||||
// Keyed on facetKey(), not the raw name, so two spellings of one facet cannot
|
||||
// land in separate buckets — the failure that silently emptied the landmark
|
||||
// bucket for Ter Mur and Tokuno.
|
||||
const facet = (name) => {
|
||||
const key = facetKey(name)
|
||||
if (!byFacet.has(key)) byFacet.set(key, { regions: [], landmarks: [] })
|
||||
return byFacet.get(key)
|
||||
}
|
||||
for (const region of regions) facet(region.facet).regions.push(region)
|
||||
for (const landmark of landmarks) facet(landmark.facet).landmarks.push(landmark)
|
||||
return byFacet
|
||||
}
|
||||
|
||||
/**
|
||||
* Turn a raw coordinate into a human place name.
|
||||
*
|
||||
* This is the transform the whole atlas exists for: it is what makes a row read
|
||||
* "Lizardman — Despise, Felucca" instead of "Lizardman — 5411, 1234".
|
||||
*
|
||||
* Resolution order:
|
||||
* 1. The highest-`priority` named region whose rect contains the point. Ties
|
||||
* break toward the SMALLEST rect, so a specific room inside a dungeon wins
|
||||
* over the dungeon-wide rect it sits in.
|
||||
* 2. Otherwise the nearest landmark within `landmarkRadius` tiles, labelled by
|
||||
* its group ("Covetous"), not the individual marker ("Level 1").
|
||||
* 3. Otherwise "Wilderness". The radius cap is what keeps step 3 reachable —
|
||||
* without it the nearest landmark is always *some* landmark, however far,
|
||||
* and open countryside would get labelled with a dungeon on the far side
|
||||
* of the map.
|
||||
*/
|
||||
function resolveRegion(x, y, facetName, index, options = {}) {
|
||||
const radius = options.landmarkRadius ?? DEFAULT_LANDMARK_RADIUS
|
||||
const bucket = index.get(facetKey(facetName))
|
||||
const result = { region: null, landmark: null, label: 'Wilderness' }
|
||||
if (!bucket) return result
|
||||
|
||||
let best = null
|
||||
let bestPriority = -Infinity
|
||||
let bestArea = Infinity
|
||||
for (const region of bucket.regions) {
|
||||
for (const rect of region.rects) {
|
||||
if (!inRect(x, y, rect)) continue
|
||||
const area = rectArea(rect)
|
||||
if (region.priority > bestPriority || (region.priority === bestPriority && area < bestArea)) {
|
||||
best = region
|
||||
bestPriority = region.priority
|
||||
bestArea = area
|
||||
}
|
||||
}
|
||||
}
|
||||
if (best) {
|
||||
result.region = best.name
|
||||
result.label = best.name
|
||||
return result
|
||||
}
|
||||
|
||||
let nearest = null
|
||||
let nearestDistance = Infinity
|
||||
const limit = radius * radius
|
||||
for (const landmark of bucket.landmarks) {
|
||||
const dx = landmark.x - x
|
||||
const dy = landmark.y - y
|
||||
const distance = dx * dx + dy * dy
|
||||
if (distance < nearestDistance) {
|
||||
nearest = landmark
|
||||
nearestDistance = distance
|
||||
}
|
||||
}
|
||||
if (nearest && nearestDistance <= limit) {
|
||||
result.landmark = nearest.group || nearest.name
|
||||
result.label = result.landmark
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
parseXml,
|
||||
parseObjects2,
|
||||
parsePoints,
|
||||
parseRegions,
|
||||
parseLocations,
|
||||
parseChampions,
|
||||
buildPlacementIndex,
|
||||
resolveRegion,
|
||||
facetKey,
|
||||
buildFacetIndex,
|
||||
resolveFacetName,
|
||||
slugify,
|
||||
decodeEntities,
|
||||
DEFAULT_LANDMARK_RADIUS,
|
||||
}
|
||||
@@ -1,336 +0,0 @@
|
||||
// Spawn atlas — the filesystem layer over a ServUO tree.
|
||||
//
|
||||
// `spawnAtlasParse.js` holds the pure parsers; this module is the only thing
|
||||
// that touches a ServUO tree on disk, and it is shared by both callers:
|
||||
//
|
||||
// - the server, which refreshes the atlas on boot (`shardAtlas.model.js`)
|
||||
// - the CLI (`scripts/importSpawnAtlas.js`)
|
||||
//
|
||||
// The shard's own files are the single source of truth. Nothing is precomputed
|
||||
// and committed, because a shard's maps change over its lifetime — facets get
|
||||
// added, replaced or renamed — and a snapshot in the repo would silently go
|
||||
// stale against the world players actually see.
|
||||
//
|
||||
// Reading and hashing the whole tree costs ~120 ms and a full parse ~400 ms, so
|
||||
// the boot path hashes first and only parses when something actually changed.
|
||||
|
||||
const crypto = require('crypto')
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
|
||||
const {
|
||||
parsePoints,
|
||||
parseRegions,
|
||||
parseLocations,
|
||||
parseChampions,
|
||||
buildPlacementIndex,
|
||||
buildFacetIndex,
|
||||
resolveFacetName,
|
||||
resolveRegion,
|
||||
facetKey,
|
||||
slugify,
|
||||
} = require('./spawnAtlasParse')
|
||||
|
||||
const REGIONS_FILE = path.join('Data', 'Regions.xml')
|
||||
const LOCATIONS_DIR = path.join('Data', 'Locations')
|
||||
const SPAWNS_DIR = 'Spawns'
|
||||
const CHAMPIONS_FILE = path.join('Config', 'ChampionSpawns.xml')
|
||||
|
||||
class AtlasSourceError extends Error {
|
||||
constructor(message, code) {
|
||||
super(message)
|
||||
this.name = 'AtlasSourceError'
|
||||
this.code = code
|
||||
}
|
||||
}
|
||||
|
||||
// ── Reading ────────────────────────────────────────────────────────────────
|
||||
|
||||
function sha256(text) {
|
||||
return crypto.createHash('sha256').update(text, 'utf8').digest('hex')
|
||||
}
|
||||
|
||||
function listXml(dir) {
|
||||
try {
|
||||
return fs
|
||||
.readdirSync(dir)
|
||||
.filter((name) => name.toLowerCase().endsWith('.xml'))
|
||||
.sort()
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') return []
|
||||
throw err
|
||||
}
|
||||
}
|
||||
|
||||
function readIfPresent(file) {
|
||||
try {
|
||||
return fs.readFileSync(file, 'utf8')
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') return null
|
||||
throw err
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read every atlas source file under `root`.
|
||||
*
|
||||
* Returns `{ files: [{ label, text, sha256, bytes }] }`, labels being
|
||||
* tree-relative and forward-slashed so a hash map compares equal across
|
||||
* platforms — the same tree read on Windows and Linux must produce the same
|
||||
* fingerprint or every boot would look like a change.
|
||||
*/
|
||||
function readSources(root) {
|
||||
if (!root || String(root).trim() === '') {
|
||||
throw new AtlasSourceError('No ServUO path configured', 'NO_PATH')
|
||||
}
|
||||
if (!fs.existsSync(root)) {
|
||||
throw new AtlasSourceError(`ServUO path does not exist: ${root}`, 'NOT_FOUND')
|
||||
}
|
||||
|
||||
const files = []
|
||||
const push = (label, file) => {
|
||||
const text = readIfPresent(file)
|
||||
if (text === null) return false
|
||||
files.push({ label, text, sha256: sha256(text), bytes: Buffer.byteLength(text, 'utf8') })
|
||||
return true
|
||||
}
|
||||
|
||||
if (!push('Data/Regions.xml', path.join(root, REGIONS_FILE))) {
|
||||
throw new AtlasSourceError(`Missing required file: ${REGIONS_FILE}`, 'NO_REGIONS')
|
||||
}
|
||||
|
||||
for (const name of listXml(path.join(root, LOCATIONS_DIR))) {
|
||||
push(`Data/Locations/${name}`, path.join(root, LOCATIONS_DIR, name))
|
||||
}
|
||||
|
||||
const spawnFiles = listXml(path.join(root, SPAWNS_DIR))
|
||||
if (spawnFiles.length === 0) {
|
||||
throw new AtlasSourceError(`No spawn files found in ${SPAWNS_DIR}`, 'NO_SPAWNS')
|
||||
}
|
||||
for (const name of spawnFiles) push(`Spawns/${name}`, path.join(root, SPAWNS_DIR, name))
|
||||
|
||||
push('Config/ChampionSpawns.xml', path.join(root, CHAMPIONS_FILE))
|
||||
|
||||
return { files }
|
||||
}
|
||||
|
||||
/**
|
||||
* A fingerprint of the tree: `{ "<label>": "<sha256>" }`.
|
||||
*
|
||||
* The boot path compares this against what was last imported and skips the
|
||||
* parse entirely when it matches, which is the normal case on every restart
|
||||
* that did not follow a map update.
|
||||
*/
|
||||
function hashSources(root) {
|
||||
const { files } = readSources(root)
|
||||
const hashes = {}
|
||||
for (const file of files) hashes[file.label] = file.sha256
|
||||
return hashes
|
||||
}
|
||||
|
||||
/**
|
||||
* Bumped whenever the parser produces DIFFERENT data from IDENTICAL source
|
||||
* files — a fixed misreading, a new field, a changed unit.
|
||||
*
|
||||
* Without it the hash gate is a trap: an install whose tree has not changed
|
||||
* would keep serving what an older parser derived, indefinitely, because the
|
||||
* only thing the boot path compares is the tree. The version is stored beside
|
||||
* the source hashes and a mismatch counts as drift, so a deploy that corrects
|
||||
* the parse actually reaches the data.
|
||||
*
|
||||
* 2 — respawn delays normalised to seconds (they are per-record minutes OR
|
||||
* seconds in the source, decided by `DelayInSec`).
|
||||
*/
|
||||
const PARSER_VERSION = 2
|
||||
|
||||
/** True when two source fingerprints describe the same tree. */
|
||||
function sameSources(a, b) {
|
||||
if (!a || !b) return false
|
||||
const aKeys = Object.keys(a).sort()
|
||||
const bKeys = Object.keys(b).sort()
|
||||
if (aKeys.length !== bKeys.length) return false
|
||||
return aKeys.every((key, i) => key === bKeys[i] && a[key] === b[key])
|
||||
}
|
||||
|
||||
// ── Aggregation ────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Choose one display spelling for a creature.
|
||||
*
|
||||
* Spawn files are not consistent about case — the same creature is `Lizardman`
|
||||
* in one file and `lizardman` in another. Slugging collapses them correctly, but
|
||||
* the display name would otherwise depend on file read order. Most frequent
|
||||
* spelling wins; ties break toward more capitals, then alphabetically.
|
||||
*/
|
||||
function displayName(spellings) {
|
||||
const capitals = (value) => (value.match(/[A-Z]/g) || []).length
|
||||
return [...spellings.entries()].sort((a, b) => {
|
||||
if (b[1] !== a[1]) return b[1] - a[1]
|
||||
const caps = capitals(b[0]) - capitals(a[0])
|
||||
if (caps !== 0) return caps
|
||||
return a[0].localeCompare(b[0])
|
||||
})[0][0]
|
||||
}
|
||||
|
||||
/**
|
||||
* Roll spawn points up into per-type creature rows.
|
||||
*
|
||||
* `total` is the sum of each type's own max across every point that spawns it —
|
||||
* how many of this creature the world holds at once. `facets` is a per-facet
|
||||
* point count, so "where does this live" answers without touching the points.
|
||||
*/
|
||||
function aggregateCreatures(points) {
|
||||
const creatures = new Map()
|
||||
for (const point of points) {
|
||||
for (const entry of point.types) {
|
||||
const slug = slugify(entry.type)
|
||||
if (slug === '') continue
|
||||
let creature = creatures.get(slug)
|
||||
if (!creature) {
|
||||
creature = { slug, name: '', total: 0, points: 0, facets: {}, spellings: new Map() }
|
||||
creatures.set(slug, creature)
|
||||
}
|
||||
creature.total += entry.max
|
||||
creature.points += 1
|
||||
creature.facets[point.facet] = (creature.facets[point.facet] || 0) + 1
|
||||
creature.spellings.set(entry.type, (creature.spellings.get(entry.type) || 0) + 1)
|
||||
}
|
||||
}
|
||||
|
||||
return [...creatures.values()]
|
||||
.map(({ spellings, ...creature }) => ({ ...creature, name: displayName(spellings) }))
|
||||
.sort((a, b) => a.slug.localeCompare(b.slug))
|
||||
}
|
||||
|
||||
// ── Build ──────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Parse a ServUO tree into the full atlas.
|
||||
*
|
||||
* Pure with respect to the database — it reads files and returns data; nothing
|
||||
* here writes. `shardAtlas.model.js` decides what to do with the result.
|
||||
*/
|
||||
function buildAtlas(root, options = {}) {
|
||||
const { files } = readSources(root)
|
||||
const byLabel = new Map(files.map((file) => [file.label, file]))
|
||||
const source = {}
|
||||
for (const file of files) source[file.label] = { bytes: file.bytes, sha256: file.sha256 }
|
||||
|
||||
const regions = parseRegions(byLabel.get('Data/Regions.xml').text)
|
||||
|
||||
const rawLandmarks = []
|
||||
for (const file of files) {
|
||||
if (!file.label.startsWith('Data/Locations/')) continue
|
||||
const basename = path.basename(file.label, '.xml')
|
||||
rawLandmarks.push(...parseLocations(file.text, basename))
|
||||
}
|
||||
|
||||
const rawPoints = []
|
||||
for (const file of files) {
|
||||
if (!file.label.startsWith('Spawns/')) continue
|
||||
rawPoints.push(...parsePoints(file.text))
|
||||
}
|
||||
|
||||
// The facet set is whatever THIS tree declares — never a built-in list. A
|
||||
// shard may add facets, replace them outright, or rename them when its maps
|
||||
// are updated, and the atlas has to follow without a code change. Spawn
|
||||
// records and region definitions are the authority, because those are the
|
||||
// names everything else is keyed on.
|
||||
const facetIndex = buildFacetIndex([
|
||||
...rawPoints.map((point) => point.facet),
|
||||
...regions.map((region) => region.facet),
|
||||
])
|
||||
|
||||
// Landmark facets are then matched against that set, which is what absorbs the
|
||||
// `Ter Mur` / `Tokuno Islands` spelling drift between Locations and <Map>.
|
||||
const landmarks = rawLandmarks.map(({ facetLabel, ...landmark }) => {
|
||||
const fromFile = resolveFacetName(landmark.facet, facetIndex)
|
||||
const matchedFile = facetIndex.has(facetKey(fromFile))
|
||||
const resolved = matchedFile ? fromFile : resolveFacetName(facetLabel, facetIndex)
|
||||
return { ...landmark, facet: resolved || landmark.facet }
|
||||
})
|
||||
|
||||
const placement = buildPlacementIndex(regions, landmarks)
|
||||
const resolveOpts = options.landmarkRadius ? { landmarkRadius: options.landmarkRadius } : {}
|
||||
|
||||
const disabled = rawPoints.filter((point) => !point.running).length
|
||||
const points = rawPoints
|
||||
// A spawner switched off in-world produces nothing; advertising it would be
|
||||
// a straight lie to a player planning a hunt.
|
||||
.filter((point) => point.running)
|
||||
// A spawner with no types is a placeholder — nothing to show.
|
||||
.filter((point) => point.types.length > 0)
|
||||
.map((point) => {
|
||||
const place = resolveRegion(point.x, point.y, point.facet, placement, resolveOpts)
|
||||
return {
|
||||
name: point.name,
|
||||
facet: point.facet,
|
||||
x: point.x,
|
||||
y: point.y,
|
||||
width: point.width,
|
||||
height: point.height,
|
||||
range: point.range,
|
||||
maxCount: point.maxCount,
|
||||
minDelay: point.minDelay,
|
||||
maxDelay: point.maxDelay,
|
||||
todStart: point.todStart,
|
||||
todEnd: point.todEnd,
|
||||
todMode: point.todMode,
|
||||
region: place.region,
|
||||
landmark: place.landmark,
|
||||
label: place.label,
|
||||
types: point.types,
|
||||
}
|
||||
})
|
||||
|
||||
const championsFile = byLabel.get('Config/ChampionSpawns.xml')
|
||||
const champions = (championsFile ? parseChampions(championsFile.text) : []).map((champ) => {
|
||||
const facet = resolveFacetName(champ.facet, facetIndex) || champ.facet
|
||||
return {
|
||||
...champ,
|
||||
facet,
|
||||
slug: slugify(`${facet}-${champ.name}`),
|
||||
label: resolveRegion(champ.x, champ.y, facet, placement, resolveOpts).label,
|
||||
}
|
||||
})
|
||||
|
||||
const creatures = aggregateCreatures(points)
|
||||
const facets = [...new Set(points.map((point) => point.facet))].sort()
|
||||
const unresolved = points.filter((point) => !point.region && !point.landmark).length
|
||||
|
||||
return {
|
||||
meta: {
|
||||
generatedAt: new Date().toISOString(),
|
||||
parserVersion: PARSER_VERSION,
|
||||
landmarkRadius: options.landmarkRadius ?? undefined,
|
||||
counts: {
|
||||
facets: facets.length,
|
||||
points: points.length,
|
||||
pointsDisabled: disabled,
|
||||
creatures: creatures.length,
|
||||
regions: regions.length,
|
||||
landmarks: landmarks.length,
|
||||
champions: champions.length,
|
||||
unresolvedPoints: unresolved,
|
||||
},
|
||||
source,
|
||||
},
|
||||
facets,
|
||||
creatures,
|
||||
regions,
|
||||
landmarks,
|
||||
champions,
|
||||
points,
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
AtlasSourceError,
|
||||
PARSER_VERSION,
|
||||
readSources,
|
||||
hashSources,
|
||||
sameSources,
|
||||
buildAtlas,
|
||||
aggregateCreatures,
|
||||
displayName,
|
||||
}
|
||||
@@ -1,224 +0,0 @@
|
||||
// ── uo-link sidecar REST client ────────────────────────────────────────────
|
||||
//
|
||||
// Server-side HTTP client for the uo-link sidecar (the bridge to the ServUO
|
||||
// shard). Same shape as botInternalClient: never throws — every call returns
|
||||
// { ok, data, status, error } so an admin poll or a public page never 500s just
|
||||
// because the sidecar/shard is down or restarting.
|
||||
//
|
||||
// The base URL + shared-secret token come from the DB-backed uoLinkConfig
|
||||
// (admin-managed, encrypted at rest) — NOT env vars, and the token is NEVER sent
|
||||
// to the browser. Every request carries `Authorization: Bearer <token>` and
|
||||
// `X-UOLink-Version: <protocol>` so a protocol mismatch is caught (409) rather
|
||||
// than mis-parsed. Config is cached for a few seconds to avoid decrypting the
|
||||
// token on every call.
|
||||
|
||||
const uoLinkConfig = require('../model/uoLinkConfig/uoLinkConfig.model')
|
||||
const log = require('./logger')('uo-link-client')
|
||||
|
||||
const TIMEOUT_MS = 12000 // sidecar waits up to 10s on the shard before 504
|
||||
const CONFIG_TTL_MS = 5000
|
||||
|
||||
let cachedConfig = null
|
||||
let cachedAt = 0
|
||||
|
||||
// Read (and briefly cache) the connection config incl. decrypted token.
|
||||
async function resolveConfig() {
|
||||
const now = Date.now()
|
||||
if (cachedConfig && now - cachedAt < CONFIG_TTL_MS) return cachedConfig
|
||||
cachedConfig = await uoLinkConfig.getWithToken()
|
||||
cachedAt = now
|
||||
return cachedConfig
|
||||
}
|
||||
|
||||
// Drop the cache after a save so the next call picks up new URL/token immediately.
|
||||
function invalidateConfig() {
|
||||
cachedConfig = null
|
||||
cachedAt = 0
|
||||
}
|
||||
|
||||
// Core request. Returns { ok, data, status, error }. `ok` is true only on a 2xx
|
||||
// with a parseable JSON body. Non-2xx responses still return their status + body
|
||||
// so callers can distinguish 503 (shard restarting — transient) from 404.
|
||||
async function call(path, { method = 'GET', body } = {}) {
|
||||
const controller = new AbortController()
|
||||
const timeout = setTimeout(() => controller.abort(), TIMEOUT_MS)
|
||||
// resolveConfig() decrypts the stored auth token, and decryption THROWS when the
|
||||
// ciphertext can't be authenticated — SECRET_ENC_KEY was rotated, or a DB dump was
|
||||
// restored into an environment keyed differently. It must stay INSIDE the try: out
|
||||
// here it escaped `call()` entirely and 500'd every live-shard route (admin and
|
||||
// player character/roster/vendor lookups, GET /admin/uo-link/config) instead of
|
||||
// degrading to "shard unavailable". This module never throws — see the header.
|
||||
let configResolved = false
|
||||
try {
|
||||
const config = await resolveConfig()
|
||||
configResolved = true
|
||||
if (!config || !config.baseUrl) {
|
||||
return { ok: false, status: 0, error: 'uo-link is not configured' }
|
||||
}
|
||||
|
||||
const headers = {
|
||||
'Content-Type': 'application/json',
|
||||
'X-UOLink-Version': String(config.protocol || 3),
|
||||
}
|
||||
if (config.token) headers.Authorization = `Bearer ${config.token}`
|
||||
|
||||
const res = await fetch(`${config.baseUrl}${path}`, {
|
||||
method,
|
||||
headers,
|
||||
body: body ? JSON.stringify(body) : undefined,
|
||||
signal: controller.signal,
|
||||
})
|
||||
|
||||
let data = null
|
||||
try {
|
||||
data = await res.json()
|
||||
} catch {
|
||||
// Non-JSON (or empty) body — leave data null; status still reported.
|
||||
}
|
||||
|
||||
if (!res.ok) {
|
||||
if (res.status === 401) log.warn('uo-link rejected auth token (401)', { path })
|
||||
if (res.status === 409) log.error('uo-link protocol mismatch (409)', { path, body: data })
|
||||
return { ok: false, status: res.status, data, error: `sidecar responded ${res.status}` }
|
||||
}
|
||||
return { ok: true, status: res.status, data }
|
||||
} catch (err) {
|
||||
// A failure before the config resolved is a misconfiguration, not a flaky
|
||||
// sidecar: log it loudly (and distinctly) so "the shard looks offline" doesn't
|
||||
// silently mean "the token can no longer be decrypted".
|
||||
if (!configResolved) {
|
||||
log.error('uo-link config unreadable — is SECRET_ENC_KEY the key the stored token was encrypted with?', {
|
||||
path,
|
||||
message: err.message,
|
||||
})
|
||||
return { ok: false, status: 0, error: 'uo-link config unreadable' }
|
||||
}
|
||||
log.warn('uo-link call failed', { path, message: err.message })
|
||||
return { ok: false, status: 0, error: err.message }
|
||||
} finally {
|
||||
clearTimeout(timeout)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Read queries ───────────────────────────────────────────────────────────
|
||||
// Liveness (no auth required by the sidecar, but we send it anyway).
|
||||
const health = () => call('/health')
|
||||
const getCharBySerial = (serial) => call(`/char/serial/${encodeURIComponent(serial)}`)
|
||||
const getCharBySlot = (account, slot) =>
|
||||
call(`/char/${encodeURIComponent(account)}/${encodeURIComponent(slot)}`)
|
||||
const getRoster = (account) => call(`/roster/${encodeURIComponent(account)}`)
|
||||
const getVendors = (account) => call(`/vendors/${encodeURIComponent(account)}`)
|
||||
|
||||
// History / economy series — used for WS-reconnect backfill and public feeds.
|
||||
function getHistory({ kind, limit = 100 } = {}) {
|
||||
const params = new URLSearchParams()
|
||||
if (kind) params.set('kind', kind)
|
||||
if (limit) params.set('limit', String(limit))
|
||||
const qs = params.toString()
|
||||
const suffix = qs ? `?${qs}` : ''
|
||||
return call(`/history${suffix}`)
|
||||
}
|
||||
const getEconomy = (limit = 100) => call(`/economy?limit=${encodeURIComponent(limit)}`)
|
||||
// Live board / queue projections — snapshotted on WS (re)connect and served from
|
||||
// our own store thereafter.
|
||||
const getChamps = () => call('/champs')
|
||||
const getPages = () => call('/pages')
|
||||
// Protocol 2.0 board projections — same snapshot-on-connect pattern.
|
||||
const getGuilds = () => call('/guilds')
|
||||
const getGovernors = () => call('/governors')
|
||||
const getHouses = () => call('/houses')
|
||||
const getPresence = () => call('/online') // aggregate population (count + byFacet/byRegion)
|
||||
// Protocol 3.0: the shard's published ruleset. Object-shaped, not a board — the
|
||||
// sidecar answers `{ ruleset: null }` until the shard has published one.
|
||||
const getRuleset = () => call('/ruleset')
|
||||
// Protocol 3.0: points/loyalty leaderboards. `/points` is board-shaped (an array
|
||||
// under `boards`); the per-system read 404s for a system the shard never published.
|
||||
const getPoints = () => call('/points')
|
||||
const getPointsBoard = (system) => call(`/points/${encodeURIComponent(system)}`)
|
||||
// Protocol 3.0: the player-vendor market index. The one PAGED sidecar read — a
|
||||
// whole-world market does not fit in a response — so it answers with
|
||||
// `{ vendors, total, limit, offset }` and the caller walks it (see uoLinkSocket).
|
||||
const getMarket = ({ limit = 200, offset = 0 } = {}) =>
|
||||
call(`/market?limit=${encodeURIComponent(limit)}&offset=${encodeURIComponent(offset)}`)
|
||||
|
||||
// ── Commands ──────────────────────────────────────────────────────────────
|
||||
const confirmLink = (code, websiteUserId) =>
|
||||
call('/link/confirm', { method: 'POST', body: { code, websiteUserId: String(websiteUserId) } })
|
||||
const linkLookup = (account) => call(`/link/${encodeURIComponent(account)}`)
|
||||
|
||||
// Account provisioning (Protocol 2.0). createAccount provisions a game account and
|
||||
// auto-links it to the website user in one step; `ip` is the END USER's browser IP
|
||||
// (read from the request), which the shard needs for its per-IP account cap — the
|
||||
// sidecar only sees our server. The password is hashed on the shard and never
|
||||
// appears in any reply/event/log. unlinkAccount severs a game account's tie from
|
||||
// the site side. `actor` is the staff/website id, recorded in the shard audit.
|
||||
const createAccount = ({ actor, account, password, websiteUserId, ip }) =>
|
||||
call('/accounts/create', {
|
||||
method: 'POST',
|
||||
body: { actor, account, password, websiteUserId: websiteUserId == null ? undefined : String(websiteUserId), ip },
|
||||
})
|
||||
const unlinkAccount = ({ actor, account }) =>
|
||||
call(`/link/${encodeURIComponent(account)}`, { method: 'DELETE', body: { actor } })
|
||||
const postTownCrier = ({ id, lines, durationSec }) =>
|
||||
call('/towncrier', { method: 'POST', body: { id, lines, durationSec } })
|
||||
const deleteTownCrier = (id) => call(`/towncrier/${encodeURIComponent(id)}`, { method: 'DELETE' })
|
||||
|
||||
// Town Cryer News gump (Protocol 2.1). A full article (title/HTML body/image/URL)
|
||||
// in the in-game News window; re-posting the same id REPLACES it. `announce`
|
||||
// (default true on the sidecar) controls whether the criers proclaim the title.
|
||||
const postNews = ({ id, title, body, image, url, announce }) =>
|
||||
call('/news', { method: 'POST', body: { id: String(id), title, body, image, url, announce } })
|
||||
const deleteNews = (id) => call(`/news/${encodeURIComponent(id)}`, { method: 'DELETE' })
|
||||
|
||||
// ── Staff write plane (§6) ─────────────────────────────────────────────────
|
||||
// Every call carries `actor` — the website username of the staff member — set by
|
||||
// the controller from the session, NEVER from the browser. The shard records it
|
||||
// for attribution and echoes an admin.audit event back over the WS feed.
|
||||
const adminKick = ({ actor, account, serial }) =>
|
||||
call('/admin/kick', { method: 'POST', body: { actor, account, serial } })
|
||||
const adminBan = ({ actor, account, serial, durationSec, reason }) =>
|
||||
call('/admin/ban', { method: 'POST', body: { actor, account, serial, durationSec, reason } })
|
||||
const adminUnban = ({ actor, account }) =>
|
||||
call('/admin/unban', { method: 'POST', body: { actor, account } })
|
||||
const adminBroadcast = ({ actor, text, hue }) =>
|
||||
call('/admin/broadcast', { method: 'POST', body: { actor, text, hue } })
|
||||
|
||||
// ── Help-page (support) queue commands (§6) ────────────────────────────────
|
||||
const respondPage = (pageId, { message, close }) =>
|
||||
call(`/pages/${encodeURIComponent(pageId)}/respond`, { method: 'POST', body: { message, close } })
|
||||
const closePage = (pageId) => call(`/pages/${encodeURIComponent(pageId)}/close`, { method: 'POST' })
|
||||
|
||||
module.exports = {
|
||||
invalidateConfig,
|
||||
health,
|
||||
getCharBySerial,
|
||||
getCharBySlot,
|
||||
getRoster,
|
||||
getVendors,
|
||||
getHistory,
|
||||
getEconomy,
|
||||
getChamps,
|
||||
getPages,
|
||||
getGuilds,
|
||||
getGovernors,
|
||||
getHouses,
|
||||
getPresence,
|
||||
getRuleset,
|
||||
getPoints,
|
||||
getPointsBoard,
|
||||
getMarket,
|
||||
confirmLink,
|
||||
linkLookup,
|
||||
createAccount,
|
||||
unlinkAccount,
|
||||
postTownCrier,
|
||||
deleteTownCrier,
|
||||
postNews,
|
||||
deleteNews,
|
||||
adminKick,
|
||||
adminBan,
|
||||
adminUnban,
|
||||
adminBroadcast,
|
||||
respondPage,
|
||||
closePage,
|
||||
}
|
||||
@@ -1,313 +0,0 @@
|
||||
// ── uo-link WebSocket ingest client ────────────────────────────────────────
|
||||
//
|
||||
// Long-lived client that connects to the sidecar's push-only WS feed and pumps
|
||||
// every frame through the ingest dispatcher. This is the server's first
|
||||
// outbound WebSocket. Lifecycle:
|
||||
// • start() — connect if the config is enabled and has a token; verify the
|
||||
// ws.hello protocol; backfill missed events via /history on every
|
||||
// (re)connect (INSERT IGNORE dedupes the overlap); reconnect with
|
||||
// capped backoff.
|
||||
// • stop() — close the socket and stop reconnecting (graceful shutdown).
|
||||
// Connection state is mirrored into uo_link_config (plugin_connected / status /
|
||||
// last_event_at) so the admin panel and public status endpoint have live data.
|
||||
|
||||
const WebSocket = require('ws')
|
||||
|
||||
const uoLinkConfig = require('../model/uoLinkConfig/uoLinkConfig.model')
|
||||
const uoLinkClient = require('./uoLinkClient')
|
||||
const shardIngest = require('./shardIngest')
|
||||
const shardState = require('../model/shardState/shardState.model')
|
||||
const newsGump = require('./newsGump')
|
||||
const log = require('./logger')('uo-link-socket')
|
||||
|
||||
const BACKOFF_MIN_MS = 1000
|
||||
const BACKOFF_MAX_MS = 30000
|
||||
const BACKFILL_LIMIT = 500
|
||||
|
||||
let ws = null
|
||||
let reconnectTimer = null
|
||||
let backoff = BACKOFF_MIN_MS
|
||||
let running = false // set by start()/stop(); guards auto-reconnect
|
||||
let helloSeen = false
|
||||
|
||||
const state = {
|
||||
connected: false,
|
||||
lastEventAt: null,
|
||||
lastConnectedAt: null,
|
||||
reconnects: 0,
|
||||
protocol: null,
|
||||
}
|
||||
|
||||
function buildUrl(wsUrl, token) {
|
||||
const sep = wsUrl.includes('?') ? '&' : '?'
|
||||
return token ? `${wsUrl}${sep}token=${encodeURIComponent(token)}` : wsUrl
|
||||
}
|
||||
|
||||
// One guarded board snapshot: fetch, verify `data[key]` is an array, hand it to
|
||||
// `apply`, and (when given) log `label` with the row count. Isolated so a
|
||||
// failed/absent board never aborts the rest of backfill — and so backfill()
|
||||
// stays a flat sequence rather than nine repetitions of the same guard.
|
||||
async function snapshot(fetchFn, key, apply, label) {
|
||||
const res = await fetchFn()
|
||||
if (!res.ok || !res.data || !Array.isArray(res.data[key])) return
|
||||
await apply(res.data[key])
|
||||
if (label) log.info(label, { count: res.data[key].length })
|
||||
}
|
||||
|
||||
// Replay events through the dispatcher oldest-first (history/economy arrive
|
||||
// newest-first) so latest-wins state settles correctly.
|
||||
async function ingestReversed(events) {
|
||||
for (const ev of [...events].reverse()) await shardIngest.ingest(ev, { fromBackfill: true })
|
||||
}
|
||||
async function ingestEach(events) {
|
||||
for (const ev of events) await shardIngest.ingest(ev, { fromBackfill: true })
|
||||
}
|
||||
|
||||
// ── Market backfill ────────────────────────────────────────────────────────
|
||||
//
|
||||
// The market is the only board that does not fit in one response, so /market is
|
||||
// paged and this walks it. Two bounds, both deliberate:
|
||||
//
|
||||
// • MARKET_SNAPSHOT_MAX caps the walk. A pathological world (or a sidecar whose
|
||||
// store was never pruned) must not be able to hang startup — backfill runs
|
||||
// before the site is serving the live feed, so an unbounded loop here is
|
||||
// downtime, not slowness.
|
||||
// • The loop stops on a SHORT page as well as on `total`, because a concurrent
|
||||
// sweep can shrink the index underneath the walk and paging to a stale total
|
||||
// would spin.
|
||||
//
|
||||
// Vendors are upserted, never reconciled-by-replacement. A vendor absent from the
|
||||
// snapshot is absent because the sidecar dropped it on vendor.listing.remove —
|
||||
// which our own ingest already processed — so clearing the table first would only
|
||||
// create a window where the market page is empty.
|
||||
const MARKET_SNAPSHOT_MAX = 5000
|
||||
const MARKET_PAGE = 200
|
||||
|
||||
async function backfillMarket() {
|
||||
let offset = 0
|
||||
let seen = 0
|
||||
|
||||
for (;;) {
|
||||
const res = await uoLinkClient.getMarket({ limit: MARKET_PAGE, offset })
|
||||
if (!res.ok || !res.data || !Array.isArray(res.data.vendors)) return
|
||||
|
||||
const page = res.data.vendors
|
||||
if (page.length === 0) break
|
||||
|
||||
await ingestEach(page)
|
||||
seen += page.length
|
||||
offset += page.length
|
||||
|
||||
if (page.length < MARKET_PAGE) break
|
||||
if (seen >= MARKET_SNAPSHOT_MAX) {
|
||||
log.warn('market snapshot truncated at the safety cap', {
|
||||
cap: MARKET_SNAPSHOT_MAX,
|
||||
total: res.data.total,
|
||||
})
|
||||
break
|
||||
}
|
||||
if (Number.isFinite(res.data.total) && offset >= res.data.total) break
|
||||
}
|
||||
|
||||
if (seen > 0) log.info('snapshotted player-vendor market from /market', { count: seen })
|
||||
}
|
||||
|
||||
// Pull recent events from the sidecar's own store and replay them through the
|
||||
// dispatcher (fromBackfill = no SSE re-broadcast). dedupe_key + INSERT IGNORE
|
||||
// make this idempotent, so overlap with what we already stored is harmless.
|
||||
async function backfill() {
|
||||
try {
|
||||
await snapshot(() => uoLinkClient.getHistory({ limit: BACKFILL_LIMIT }), 'events', ingestReversed, 'backfilled events from /history')
|
||||
await snapshot(() => uoLinkClient.getEconomy(200), 'series', ingestReversed)
|
||||
|
||||
// Champ board + help-page queue have no replay stream — snapshot the
|
||||
// authoritative current state directly (the sidecar guide's advice for both),
|
||||
// reconciling our tables to it so a stale row from before a disconnect can't
|
||||
// linger. Live champ.*/page.* deltas keep them fresh thereafter.
|
||||
await snapshot(() => uoLinkClient.getChamps(), 'spawns', (s) => shardState.replaceChamps(s), 'snapshotted champ board from /champs')
|
||||
await snapshot(() => uoLinkClient.getPages(), 'pages', (p) => shardState.replacePages(p), 'snapshotted help-page queue from /pages')
|
||||
|
||||
// ── Protocol 2.0 boards ──────────────────────────────────────────────
|
||||
// Same as champs/pages: snapshot the authoritative current state and
|
||||
// reconcile our tables to it. Each call is independently guarded so a
|
||||
// failed/absent board (e.g. no City Loyalty → empty /governors) never wipes
|
||||
// another. Governors are NOT cleared before upsert (cities are fixed and the
|
||||
// term-capture is idempotent, so a reconnect can't spawn spurious terms).
|
||||
await snapshot(() => uoLinkClient.getGuilds(), 'guilds', (g) => shardState.replaceGuilds(g), 'snapshotted guild board from /guilds')
|
||||
await snapshot(() => uoLinkClient.getGovernors(), 'cities', (c) => shardState.replaceGovernors(c), 'snapshotted governor board from /governors')
|
||||
await snapshot(() => uoLinkClient.getHouses(), 'houses', ingestEach, 'snapshotted house registry from /houses')
|
||||
|
||||
// ── Protocol 3.0 ─────────────────────────────────────────────────────
|
||||
// The ruleset is object-shaped, not a board, so it can't go through
|
||||
// snapshot() (which asserts an array under `key`). The shard also re-emits
|
||||
// world.ruleset on its own connect — this covers the other order, where the
|
||||
// sidecar was already up and holding the ruleset when WE reconnected.
|
||||
//
|
||||
// Routed through the dispatcher rather than straight to shardState, exactly as
|
||||
// ingestEach does for the array-shaped boards: the two orders must produce the
|
||||
// same stored frame, and calling setRuleset directly here made this a second
|
||||
// write path that silently skipped the shard-name normalization the live frame
|
||||
// gets. One writer, one set of rules.
|
||||
const ruleset = await uoLinkClient.getRuleset()
|
||||
if (ruleset.ok && ruleset.data && ruleset.data.ruleset) {
|
||||
await shardIngest.ingest(ruleset.data.ruleset, { fromBackfill: true })
|
||||
log.info('snapshotted shard ruleset from /ruleset', { rev: ruleset.data.ruleset.rev })
|
||||
}
|
||||
|
||||
// Points boards ARE array-shaped, so they go through snapshot() — but with
|
||||
// ingestEach rather than a replace*: there is no points.remove and the shard's
|
||||
// system set is fixed, so upserting is the whole reconciliation. A system the
|
||||
// operator has since excluded keeps its last-known board rather than vanishing,
|
||||
// which is the right answer for a month-scale standing.
|
||||
await snapshot(() => uoLinkClient.getPoints(), 'boards', ingestEach, 'snapshotted points boards from /points')
|
||||
|
||||
await backfillMarket()
|
||||
|
||||
const presence = await uoLinkClient.getPresence()
|
||||
if (presence.ok && presence.data && typeof presence.data.count === 'number') {
|
||||
await shardState.setPresence(presence.data)
|
||||
log.info('snapshotted online population from /online', { count: presence.data.count })
|
||||
}
|
||||
|
||||
// Re-assert our published news into the in-game Town Cryer News gump. The
|
||||
// website is the source of truth; this reconciles the gump on every
|
||||
// (re)connect (and recovers any article whose original live push failed).
|
||||
// Silent (announce:false) so a reconnect never re-proclaims old news.
|
||||
await newsGump.reassertAll()
|
||||
} catch (err) {
|
||||
log.warn('backfill failed (continuing on live feed)', { message: err.message })
|
||||
}
|
||||
}
|
||||
|
||||
function scheduleReconnect() {
|
||||
if (!running) return
|
||||
clearTimeout(reconnectTimer)
|
||||
reconnectTimer = setTimeout(connect, backoff)
|
||||
log.info(`reconnecting in ${backoff}ms`)
|
||||
backoff = Math.min(backoff * 2, BACKOFF_MAX_MS)
|
||||
}
|
||||
|
||||
async function connect() {
|
||||
if (!running) return
|
||||
let config
|
||||
try {
|
||||
config = await uoLinkConfig.getWithToken()
|
||||
} catch (err) {
|
||||
log.error('could not read uo-link config', err)
|
||||
scheduleReconnect()
|
||||
return
|
||||
}
|
||||
if (!config || !config.enabled || !config.wsUrl || !config.token) {
|
||||
log.info('uo-link WS not started (disabled or missing url/token)')
|
||||
running = false
|
||||
return
|
||||
}
|
||||
|
||||
state.protocol = config.protocol || 3
|
||||
helloSeen = false
|
||||
const url = buildUrl(config.wsUrl, config.token)
|
||||
|
||||
try {
|
||||
ws = new WebSocket(url)
|
||||
} catch (err) {
|
||||
log.error('failed to open WS', err)
|
||||
scheduleReconnect()
|
||||
return
|
||||
}
|
||||
|
||||
ws.on('open', handleOpen)
|
||||
ws.on('message', handleMessage)
|
||||
ws.on('close', handleClose)
|
||||
ws.on('error', (err) => {
|
||||
log.warn('uo-link WS error', { message: err.message })
|
||||
// 'close' fires after 'error'; reconnect is scheduled there.
|
||||
})
|
||||
}
|
||||
|
||||
// WS lifecycle handlers, split out of connect() so it stays a flat setup path.
|
||||
async function handleOpen() {
|
||||
log.info('uo-link WS connected')
|
||||
state.connected = true
|
||||
state.lastConnectedAt = Date.now()
|
||||
backoff = BACKOFF_MIN_MS
|
||||
await uoLinkConfig.recordStatus({ status: 'connected', statusDetail: null, pluginConnected: true }).catch(() => {})
|
||||
await backfill()
|
||||
}
|
||||
|
||||
// A ws.hello frame: mark it seen and, on a protocol mismatch, record the error
|
||||
// and close (we won't run against an incompatible sidecar).
|
||||
async function handleHello(event) {
|
||||
helloSeen = true
|
||||
if (!event.protocol || event.protocol === state.protocol) return
|
||||
log.error('uo-link protocol mismatch on ws.hello — closing', { expected: state.protocol, got: event.protocol })
|
||||
await uoLinkConfig
|
||||
.recordStatus({ status: 'error', statusDetail: `protocol mismatch: expected ${state.protocol}, got ${event.protocol}` })
|
||||
.catch(() => {})
|
||||
running = false
|
||||
try {
|
||||
ws.close()
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
|
||||
async function handleMessage(raw) {
|
||||
let event
|
||||
try {
|
||||
event = JSON.parse(raw.toString())
|
||||
} catch {
|
||||
log.warn('dropping non-JSON WS frame')
|
||||
return
|
||||
}
|
||||
|
||||
if (event.kind === 'ws.hello') return handleHello(event)
|
||||
if (event.kind === 'pong') return // sidecar heartbeat — ignore
|
||||
|
||||
state.lastEventAt = Number.isFinite(event.t) ? event.t : Date.now()
|
||||
await shardIngest.ingest(event)
|
||||
}
|
||||
|
||||
async function handleClose() {
|
||||
state.connected = false
|
||||
if (running) state.reconnects += 1
|
||||
log.warn('uo-link WS closed')
|
||||
await uoLinkConfig
|
||||
.recordStatus({ status: running ? 'reconnecting' : 'disconnected', pluginConnected: false })
|
||||
.catch(() => {})
|
||||
ws = null
|
||||
scheduleReconnect()
|
||||
}
|
||||
|
||||
// Begin (or restart) the WS client. Idempotent — a running client is stopped
|
||||
// first so a config save can re-point it at a new URL/token.
|
||||
async function start() {
|
||||
stop()
|
||||
running = true
|
||||
backoff = BACKOFF_MIN_MS
|
||||
await connect()
|
||||
}
|
||||
|
||||
// Stop the client and cancel any pending reconnect. Called on shutdown and
|
||||
// before a restart.
|
||||
function stop() {
|
||||
running = false
|
||||
clearTimeout(reconnectTimer)
|
||||
reconnectTimer = null
|
||||
if (ws) {
|
||||
try {
|
||||
ws.removeAllListeners()
|
||||
ws.close()
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
ws = null
|
||||
}
|
||||
state.connected = false
|
||||
}
|
||||
|
||||
// Ingestion stats for the admin panel.
|
||||
function getState() {
|
||||
return { ...state, running }
|
||||
}
|
||||
|
||||
module.exports = { start, stop, getState }
|
||||
Reference in New Issue
Block a user