refactor(atlas): derive the atlas from the shard's tree on every boot
Replaces the committed-artifact design from the first commit. Two problems with it, both raised in review: **Facets are not a fixed list.** The first pass carried a hardcoded table of the six stock UO facets to reconcile the spelling drift between sources. That is wrong: a shard may add facets, replace them outright, or rename them when its maps are updated, and a built-in list quietly mishandles all three. Nothing in the atlas names a facet any more. The facet set is discovered from the tree — spawn records and region definitions are the authority — and the loose spellings in Data/Locations are matched against it by key and prefix. Custom facets get identical treatment; the tests use `Sosaria` and `Underdark` precisely so a stock-facet assumption cannot creep back in. **A snapshot goes stale.** Maps change over a server's life, so a build-once artifact silently drifts from the world players actually see. The tree is now the single source of truth and the atlas is re-derived on every boot. ## What that changed - **The committed artifact is gone** — 1.41 MB of generated JSON removed, along with `scripts/buildSpawnAtlas.js` and the whole encode/decode seam it needed (`encodePoint`/`readPoint`, the tuple encoding, the omitted-defaults scheme and their round-trip tests). Nothing to keep in sync, nothing to go stale. - **NEW `src/utils/spawnAtlasSource.js`** — the only thing that touches a ServUO tree; shared by the boot path and the CLI. Parsers stay pure and fs-free. - **NEW `src/model/shardAtlas/`** — `.db.js` (the one-transaction replace) and `.model.js` (the refresh decision). - **`scripts/importSpawnAtlas.js`** is now a thin CLI over the model: `--servuo`, `--force`, `--approve`, `--reject`, `--status`. `atlas:build` is gone; `atlas:import` remains. - Path comes from the `spawn_atlas_servuo_path` admin setting, falling back to `SERVUO_PATH`. The setting wins, matching how the rest of the shard integration is admin-managed rather than env-configured. ## Two contracts on the boot path **It never blocks startup.** No path, an unreadable mount, a malformed file, a database error — every one is caught and logged, and the site comes up serving whatever atlas it already had. Verified by booting the real server with no path, a broken path, and a good path. **A facet disappearing is never applied automatically.** Losing a facet is the signature of a half-copied or mid-update tree as much as of a real map change, and boot cannot tell them apart. The refresh is staged in `shard_atlas_pending` for an admin to approve or reject, and startup continues regardless. Additions and every other change apply immediately, since none of them can destroy something an operator would miss. Only the decision is stored, not the parsed world: a few KB of source hashes and the facet diff. Approving re-parses, so what gets applied matches the tree at approval time rather than at boot. A rejection is remembered against those exact hashes, so a declined refresh does not re-prompt on every restart — changing the tree changes the hashes and asks again. Hash-gated, so the common case (restart, maps unchanged) reads and hashes the tree (~120 ms) and writes nothing. A real change costs a ~400 ms parse. The admin approve/reject UI is part of the second PR, with the rest of the routes and pages. Until then the CLI covers it. ## Verification - **564 server tests pass**, 28 new in `spawnAtlas.source.test.js` covering the custom-facet build, the spelling reconciliation, hash gating, and every branch of the refresh decision — including that `refreshOnBoot` survives a database that throws on every call. - End-to-end against the local MariaDB and the real ServUO tree: 6,455 points, 800 creatures, 23,927 point/type rows, 387 regions, 558 landmarks, 25 altars, 83.2% of points resolved to a place name. - The facet gate exercised against a real tree copy with `malas.xml` removed: staged rather than applied, atlas untouched with all 293 Malas points intact, reject then stays quiet on re-run, approve applies and drops the facet. - Booted the real server under all three source conditions; none blocked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP
This commit is contained in:
319
server/src/utils/spawnAtlasSource.js
Normal file
319
server/src/utils/spawnAtlasSource.js
Normal file
@@ -0,0 +1,319 @@
|
||||
// Spawn atlas — the filesystem layer over a ServUO tree.
|
||||
//
|
||||
// `spawnAtlasParse.js` holds the pure parsers; this module is the only thing
|
||||
// that touches a ServUO tree on disk, and it is shared by both callers:
|
||||
//
|
||||
// - the server, which refreshes the atlas on boot (`shardAtlas.model.js`)
|
||||
// - the CLI (`scripts/importSpawnAtlas.js`)
|
||||
//
|
||||
// The shard's own files are the single source of truth. Nothing is precomputed
|
||||
// and committed, because a shard's maps change over its lifetime — facets get
|
||||
// added, replaced or renamed — and a snapshot in the repo would silently go
|
||||
// stale against the world players actually see.
|
||||
//
|
||||
// Reading and hashing the whole tree costs ~120 ms and a full parse ~400 ms, so
|
||||
// the boot path hashes first and only parses when something actually changed.
|
||||
|
||||
const crypto = require('crypto')
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
|
||||
const {
|
||||
parsePoints,
|
||||
parseRegions,
|
||||
parseLocations,
|
||||
parseChampions,
|
||||
buildPlacementIndex,
|
||||
buildFacetIndex,
|
||||
resolveFacetName,
|
||||
resolveRegion,
|
||||
facetKey,
|
||||
slugify,
|
||||
} = require('./spawnAtlasParse')
|
||||
|
||||
const REGIONS_FILE = path.join('Data', 'Regions.xml')
|
||||
const LOCATIONS_DIR = path.join('Data', 'Locations')
|
||||
const SPAWNS_DIR = 'Spawns'
|
||||
const CHAMPIONS_FILE = path.join('Config', 'ChampionSpawns.xml')
|
||||
|
||||
class AtlasSourceError extends Error {
|
||||
constructor(message, code) {
|
||||
super(message)
|
||||
this.name = 'AtlasSourceError'
|
||||
this.code = code
|
||||
}
|
||||
}
|
||||
|
||||
// ── Reading ────────────────────────────────────────────────────────────────
|
||||
|
||||
function sha256(text) {
|
||||
return crypto.createHash('sha256').update(text, 'utf8').digest('hex')
|
||||
}
|
||||
|
||||
function listXml(dir) {
|
||||
try {
|
||||
return fs
|
||||
.readdirSync(dir)
|
||||
.filter((name) => name.toLowerCase().endsWith('.xml'))
|
||||
.sort()
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') return []
|
||||
throw err
|
||||
}
|
||||
}
|
||||
|
||||
function readIfPresent(file) {
|
||||
try {
|
||||
return fs.readFileSync(file, 'utf8')
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') return null
|
||||
throw err
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read every atlas source file under `root`.
|
||||
*
|
||||
* Returns `{ files: [{ label, text, sha256, bytes }] }`, labels being
|
||||
* tree-relative and forward-slashed so a hash map compares equal across
|
||||
* platforms — the same tree read on Windows and Linux must produce the same
|
||||
* fingerprint or every boot would look like a change.
|
||||
*/
|
||||
function readSources(root) {
|
||||
if (!root || String(root).trim() === '') {
|
||||
throw new AtlasSourceError('No ServUO path configured', 'NO_PATH')
|
||||
}
|
||||
if (!fs.existsSync(root)) {
|
||||
throw new AtlasSourceError(`ServUO path does not exist: ${root}`, 'NOT_FOUND')
|
||||
}
|
||||
|
||||
const files = []
|
||||
const push = (label, file) => {
|
||||
const text = readIfPresent(file)
|
||||
if (text === null) return false
|
||||
files.push({ label, text, sha256: sha256(text), bytes: Buffer.byteLength(text, 'utf8') })
|
||||
return true
|
||||
}
|
||||
|
||||
if (!push('Data/Regions.xml', path.join(root, REGIONS_FILE))) {
|
||||
throw new AtlasSourceError(`Missing required file: ${REGIONS_FILE}`, 'NO_REGIONS')
|
||||
}
|
||||
|
||||
for (const name of listXml(path.join(root, LOCATIONS_DIR))) {
|
||||
push(`Data/Locations/${name}`, path.join(root, LOCATIONS_DIR, name))
|
||||
}
|
||||
|
||||
const spawnFiles = listXml(path.join(root, SPAWNS_DIR))
|
||||
if (spawnFiles.length === 0) {
|
||||
throw new AtlasSourceError(`No spawn files found in ${SPAWNS_DIR}`, 'NO_SPAWNS')
|
||||
}
|
||||
for (const name of spawnFiles) push(`Spawns/${name}`, path.join(root, SPAWNS_DIR, name))
|
||||
|
||||
push('Config/ChampionSpawns.xml', path.join(root, CHAMPIONS_FILE))
|
||||
|
||||
return { files }
|
||||
}
|
||||
|
||||
/**
|
||||
* A fingerprint of the tree: `{ "<label>": "<sha256>" }`.
|
||||
*
|
||||
* The boot path compares this against what was last imported and skips the
|
||||
* parse entirely when it matches, which is the normal case on every restart
|
||||
* that did not follow a map update.
|
||||
*/
|
||||
function hashSources(root) {
|
||||
const { files } = readSources(root)
|
||||
const hashes = {}
|
||||
for (const file of files) hashes[file.label] = file.sha256
|
||||
return hashes
|
||||
}
|
||||
|
||||
/** True when two source fingerprints describe the same tree. */
|
||||
function sameSources(a, b) {
|
||||
if (!a || !b) return false
|
||||
const aKeys = Object.keys(a).sort()
|
||||
const bKeys = Object.keys(b).sort()
|
||||
if (aKeys.length !== bKeys.length) return false
|
||||
return aKeys.every((key, i) => key === bKeys[i] && a[key] === b[key])
|
||||
}
|
||||
|
||||
// ── Aggregation ────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Choose one display spelling for a creature.
|
||||
*
|
||||
* Spawn files are not consistent about case — the same creature is `Lizardman`
|
||||
* in one file and `lizardman` in another. Slugging collapses them correctly, but
|
||||
* the display name would otherwise depend on file read order. Most frequent
|
||||
* spelling wins; ties break toward more capitals, then alphabetically.
|
||||
*/
|
||||
function displayName(spellings) {
|
||||
const capitals = (value) => (value.match(/[A-Z]/g) || []).length
|
||||
return [...spellings.entries()].sort((a, b) => {
|
||||
if (b[1] !== a[1]) return b[1] - a[1]
|
||||
const caps = capitals(b[0]) - capitals(a[0])
|
||||
if (caps !== 0) return caps
|
||||
return a[0].localeCompare(b[0])
|
||||
})[0][0]
|
||||
}
|
||||
|
||||
/**
|
||||
* Roll spawn points up into per-type creature rows.
|
||||
*
|
||||
* `total` is the sum of each type's own max across every point that spawns it —
|
||||
* how many of this creature the world holds at once. `facets` is a per-facet
|
||||
* point count, so "where does this live" answers without touching the points.
|
||||
*/
|
||||
function aggregateCreatures(points) {
|
||||
const creatures = new Map()
|
||||
for (const point of points) {
|
||||
for (const entry of point.types) {
|
||||
const slug = slugify(entry.type)
|
||||
if (slug === '') continue
|
||||
let creature = creatures.get(slug)
|
||||
if (!creature) {
|
||||
creature = { slug, name: '', total: 0, points: 0, facets: {}, spellings: new Map() }
|
||||
creatures.set(slug, creature)
|
||||
}
|
||||
creature.total += entry.max
|
||||
creature.points += 1
|
||||
creature.facets[point.facet] = (creature.facets[point.facet] || 0) + 1
|
||||
creature.spellings.set(entry.type, (creature.spellings.get(entry.type) || 0) + 1)
|
||||
}
|
||||
}
|
||||
|
||||
return [...creatures.values()]
|
||||
.map(({ spellings, ...creature }) => ({ ...creature, name: displayName(spellings) }))
|
||||
.sort((a, b) => a.slug.localeCompare(b.slug))
|
||||
}
|
||||
|
||||
// ── Build ──────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Parse a ServUO tree into the full atlas.
|
||||
*
|
||||
* Pure with respect to the database — it reads files and returns data; nothing
|
||||
* here writes. `shardAtlas.model.js` decides what to do with the result.
|
||||
*/
|
||||
function buildAtlas(root, options = {}) {
|
||||
const { files } = readSources(root)
|
||||
const byLabel = new Map(files.map((file) => [file.label, file]))
|
||||
const source = {}
|
||||
for (const file of files) source[file.label] = { bytes: file.bytes, sha256: file.sha256 }
|
||||
|
||||
const regions = parseRegions(byLabel.get('Data/Regions.xml').text)
|
||||
|
||||
const rawLandmarks = []
|
||||
for (const file of files) {
|
||||
if (!file.label.startsWith('Data/Locations/')) continue
|
||||
const basename = path.basename(file.label, '.xml')
|
||||
rawLandmarks.push(...parseLocations(file.text, basename))
|
||||
}
|
||||
|
||||
const rawPoints = []
|
||||
for (const file of files) {
|
||||
if (!file.label.startsWith('Spawns/')) continue
|
||||
rawPoints.push(...parsePoints(file.text))
|
||||
}
|
||||
|
||||
// The facet set is whatever THIS tree declares — never a built-in list. A
|
||||
// shard may add facets, replace them outright, or rename them when its maps
|
||||
// are updated, and the atlas has to follow without a code change. Spawn
|
||||
// records and region definitions are the authority, because those are the
|
||||
// names everything else is keyed on.
|
||||
const facetIndex = buildFacetIndex([
|
||||
...rawPoints.map((point) => point.facet),
|
||||
...regions.map((region) => region.facet),
|
||||
])
|
||||
|
||||
// Landmark facets are then matched against that set, which is what absorbs the
|
||||
// `Ter Mur` / `Tokuno Islands` spelling drift between Locations and <Map>.
|
||||
const landmarks = rawLandmarks.map(({ facetLabel, ...landmark }) => {
|
||||
const fromFile = resolveFacetName(landmark.facet, facetIndex)
|
||||
const matchedFile = facetIndex.has(facetKey(fromFile))
|
||||
const resolved = matchedFile ? fromFile : resolveFacetName(facetLabel, facetIndex)
|
||||
return { ...landmark, facet: resolved || landmark.facet }
|
||||
})
|
||||
|
||||
const placement = buildPlacementIndex(regions, landmarks)
|
||||
const resolveOpts = options.landmarkRadius ? { landmarkRadius: options.landmarkRadius } : {}
|
||||
|
||||
const disabled = rawPoints.filter((point) => !point.running).length
|
||||
const points = rawPoints
|
||||
// A spawner switched off in-world produces nothing; advertising it would be
|
||||
// a straight lie to a player planning a hunt.
|
||||
.filter((point) => point.running)
|
||||
// A spawner with no types is a placeholder — nothing to show.
|
||||
.filter((point) => point.types.length > 0)
|
||||
.map((point) => {
|
||||
const place = resolveRegion(point.x, point.y, point.facet, placement, resolveOpts)
|
||||
return {
|
||||
name: point.name,
|
||||
facet: point.facet,
|
||||
x: point.x,
|
||||
y: point.y,
|
||||
width: point.width,
|
||||
height: point.height,
|
||||
range: point.range,
|
||||
maxCount: point.maxCount,
|
||||
minDelay: point.minDelay,
|
||||
maxDelay: point.maxDelay,
|
||||
todStart: point.todStart,
|
||||
todEnd: point.todEnd,
|
||||
todMode: point.todMode,
|
||||
region: place.region,
|
||||
landmark: place.landmark,
|
||||
label: place.label,
|
||||
types: point.types,
|
||||
}
|
||||
})
|
||||
|
||||
const championsFile = byLabel.get('Config/ChampionSpawns.xml')
|
||||
const champions = (championsFile ? parseChampions(championsFile.text) : []).map((champ) => {
|
||||
const facet = resolveFacetName(champ.facet, facetIndex) || champ.facet
|
||||
return {
|
||||
...champ,
|
||||
facet,
|
||||
slug: slugify(`${facet}-${champ.name}`),
|
||||
label: resolveRegion(champ.x, champ.y, facet, placement, resolveOpts).label,
|
||||
}
|
||||
})
|
||||
|
||||
const creatures = aggregateCreatures(points)
|
||||
const facets = [...new Set(points.map((point) => point.facet))].sort()
|
||||
const unresolved = points.filter((point) => !point.region && !point.landmark).length
|
||||
|
||||
return {
|
||||
meta: {
|
||||
generatedAt: new Date().toISOString(),
|
||||
landmarkRadius: options.landmarkRadius ?? undefined,
|
||||
counts: {
|
||||
facets: facets.length,
|
||||
points: points.length,
|
||||
pointsDisabled: disabled,
|
||||
creatures: creatures.length,
|
||||
regions: regions.length,
|
||||
landmarks: landmarks.length,
|
||||
champions: champions.length,
|
||||
unresolvedPoints: unresolved,
|
||||
},
|
||||
source,
|
||||
},
|
||||
facets,
|
||||
creatures,
|
||||
regions,
|
||||
landmarks,
|
||||
champions,
|
||||
points,
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
AtlasSourceError,
|
||||
readSources,
|
||||
hashSources,
|
||||
sameSources,
|
||||
buildAtlas,
|
||||
aggregateCreatures,
|
||||
displayName,
|
||||
}
|
||||
Reference in New Issue
Block a user