refactor(atlas): derive the atlas from the shard's tree on every boot

Replaces the committed-artifact design from the first commit. Two problems with
it, both raised in review:

**Facets are not a fixed list.** The first pass carried a hardcoded table of the
six stock UO facets to reconcile the spelling drift between sources. That is
wrong: a shard may add facets, replace them outright, or rename them when its
maps are updated, and a built-in list quietly mishandles all three. Nothing in
the atlas names a facet any more. The facet set is discovered from the tree —
spawn records and region definitions are the authority — and the loose spellings
in Data/Locations are matched against it by key and prefix. Custom facets get
identical treatment; the tests use `Sosaria` and `Underdark` precisely so a
stock-facet assumption cannot creep back in.

**A snapshot goes stale.** Maps change over a server's life, so a build-once
artifact silently drifts from the world players actually see. The tree is now
the single source of truth and the atlas is re-derived on every boot.

## What that changed

- **The committed artifact is gone** — 1.41 MB of generated JSON removed, along
  with `scripts/buildSpawnAtlas.js` and the whole encode/decode seam it needed
  (`encodePoint`/`readPoint`, the tuple encoding, the omitted-defaults scheme and
  their round-trip tests). Nothing to keep in sync, nothing to go stale.
- **NEW `src/utils/spawnAtlasSource.js`** — the only thing that touches a ServUO
  tree; shared by the boot path and the CLI. Parsers stay pure and fs-free.
- **NEW `src/model/shardAtlas/`** — `.db.js` (the one-transaction replace) and
  `.model.js` (the refresh decision).
- **`scripts/importSpawnAtlas.js`** is now a thin CLI over the model:
  `--servuo`, `--force`, `--approve`, `--reject`, `--status`. `atlas:build` is
  gone; `atlas:import` remains.
- Path comes from the `spawn_atlas_servuo_path` admin setting, falling back to
  `SERVUO_PATH`. The setting wins, matching how the rest of the shard
  integration is admin-managed rather than env-configured.

## Two contracts on the boot path

**It never blocks startup.** No path, an unreadable mount, a malformed file, a
database error — every one is caught and logged, and the site comes up serving
whatever atlas it already had. Verified by booting the real server with no path,
a broken path, and a good path.

**A facet disappearing is never applied automatically.** Losing a facet is the
signature of a half-copied or mid-update tree as much as of a real map change,
and boot cannot tell them apart. The refresh is staged in `shard_atlas_pending`
for an admin to approve or reject, and startup continues regardless. Additions
and every other change apply immediately, since none of them can destroy
something an operator would miss.

Only the decision is stored, not the parsed world: a few KB of source hashes and
the facet diff. Approving re-parses, so what gets applied matches the tree at
approval time rather than at boot. A rejection is remembered against those exact
hashes, so a declined refresh does not re-prompt on every restart — changing the
tree changes the hashes and asks again.

Hash-gated, so the common case (restart, maps unchanged) reads and hashes the
tree (~120 ms) and writes nothing. A real change costs a ~400 ms parse.

The admin approve/reject UI is part of the second PR, with the rest of the
routes and pages. Until then the CLI covers it.

## Verification

- **564 server tests pass**, 28 new in `spawnAtlas.source.test.js` covering the
  custom-facet build, the spelling reconciliation, hash gating, and every branch
  of the refresh decision — including that `refreshOnBoot` survives a database
  that throws on every call.
- End-to-end against the local MariaDB and the real ServUO tree: 6,455 points,
  800 creatures, 23,927 point/type rows, 387 regions, 558 landmarks, 25 altars,
  83.2% of points resolved to a place name.
- The facet gate exercised against a real tree copy with `malas.xml` removed:
  staged rather than applied, atlas untouched with all 293 Malas points intact,
  reject then stays quiet on re-run, approve applies and drops the facet.
- Booted the real server under all three source conditions; none blocked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP

server/src/utils/spawnAtlasParse.js

@@ -182,33 +182,75 @@ function childrenNamed(node, name) {
 }
 
 // ── Facet names ────────────────────────────────────────────────────────────
-
-// The three sources disagree about facet spelling and nothing in the files
-// reconciles them: `Spawns/*.xml` `<Map>` and `Regions.xml` `<Facet name>` both
-// say `TerMur`/`Tokuno`, while `Data/Locations/*.xml` spells the same facets
-// `Ter Mur` and `Tokuno Islands`. Left alone this is silent — the landmark
-// fallback simply never matches on those two facets and every unregioned spawn
-// in Ter Mur and Tokuno reads "Wilderness" — so every facet name entering the
-// atlas is canonicalised through here first.
-const FACET_CANONICAL = new Map([
-  ['felucca', 'Felucca'],
-  ['trammel', 'Trammel'],
-  ['ilshenar', 'Ilshenar'],
-  ['malas', 'Malas'],
-  ['tokuno', 'Tokuno'],
-  ['tokunoislands', 'Tokuno'],
-  ['termur', 'TerMur'],
-])
+//
+// Facets are NOT a fixed list. A shard may add facets, replace them wholesale,
+// or rename them when its maps are updated, so nothing here may name Felucca,
+// Trammel or any other stock facet. The facet set is whatever the shard's own
+// files say it is, discovered at parse time.
+//
+// The complication is that the sources disagree about spelling for the SAME
+// facet and nothing in the files reconciles them: `Spawns/*.xml` `<Map>` and
+// `Regions.xml` `<Facet name>` say `TerMur`, while `Data/Locations/*.xml` spells
+// it `Ter Mur` and calls Tokuno `Tokuno Islands`. Left unreconciled this fails
+// silently — the landmark bucket is keyed differently from the points looking it
+// up, so the fallback never fires and every unregioned spawn on those facets
+// reads "Wilderness".
+//
+// Reconciliation is therefore done by MATCHING, not by a lookup table:
+// `facetKey()` collapses spelling differences, and `resolveFacetName()` matches
+// a loosely-spelled name against the canonical set discovered from the shard's
+// own data. A facet nobody else mentions keeps its own name rather than being
+// dropped.
 
 /**
- * Canonicalise a facet name to the `<Map>` spelling the atlas keys on.
- * Unknown facets pass through trimmed rather than being dropped, so a custom
- * shard facet still gets an atlas rather than vanishing.
+ * Collapse a facet name to a comparison key: lowercase, alphanumerics only.
+ * `TerMur`, `Ter Mur` and `ter-mur` all key alike.
  */
-function normalizeFacet(value) {
+function facetKey(value) {
+  return String(value ?? '')
+    .toLowerCase()
+    .replace(/[^a-z0-9]+/g, '')
+}
+
+/**
+ * Build a key → canonical-spelling lookup from the authoritative facet names.
+ *
+ * The authority is what the spawn records and region definitions actually say,
+ * since those are the names the atlas keys everything on. Later names do not
+ * overwrite earlier ones, so the first source wins consistently.
+ */
+function buildFacetIndex(names) {
+  const index = new Map()
+  for (const name of names) {
+    const key = facetKey(name)
+    if (key !== '' && !index.has(key)) index.set(key, String(name).trim())
+  }
+  return index
+}
+
+/**
+ * Resolve a loosely-spelled facet name against the discovered canonical set.
+ *
+ * Tried in order: exact key match (`Ter Mur` → `TerMur`), then a prefix match in
+ * either direction (`Tokuno Islands` → `Tokuno`), longest candidate first so a
+ * more specific facet wins over a shorter one that merely prefixes it.
+ *
+ * A name matching nothing is returned trimmed rather than dropped — on a shard
+ * with a custom facet that is a real facet the atlas simply has no spawns for
+ * yet, and inventing a match would be worse than leaving it alone.
+ */
+function resolveFacetName(value, index) {
   const raw = String(value ?? '').trim()
-  if (raw === '') return ''
-  return FACET_CANONICAL.get(raw.toLowerCase().replace(/[\s_-]+/g, '')) ?? raw
+  const key = facetKey(raw)
+  if (key === '') return ''
+  if (index.has(key)) return index.get(key)
+
+  let best = null
+  for (const [candidateKey, canonical] of index) {
+    if (!key.startsWith(candidateKey) && !candidateKey.startsWith(key)) continue
+    if (best === null || candidateKey.length > facetKey(best).length) best = canonical
+  }
+  return best ?? raw
 }
 
 // ── Small coercions ────────────────────────────────────────────────────────
@@ -326,7 +368,9 @@ function parsePoints(source) {
 
   while ((match = POINT_RE.exec(text)) !== null) {
     const block = match[1]
-    const facet = normalizeFacet(tagValue(block, 'Map'))
+    // Reported exactly as written. `<Map>` is the authority the rest of the
+    // atlas keys on, so it is never rewritten.
+    const facet = tagValue(block, 'Map')
     if (facet === '') continue
 
     points.push({
@@ -374,7 +418,7 @@ function parseRegions(source) {
   if (!root) return regions
 
   for (const facetNode of childrenNamed(root, 'Facet')) {
-    const facet = normalizeFacet(facetNode.attrs.name)
+    const facet = (facetNode.attrs.name || '').trim()
     if (facet === '') continue
     walkRegions(facetNode, facet, null, 0, regions)
   }
@@ -430,12 +474,25 @@ function parseLocations(source, facetHint = '') {
   if (!root) return landmarks
 
   for (const top of childrenNamed(root, 'parent')) {
-    const facet = normalizeFacet(top.attrs.name || facetHint)
-    walkLocations(top, facet, [], landmarks)
+    // The file name (`Data/Locations/termur.xml`) is the more reliable signal
+    // and is preferred over the display label inside the file, which is where
+    // the `Ter Mur` / `Tokuno Islands` drift lives. Both are carried so the
+    // build can fall back to matching the label if the file name resolves to
+    // nothing — a shard may well name its files differently from its facets.
+    landmarks.push(
+      ...collectLocations(top, facetHint || top.attrs.name || '', top.attrs.name || ''),
+    )
   }
   return landmarks
 }
 
+function collectLocations(top, facet, label) {
+  const out = []
+  walkLocations(top, facet, [], out)
+  for (const landmark of out) landmark.facetLabel = label
+  return out
+}
+
 function walkLocations(node, facet, path, out) {
   for (const child of childrenNamed(node, 'child')) {
     const name = child.attrs.name || ''
@@ -482,7 +539,7 @@ function parseChampions(source) {
       group: spawnNode.attrs.group || '',
       type: spawnNode.attrs.type || '',
       randomType: !spawnNode.attrs.type,
-      facet: normalizeFacet(attrs.map),
+      facet: (attrs.map || '').trim(),
       x: toInt(attrs.x),
       y: toInt(attrs.y),
       z: toInt(attrs.z),
@@ -514,9 +571,13 @@ function rectArea(rect) {
  */
 function buildPlacementIndex(regions, landmarks) {
   const byFacet = new Map()
+  // Keyed on facetKey(), not the raw name, so two spellings of one facet cannot
+  // land in separate buckets — the failure that silently emptied the landmark
+  // bucket for Ter Mur and Tokuno.
   const facet = (name) => {
-    if (!byFacet.has(name)) byFacet.set(name, { regions: [], landmarks: [] })
-    return byFacet.get(name)
+    const key = facetKey(name)
+    if (!byFacet.has(key)) byFacet.set(key, { regions: [], landmarks: [] })
+    return byFacet.get(key)
   }
   for (const region of regions) facet(region.facet).regions.push(region)
   for (const landmark of landmarks) facet(landmark.facet).landmarks.push(landmark)
@@ -542,7 +603,7 @@ function buildPlacementIndex(regions, landmarks) {
  */
 function resolveRegion(x, y, facetName, index, options = {}) {
   const radius = options.landmarkRadius ?? DEFAULT_LANDMARK_RADIUS
-  const bucket = index.get(facetName)
+  const bucket = index.get(facetKey(facetName))
   const result = { region: null, landmark: null, label: 'Wilderness' }
   if (!bucket) return result
 
@@ -594,7 +655,9 @@ module.exports = {
   parseChampions,
   buildPlacementIndex,
   resolveRegion,
-  normalizeFacet,
+  facetKey,
+  buildFacetIndex,
+  resolveFacetName,
   slugify,
   decodeEntities,
   DEFAULT_LANDMARK_RADIUS,

server/src/utils/spawnAtlasSource.js

@@ -0,0 +1,319 @@
+// Spawn atlas — the filesystem layer over a ServUO tree.
+//
+// `spawnAtlasParse.js` holds the pure parsers; this module is the only thing
+// that touches a ServUO tree on disk, and it is shared by both callers:
+//
+//   - the server, which refreshes the atlas on boot (`shardAtlas.model.js`)
+//   - the CLI (`scripts/importSpawnAtlas.js`)
+//
+// The shard's own files are the single source of truth. Nothing is precomputed
+// and committed, because a shard's maps change over its lifetime — facets get
+// added, replaced or renamed — and a snapshot in the repo would silently go
+// stale against the world players actually see.
+//
+// Reading and hashing the whole tree costs ~120 ms and a full parse ~400 ms, so
+// the boot path hashes first and only parses when something actually changed.
+
+const crypto = require('crypto')
+const fs = require('fs')
+const path = require('path')
+
+const {
+  parsePoints,
+  parseRegions,
+  parseLocations,
+  parseChampions,
+  buildPlacementIndex,
+  buildFacetIndex,
+  resolveFacetName,
+  resolveRegion,
+  facetKey,
+  slugify,
+} = require('./spawnAtlasParse')
+
+const REGIONS_FILE = path.join('Data', 'Regions.xml')
+const LOCATIONS_DIR = path.join('Data', 'Locations')
+const SPAWNS_DIR = 'Spawns'
+const CHAMPIONS_FILE = path.join('Config', 'ChampionSpawns.xml')
+
+class AtlasSourceError extends Error {
+  constructor(message, code) {
+    super(message)
+    this.name = 'AtlasSourceError'
+    this.code = code
+  }
+}
+
+// ── Reading ────────────────────────────────────────────────────────────────
+
+function sha256(text) {
+  return crypto.createHash('sha256').update(text, 'utf8').digest('hex')
+}
+
+function listXml(dir) {
+  try {
+    return fs
+      .readdirSync(dir)
+      .filter((name) => name.toLowerCase().endsWith('.xml'))
+      .sort()
+  } catch (err) {
+    if (err.code === 'ENOENT' || err.code === 'ENOTDIR') return []
+    throw err
+  }
+}
+
+function readIfPresent(file) {
+  try {
+    return fs.readFileSync(file, 'utf8')
+  } catch (err) {
+    if (err.code === 'ENOENT' || err.code === 'ENOTDIR') return null
+    throw err
+  }
+}
+
+/**
+ * Read every atlas source file under `root`.
+ *
+ * Returns `{ files: [{ label, text, sha256, bytes }] }`, labels being
+ * tree-relative and forward-slashed so a hash map compares equal across
+ * platforms — the same tree read on Windows and Linux must produce the same
+ * fingerprint or every boot would look like a change.
+ */
+function readSources(root) {
+  if (!root || String(root).trim() === '') {
+    throw new AtlasSourceError('No ServUO path configured', 'NO_PATH')
+  }
+  if (!fs.existsSync(root)) {
+    throw new AtlasSourceError(`ServUO path does not exist: ${root}`, 'NOT_FOUND')
+  }
+
+  const files = []
+  const push = (label, file) => {
+    const text = readIfPresent(file)
+    if (text === null) return false
+    files.push({ label, text, sha256: sha256(text), bytes: Buffer.byteLength(text, 'utf8') })
+    return true
+  }
+
+  if (!push('Data/Regions.xml', path.join(root, REGIONS_FILE))) {
+    throw new AtlasSourceError(`Missing required file: ${REGIONS_FILE}`, 'NO_REGIONS')
+  }
+
+  for (const name of listXml(path.join(root, LOCATIONS_DIR))) {
+    push(`Data/Locations/${name}`, path.join(root, LOCATIONS_DIR, name))
+  }
+
+  const spawnFiles = listXml(path.join(root, SPAWNS_DIR))
+  if (spawnFiles.length === 0) {
+    throw new AtlasSourceError(`No spawn files found in ${SPAWNS_DIR}`, 'NO_SPAWNS')
+  }
+  for (const name of spawnFiles) push(`Spawns/${name}`, path.join(root, SPAWNS_DIR, name))
+
+  push('Config/ChampionSpawns.xml', path.join(root, CHAMPIONS_FILE))
+
+  return { files }
+}
+
+/**
+ * A fingerprint of the tree: `{ "<label>": "<sha256>" }`.
+ *
+ * The boot path compares this against what was last imported and skips the
+ * parse entirely when it matches, which is the normal case on every restart
+ * that did not follow a map update.
+ */
+function hashSources(root) {
+  const { files } = readSources(root)
+  const hashes = {}
+  for (const file of files) hashes[file.label] = file.sha256
+  return hashes
+}
+
+/** True when two source fingerprints describe the same tree. */
+function sameSources(a, b) {
+  if (!a || !b) return false
+  const aKeys = Object.keys(a).sort()
+  const bKeys = Object.keys(b).sort()
+  if (aKeys.length !== bKeys.length) return false
+  return aKeys.every((key, i) => key === bKeys[i] && a[key] === b[key])
+}
+
+// ── Aggregation ────────────────────────────────────────────────────────────
+
+/**
+ * Choose one display spelling for a creature.
+ *
+ * Spawn files are not consistent about case — the same creature is `Lizardman`
+ * in one file and `lizardman` in another. Slugging collapses them correctly, but
+ * the display name would otherwise depend on file read order. Most frequent
+ * spelling wins; ties break toward more capitals, then alphabetically.
+ */
+function displayName(spellings) {
+  const capitals = (value) => (value.match(/[A-Z]/g) || []).length
+  return [...spellings.entries()].sort((a, b) => {
+    if (b[1] !== a[1]) return b[1] - a[1]
+    const caps = capitals(b[0]) - capitals(a[0])
+    if (caps !== 0) return caps
+    return a[0].localeCompare(b[0])
+  })[0][0]
+}
+
+/**
+ * Roll spawn points up into per-type creature rows.
+ *
+ * `total` is the sum of each type's own max across every point that spawns it —
+ * how many of this creature the world holds at once. `facets` is a per-facet
+ * point count, so "where does this live" answers without touching the points.
+ */
+function aggregateCreatures(points) {
+  const creatures = new Map()
+  for (const point of points) {
+    for (const entry of point.types) {
+      const slug = slugify(entry.type)
+      if (slug === '') continue
+      let creature = creatures.get(slug)
+      if (!creature) {
+        creature = { slug, name: '', total: 0, points: 0, facets: {}, spellings: new Map() }
+        creatures.set(slug, creature)
+      }
+      creature.total += entry.max
+      creature.points += 1
+      creature.facets[point.facet] = (creature.facets[point.facet] || 0) + 1
+      creature.spellings.set(entry.type, (creature.spellings.get(entry.type) || 0) + 1)
+    }
+  }
+
+  return [...creatures.values()]
+    .map(({ spellings, ...creature }) => ({ ...creature, name: displayName(spellings) }))
+    .sort((a, b) => a.slug.localeCompare(b.slug))
+}
+
+// ── Build ──────────────────────────────────────────────────────────────────
+
+/**
+ * Parse a ServUO tree into the full atlas.
+ *
+ * Pure with respect to the database — it reads files and returns data; nothing
+ * here writes. `shardAtlas.model.js` decides what to do with the result.
+ */
+function buildAtlas(root, options = {}) {
+  const { files } = readSources(root)
+  const byLabel = new Map(files.map((file) => [file.label, file]))
+  const source = {}
+  for (const file of files) source[file.label] = { bytes: file.bytes, sha256: file.sha256 }
+
+  const regions = parseRegions(byLabel.get('Data/Regions.xml').text)
+
+  const rawLandmarks = []
+  for (const file of files) {
+    if (!file.label.startsWith('Data/Locations/')) continue
+    const basename = path.basename(file.label, '.xml')
+    rawLandmarks.push(...parseLocations(file.text, basename))
+  }
+
+  const rawPoints = []
+  for (const file of files) {
+    if (!file.label.startsWith('Spawns/')) continue
+    rawPoints.push(...parsePoints(file.text))
+  }
+
+  // The facet set is whatever THIS tree declares — never a built-in list. A
+  // shard may add facets, replace them outright, or rename them when its maps
+  // are updated, and the atlas has to follow without a code change. Spawn
+  // records and region definitions are the authority, because those are the
+  // names everything else is keyed on.
+  const facetIndex = buildFacetIndex([
+    ...rawPoints.map((point) => point.facet),
+    ...regions.map((region) => region.facet),
+  ])
+
+  // Landmark facets are then matched against that set, which is what absorbs the
+  // `Ter Mur` / `Tokuno Islands` spelling drift between Locations and <Map>.
+  const landmarks = rawLandmarks.map(({ facetLabel, ...landmark }) => {
+    const fromFile = resolveFacetName(landmark.facet, facetIndex)
+    const matchedFile = facetIndex.has(facetKey(fromFile))
+    const resolved = matchedFile ? fromFile : resolveFacetName(facetLabel, facetIndex)
+    return { ...landmark, facet: resolved || landmark.facet }
+  })
+
+  const placement = buildPlacementIndex(regions, landmarks)
+  const resolveOpts = options.landmarkRadius ? { landmarkRadius: options.landmarkRadius } : {}
+
+  const disabled = rawPoints.filter((point) => !point.running).length
+  const points = rawPoints
+    // A spawner switched off in-world produces nothing; advertising it would be
+    // a straight lie to a player planning a hunt.
+    .filter((point) => point.running)
+    // A spawner with no types is a placeholder — nothing to show.
+    .filter((point) => point.types.length > 0)
+    .map((point) => {
+      const place = resolveRegion(point.x, point.y, point.facet, placement, resolveOpts)
+      return {
+        name: point.name,
+        facet: point.facet,
+        x: point.x,
+        y: point.y,
+        width: point.width,
+        height: point.height,
+        range: point.range,
+        maxCount: point.maxCount,
+        minDelay: point.minDelay,
+        maxDelay: point.maxDelay,
+        todStart: point.todStart,
+        todEnd: point.todEnd,
+        todMode: point.todMode,
+        region: place.region,
+        landmark: place.landmark,
+        label: place.label,
+        types: point.types,
+      }
+    })
+
+  const championsFile = byLabel.get('Config/ChampionSpawns.xml')
+  const champions = (championsFile ? parseChampions(championsFile.text) : []).map((champ) => {
+    const facet = resolveFacetName(champ.facet, facetIndex) || champ.facet
+    return {
+      ...champ,
+      facet,
+      slug: slugify(`${facet}-${champ.name}`),
+      label: resolveRegion(champ.x, champ.y, facet, placement, resolveOpts).label,
+    }
+  })
+
+  const creatures = aggregateCreatures(points)
+  const facets = [...new Set(points.map((point) => point.facet))].sort()
+  const unresolved = points.filter((point) => !point.region && !point.landmark).length
+
+  return {
+    meta: {
+      generatedAt: new Date().toISOString(),
+      landmarkRadius: options.landmarkRadius ?? undefined,
+      counts: {
+        facets: facets.length,
+        points: points.length,
+        pointsDisabled: disabled,
+        creatures: creatures.length,
+        regions: regions.length,
+        landmarks: landmarks.length,
+        champions: champions.length,
+        unresolvedPoints: unresolved,
+      },
+      source,
+    },
+    facets,
+    creatures,
+    regions,
+    landmarks,
+    champions,
+    points,
+  }
+}
+
+module.exports = {
+  AtlasSourceError,
+  readSources,
+  hashSources,
+  sameSources,
+  buildAtlas,
+  aggregateCreatures,
+  displayName,
+}