refactor(atlas): derive the atlas from the shard's tree on every boot

Replaces the committed-artifact design from the first commit. Two problems with
it, both raised in review:

**Facets are not a fixed list.** The first pass carried a hardcoded table of the
six stock UO facets to reconcile the spelling drift between sources. That is
wrong: a shard may add facets, replace them outright, or rename them when its
maps are updated, and a built-in list quietly mishandles all three. Nothing in
the atlas names a facet any more. The facet set is discovered from the tree —
spawn records and region definitions are the authority — and the loose spellings
in Data/Locations are matched against it by key and prefix. Custom facets get
identical treatment; the tests use `Sosaria` and `Underdark` precisely so a
stock-facet assumption cannot creep back in.

**A snapshot goes stale.** Maps change over a server's life, so a build-once
artifact silently drifts from the world players actually see. The tree is now
the single source of truth and the atlas is re-derived on every boot.

## What that changed

- **The committed artifact is gone** — 1.41 MB of generated JSON removed, along
  with `scripts/buildSpawnAtlas.js` and the whole encode/decode seam it needed
  (`encodePoint`/`readPoint`, the tuple encoding, the omitted-defaults scheme and
  their round-trip tests). Nothing to keep in sync, nothing to go stale.
- **NEW `src/utils/spawnAtlasSource.js`** — the only thing that touches a ServUO
  tree; shared by the boot path and the CLI. Parsers stay pure and fs-free.
- **NEW `src/model/shardAtlas/`** — `.db.js` (the one-transaction replace) and
  `.model.js` (the refresh decision).
- **`scripts/importSpawnAtlas.js`** is now a thin CLI over the model:
  `--servuo`, `--force`, `--approve`, `--reject`, `--status`. `atlas:build` is
  gone; `atlas:import` remains.
- Path comes from the `spawn_atlas_servuo_path` admin setting, falling back to
  `SERVUO_PATH`. The setting wins, matching how the rest of the shard
  integration is admin-managed rather than env-configured.

## Two contracts on the boot path

**It never blocks startup.** No path, an unreadable mount, a malformed file, a
database error — every one is caught and logged, and the site comes up serving
whatever atlas it already had. Verified by booting the real server with no path,
a broken path, and a good path.

**A facet disappearing is never applied automatically.** Losing a facet is the
signature of a half-copied or mid-update tree as much as of a real map change,
and boot cannot tell them apart. The refresh is staged in `shard_atlas_pending`
for an admin to approve or reject, and startup continues regardless. Additions
and every other change apply immediately, since none of them can destroy
something an operator would miss.

Only the decision is stored, not the parsed world: a few KB of source hashes and
the facet diff. Approving re-parses, so what gets applied matches the tree at
approval time rather than at boot. A rejection is remembered against those exact
hashes, so a declined refresh does not re-prompt on every restart — changing the
tree changes the hashes and asks again.

Hash-gated, so the common case (restart, maps unchanged) reads and hashes the
tree (~120 ms) and writes nothing. A real change costs a ~400 ms parse.

The admin approve/reject UI is part of the second PR, with the rest of the
routes and pages. Until then the CLI covers it.

## Verification

- **564 server tests pass**, 28 new in `spawnAtlas.source.test.js` covering the
  custom-facet build, the spelling reconciliation, hash gating, and every branch
  of the refresh decision — including that `refreshOnBoot` survives a database
  that throws on every call.
- End-to-end against the local MariaDB and the real ServUO tree: 6,455 points,
  800 creatures, 23,927 point/type rows, 387 regions, 558 landmarks, 25 altars,
  83.2% of points resolved to a place name.
- The facet gate exercised against a real tree copy with `malas.xml` removed:
  staged rather than applied, atlas untouched with all 293 Malas points intact,
  reject then stays quiet on re-run, approve applies and drops the facet.
- Booted the real server under all three source conditions; none blocked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP

server/src/model/shardAtlas/shardAtlas.db.js

@@ -0,0 +1,202 @@
+const { pool, query } = require('../../utils/db')
+
+// Raw SQL for the spawn atlas. Every table here is IMPORT-OWNED: `replaceAtlas`
+// empties and refills all six inside one transaction, and nothing else in the
+// codebase writes to them. There are no foreign keys, consistent with every
+// other shard_* table.
+
+const BATCH = 500
+
+const ATLAS_TABLES = [
+  'shard_spawn_point_types',
+  'shard_spawn_points',
+  'shard_spawn_creatures',
+  'shard_regions',
+  'shard_landmarks',
+  'shard_champion_spawns',
+]
+
+async function insertBatched(conn, sql, rows) {
+  for (let i = 0; i < rows.length; i += BATCH) {
+    await conn.batch(sql, rows.slice(i, i + BATCH))
+  }
+  return rows.length
+}
+
+/**
+ * Replace the entire atlas in one transaction.
+ *
+ * All-or-nothing on purpose: a failed reload must leave the previous atlas
+ * intact rather than a half-loaded world, since a partially-imported atlas is
+ * indistinguishable from a real one to anyone reading it.
+ *
+ * `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and implicitly
+ * commits, which would defeat exactly that guarantee. At ~7k rows the cost of
+ * `DELETE` is irrelevant.
+ */
+async function replaceAtlas(atlas, art = {}) {
+  const conn = await pool.getConnection()
+  const counts = {}
+  try {
+    await conn.beginTransaction()
+
+    for (const table of ATLAS_TABLES) await conn.query(`DELETE FROM ${table}`)
+
+    counts.creatures = await insertBatched(
+      conn,
+      'INSERT INTO shard_spawn_creatures (slug, name, total, points, facets, art) VALUES (?,?,?,?,?,?)',
+      atlas.creatures.map((c) => [
+        c.slug,
+        c.name,
+        c.total ?? 0,
+        c.points ?? 0,
+        JSON.stringify(c.facets ?? {}),
+        art[c.slug] ?? null,
+      ]),
+    )
+
+    counts.regions = await insertBatched(
+      conn,
+      'INSERT INTO shard_regions (facet, name, type, priority, parent, rects) VALUES (?,?,?,?,?,?)',
+      atlas.regions.map((r) => [
+        r.facet,
+        r.name,
+        r.type || null,
+        r.priority ?? 0,
+        r.parent || null,
+        JSON.stringify(r.rects ?? []),
+      ]),
+    )
+
+    counts.landmarks = await insertBatched(
+      conn,
+      'INSERT INTO shard_landmarks (facet, name, grp, x, y, z) VALUES (?,?,?,?,?,?)',
+      atlas.landmarks.map((l) => [
+        l.facet,
+        l.name,
+        l.group || null,
+        l.x ?? 0,
+        l.y ?? 0,
+        l.z ?? 0,
+      ]),
+    )
+
+    counts.champions = await insertBatched(
+      conn,
+      'INSERT INTO shard_champion_spawns ' +
+        '(slug, name, grp, type, random_type, facet, x, y, z, radius, label) ' +
+        'VALUES (?,?,?,?,?,?,?,?,?,?,?)',
+      atlas.champions.map((c) => [
+        c.slug,
+        c.name,
+        c.group || null,
+        c.type || null,
+        c.randomType ? 1 : 0,
+        c.facet,
+        c.x ?? 0,
+        c.y ?? 0,
+        c.z ?? 0,
+        c.radius ?? 0,
+        c.label || null,
+      ]),
+    )
+
+    // Point ids are assigned explicitly rather than left to AUTO_INCREMENT: the
+    // join rows need to know them and `conn.batch()` reports no usable insertId
+    // for a multi-row insert. Safe because this transaction just emptied the
+    // table and nothing else writes to it.
+    counts.points = await insertBatched(
+      conn,
+      'INSERT INTO shard_spawn_points ' +
+        '(id, facet, name, x, y, width, height, spawn_range, max_count, min_delay, max_delay, ' +
+        'tod_start, tod_end, tod_mode, region, landmark, label) ' +
+        'VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)',
+      atlas.points.map((p, i) => [
+        i + 1,
+        p.facet,
+        p.name,
+        p.x,
+        p.y,
+        p.width ?? 0,
+        p.height ?? 0,
+        p.range ?? 0,
+        p.maxCount ?? 0,
+        p.minDelay ?? 0,
+        p.maxDelay ?? 0,
+        p.todStart ?? 0,
+        p.todEnd ?? 0,
+        p.todMode ?? 0,
+        p.region,
+        p.landmark,
+        p.label || 'Wilderness',
+      ]),
+    )
+
+    counts.pointTypes = await insertBatched(
+      conn,
+      'INSERT INTO shard_spawn_point_types (point_id, slug, max_count) VALUES (?,?,?)',
+      atlas.pointTypes,
+    )
+
+    await conn.query(
+      'INSERT INTO shard_atlas_meta (id, payload) VALUES (1, ?) ' +
+        'ON DUPLICATE KEY UPDATE payload = VALUES(payload), imported_at = CURRENT_TIMESTAMP',
+      [JSON.stringify({ ...atlas.meta, importedCounts: counts })],
+    )
+
+    // A completed import answers whatever was pending.
+    await conn.query('DELETE FROM shard_atlas_pending')
+
+    await conn.commit()
+    return counts
+  } catch (err) {
+    await conn.rollback().catch(() => {})
+    throw err
+  } finally {
+    conn.release()
+  }
+}
+
+async function getMeta() {
+  const rows = await query('SELECT payload, imported_at FROM shard_atlas_meta WHERE id = 1')
+  if (rows.length === 0) return null
+  const payload = typeof rows[0].payload === 'string' ? JSON.parse(rows[0].payload) : rows[0].payload
+  return { ...payload, importedAt: rows[0].imported_at }
+}
+
+/** Facet names currently loaded, used to detect a facet disappearing. */
+async function getFacets() {
+  const rows = await query('SELECT DISTINCT facet FROM shard_spawn_points ORDER BY facet')
+  return rows.map((row) => row.facet)
+}
+
+// ── Pending review ─────────────────────────────────────────────────────────
+
+async function getPending() {
+  const rows = await query('SELECT payload, status, detected_at FROM shard_atlas_pending WHERE id = 1')
+  if (rows.length === 0) return null
+  const payload = typeof rows[0].payload === 'string' ? JSON.parse(rows[0].payload) : rows[0].payload
+  return { ...payload, status: rows[0].status, detectedAt: rows[0].detected_at }
+}
+
+async function setPending(payload, status = 'pending') {
+  return query(
+    'INSERT INTO shard_atlas_pending (id, status, payload) VALUES (1, ?, ?) ' +
+      'ON DUPLICATE KEY UPDATE status = VALUES(status), payload = VALUES(payload), ' +
+      'detected_at = CURRENT_TIMESTAMP',
+    [status, JSON.stringify(payload)],
+  )
+}
+
+async function clearPending() {
+  return query('DELETE FROM shard_atlas_pending')
+}
+
+module.exports = {
+  replaceAtlas,
+  getMeta,
+  getFacets,
+  getPending,
+  setPending,
+  clearPending,
+}

server/src/model/shardAtlas/shardAtlas.model.js

@@ -0,0 +1,296 @@
+const fs = require('fs')
+const path = require('path')
+
+const db = require('./shardAtlas.db')
+const settings = require('../settings/settings.model')
+const { slugify } = require('../../utils/spawnAtlasParse')
+const {
+  AtlasSourceError,
+  buildAtlas,
+  hashSources,
+  sameSources,
+} = require('../../utils/spawnAtlasSource')
+const log = require('../../utils/logger')('shardAtlas')
+
+// The spawn atlas, refreshed from the shard's own ServUO tree.
+//
+// The tree is the single source of truth. Nothing is precomputed and committed,
+// because a shard's maps change over its lifetime — facets get added, replaced
+// or renamed — and a snapshot in the repo would go stale against the world
+// players actually see. So the atlas is re-derived on every boot.
+//
+// Two rules govern the boot path:
+//
+//   1. **It never blocks startup.** No configured path, an unreadable path, a
+//      malformed file, a database error — all of it is caught and logged. The
+//      site comes up either way, serving whatever atlas it already had.
+//   2. **A facet disappearing is not applied automatically.** Losing a facet is
+//      the signature of a half-copied or mid-update tree as much as of a real
+//      map change, and the two are indistinguishable from here. The refresh is
+//      staged for a human instead, and an admin approves or rejects it.
+//
+// Everything else — new facets, renamed regions, changed spawns — applies
+// straight away, because none of it can silently destroy data an operator would
+// miss.
+
+const SETTING_KEY = 'spawn_atlas_servuo_path'
+
+/**
+ * Where the ServUO tree lives.
+ *
+ * The admin setting wins over the environment so an operator can point the
+ * atlas at a different tree without a redeploy, matching how the rest of the
+ * shard integration is admin-managed rather than env-configured. `SERVUO_PATH`
+ * remains as the deploy-time default, since the path usually describes a mount
+ * that the deployment sets up.
+ */
+async function getServuoPath() {
+  try {
+    const configured = await settings.get(SETTING_KEY)
+    if (configured && String(configured).trim() !== '') return String(configured).trim()
+  } catch {
+    // Settings unavailable is not fatal — fall through to the env default.
+  }
+  const fromEnv = process.env.SERVUO_PATH
+  return fromEnv && fromEnv.trim() !== '' ? fromEnv.trim() : ''
+}
+
+async function setServuoPath(value, updatedBy = null) {
+  return settings.set(SETTING_KEY, String(value ?? '').trim(), updatedBy)
+}
+
+/**
+ * Optional operator-supplied art map, `{ "<slug>": "<file under uploads/atlas/>" }`.
+ *
+ * Never committed and never shipped — creature sprites come out of the
+ * operator's own client `.mul`/`.uop` files, which are theirs, not ours to
+ * redistribute. Absent (the normal case) every `art` stays NULL and the UI
+ * renders text-only.
+ */
+function loadArtMap(dir = path.join(__dirname, '..', '..', '..', 'db', 'data')) {
+  try {
+    const file = path.join(dir, 'spawnAtlas.art.json')
+    if (!fs.existsSync(file)) return {}
+    const map = JSON.parse(fs.readFileSync(file, 'utf8'))
+    return map && typeof map === 'object' ? map : {}
+  } catch (err) {
+    log.warn('spawn atlas art map could not be read', { error: err.message })
+    return {}
+  }
+}
+
+/**
+ * Flatten each point's types into `shard_spawn_point_types` rows.
+ *
+ * A spawner may legitimately list the same type twice, and the primary key is
+ * (point_id, slug), so duplicates collapse to the larger max rather than
+ * failing the insert.
+ */
+function pointTypeRows(points) {
+  const rows = []
+  points.forEach((point, i) => {
+    const bySlug = new Map()
+    for (const entry of point.types ?? []) {
+      const slug = slugify(entry.type)
+      if (slug === '') continue
+      bySlug.set(slug, Math.max(bySlug.get(slug) ?? 0, entry.max ?? 1))
+    }
+    for (const [slug, max] of bySlug) rows.push([i + 1, slug, max])
+  })
+  return rows
+}
+
+async function applyAtlas(atlas) {
+  return db.replaceAtlas({ ...atlas, pointTypes: pointTypeRows(atlas.points) }, loadArtMap())
+}
+
+/**
+ * Refresh the atlas from the configured ServUO tree.
+ *
+ * Returns a result describing what happened rather than throwing, so the caller
+ * — including the boot path — can log it and move on:
+ *
+ *   `skipped`     no path configured
+ *   `unavailable` path configured but unreadable / missing required files
+ *   `unchanged`   source hashes match the loaded atlas; nothing parsed
+ *   `imported`    parsed and applied
+ *   `needsReview` parsed, but a facet would be lost; staged for an admin
+ *   `failed`      parsed or applied and something went wrong
+ *
+ * `force` skips the hash check (an admin asking for a reimport) and `approve`
+ * additionally accepts facet loss (an admin approving a staged refresh).
+ */
+async function refresh({ force = false, approve = false, path: pathOverride = '' } = {}) {
+  // An explicit override wins outright — it is a one-off "use this tree", and it
+  // must not be silently overruled by the configured path the way an env default
+  // would be.
+  const root = pathOverride.trim() !== '' ? pathOverride.trim() : await getServuoPath()
+  if (root === '') return { status: 'skipped', reason: 'no ServUO path configured' }
+
+  let hashes
+  try {
+    hashes = hashSources(root)
+  } catch (err) {
+    if (err instanceof AtlasSourceError) {
+      return { status: 'unavailable', reason: err.message, code: err.code, path: root }
+    }
+    return { status: 'failed', reason: err.message, path: root }
+  }
+
+  const meta = await db.getMeta().catch(() => null)
+  const loaded = meta?.source
+    ? Object.fromEntries(Object.entries(meta.source).map(([label, v]) => [label, v.sha256]))
+    : null
+
+  if (!force && sameSources(hashes, loaded)) {
+    return { status: 'unchanged', path: root }
+  }
+
+  // A rejected refresh must not re-prompt on every boot. It stays rejected until
+  // the tree changes again, at which point the hashes differ and it is a new
+  // decision.
+  const pending = await db.getPending().catch(() => null)
+  if (!approve && !force && pending?.status === 'rejected' && sameSources(hashes, pending.hashes)) {
+    return { status: 'unchanged', path: root, reason: 'refresh previously rejected' }
+  }
+
+  let atlas
+  try {
+    atlas = buildAtlas(root)
+  } catch (err) {
+    return { status: 'failed', reason: err.message, path: root }
+  }
+
+  const currentFacets = await db.getFacets().catch(() => [])
+  const incomingFacets = atlas.facets
+  const removedFacets = currentFacets.filter((facet) => !incomingFacets.includes(facet))
+  const addedFacets = incomingFacets.filter((facet) => !currentFacets.includes(facet))
+
+  // Losing a facet is indistinguishable here from a half-copied tree, so it is
+  // staged rather than applied — but startup is never blocked by it.
+  if (removedFacets.length > 0 && !approve) {
+    const summary = {
+      hashes,
+      path: root,
+      currentFacets,
+      incomingFacets,
+      removedFacets,
+      addedFacets,
+      counts: atlas.meta.counts,
+    }
+    await db.setPending(summary, 'pending').catch((err) => {
+      log.warn('could not stage spawn atlas refresh', { error: err.message })
+    })
+    return { status: 'needsReview', ...summary }
+  }
+
+  try {
+    const counts = await applyAtlas(atlas)
+    return { status: 'imported', path: root, counts, addedFacets, removedFacets }
+  } catch (err) {
+    return { status: 'failed', reason: err.message, path: root }
+  }
+}
+
+/** Admin approved a staged refresh: apply it, facet loss and all. */
+async function approvePending(options = {}) {
+  return refresh({ ...options, approve: true, force: true })
+}
+
+/**
+ * Admin rejected a staged refresh: keep the current atlas and remember the
+ * decision against those exact source hashes, so it does not re-prompt every
+ * boot. A further change to the tree produces different hashes and asks again.
+ */
+async function rejectPending() {
+  const pending = await db.getPending()
+  if (!pending) return { status: 'none' }
+  await db.setPending({ ...pending, rejectedAt: new Date().toISOString() }, 'rejected')
+  return { status: 'rejected' }
+}
+
+/** Everything the admin panel needs to describe atlas state. */
+async function status({ path: pathOverride = '' } = {}) {
+  const root = pathOverride.trim() !== '' ? pathOverride.trim() : await getServuoPath()
+  const [meta, pending, facets] = await Promise.all([
+    db.getMeta().catch(() => null),
+    db.getPending().catch(() => null),
+    db.getFacets().catch(() => []),
+  ])
+
+  let treeReadable = false
+  let drift = null
+  if (root !== '') {
+    try {
+      const hashes = hashSources(root)
+      treeReadable = true
+      const loaded = meta?.source
+        ? Object.fromEntries(Object.entries(meta.source).map(([l, v]) => [l, v.sha256]))
+        : null
+      drift = !sameSources(hashes, loaded)
+    } catch {
+      treeReadable = false
+    }
+  }
+
+  return {
+    configured: root !== '',
+    path: root,
+    treeReadable,
+    drift,
+    facets,
+    importedAt: meta?.importedAt ?? null,
+    counts: meta?.counts ?? null,
+    pending,
+  }
+}
+
+/**
+ * Boot hook. Best-effort by contract: it logs and returns, never throws, so a
+ * missing tree or a bad file can never stop the site coming up.
+ */
+async function refreshOnBoot() {
+  try {
+    const result = await refresh()
+    switch (result.status) {
+      case 'imported':
+        log.info('spawn atlas refreshed from ServUO tree', {
+          ...result.counts,
+          added: result.addedFacets,
+        })
+        break
+      case 'needsReview':
+        log.warn(
+          'spawn atlas refresh staged for admin review — a facet would be removed; ' +
+            'the existing atlas is unchanged',
+          { removed: result.removedFacets, added: result.addedFacets },
+        )
+        break
+      case 'unavailable':
+        log.warn('spawn atlas source unavailable', { reason: result.reason, path: result.path })
+        break
+      case 'failed':
+        log.warn('spawn atlas refresh failed', { reason: result.reason })
+        break
+      default:
+        break
+    }
+    return result
+  } catch (err) {
+    log.warn('spawn atlas refresh errored', { error: err.message })
+    return { status: 'failed', reason: err.message }
+  }
+}
+
+module.exports = {
+  refresh,
+  refreshOnBoot,
+  approvePending,
+  rejectPending,
+  status,
+  getServuoPath,
+  setServuoPath,
+  pointTypeRows,
+  loadArtMap,
+  SETTING_KEY,
+}