refactor(atlas): derive the atlas from the shard's tree on every boot

Replaces the committed-artifact design from the first commit. Two problems with
it, both raised in review:

**Facets are not a fixed list.** The first pass carried a hardcoded table of the
six stock UO facets to reconcile the spelling drift between sources. That is
wrong: a shard may add facets, replace them outright, or rename them when its
maps are updated, and a built-in list quietly mishandles all three. Nothing in
the atlas names a facet any more. The facet set is discovered from the tree —
spawn records and region definitions are the authority — and the loose spellings
in Data/Locations are matched against it by key and prefix. Custom facets get
identical treatment; the tests use `Sosaria` and `Underdark` precisely so a
stock-facet assumption cannot creep back in.

**A snapshot goes stale.** Maps change over a server's life, so a build-once
artifact silently drifts from the world players actually see. The tree is now
the single source of truth and the atlas is re-derived on every boot.

## What that changed

- **The committed artifact is gone** — 1.41 MB of generated JSON removed, along
  with `scripts/buildSpawnAtlas.js` and the whole encode/decode seam it needed
  (`encodePoint`/`readPoint`, the tuple encoding, the omitted-defaults scheme and
  their round-trip tests). Nothing to keep in sync, nothing to go stale.
- **NEW `src/utils/spawnAtlasSource.js`** — the only thing that touches a ServUO
  tree; shared by the boot path and the CLI. Parsers stay pure and fs-free.
- **NEW `src/model/shardAtlas/`** — `.db.js` (the one-transaction replace) and
  `.model.js` (the refresh decision).
- **`scripts/importSpawnAtlas.js`** is now a thin CLI over the model:
  `--servuo`, `--force`, `--approve`, `--reject`, `--status`. `atlas:build` is
  gone; `atlas:import` remains.
- Path comes from the `spawn_atlas_servuo_path` admin setting, falling back to
  `SERVUO_PATH`. The setting wins, matching how the rest of the shard
  integration is admin-managed rather than env-configured.

## Two contracts on the boot path

**It never blocks startup.** No path, an unreadable mount, a malformed file, a
database error — every one is caught and logged, and the site comes up serving
whatever atlas it already had. Verified by booting the real server with no path,
a broken path, and a good path.

**A facet disappearing is never applied automatically.** Losing a facet is the
signature of a half-copied or mid-update tree as much as of a real map change,
and boot cannot tell them apart. The refresh is staged in `shard_atlas_pending`
for an admin to approve or reject, and startup continues regardless. Additions
and every other change apply immediately, since none of them can destroy
something an operator would miss.

Only the decision is stored, not the parsed world: a few KB of source hashes and
the facet diff. Approving re-parses, so what gets applied matches the tree at
approval time rather than at boot. A rejection is remembered against those exact
hashes, so a declined refresh does not re-prompt on every restart — changing the
tree changes the hashes and asks again.

Hash-gated, so the common case (restart, maps unchanged) reads and hashes the
tree (~120 ms) and writes nothing. A real change costs a ~400 ms parse.

The admin approve/reject UI is part of the second PR, with the rest of the
routes and pages. Until then the CLI covers it.

## Verification

- **564 server tests pass**, 28 new in `spawnAtlas.source.test.js` covering the
  custom-facet build, the spelling reconciliation, hash gating, and every branch
  of the refresh decision — including that `refreshOnBoot` survives a database
  that throws on every call.
- End-to-end against the local MariaDB and the real ServUO tree: 6,455 points,
  800 creatures, 23,927 point/type rows, 387 regions, 558 landmarks, 25 altars,
  83.2% of points resolved to a place name.
- The facet gate exercised against a real tree copy with `malas.xml` removed:
  staged rather than applied, atlas untouched with all 293 Malas points intact,
  reject then stays quiet on re-run, approve applies and drops the facet.
- Booted the real server under all three source conditions; none blocked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP

server/scripts/importSpawnAtlas.js

@@ -1,309 +1,118 @@
 #!/usr/bin/env node
 //
-// Load the committed spawn atlas artifact into the database.
+// Refresh the spawn atlas from a ServUO tree, from the command line.
 //
-//   npm run atlas:import [-- --dir db/data]
+//   npm run atlas:import                       # use the configured path
+//   npm run atlas:import -- --servuo <path>    # override it for this run
+//   npm run atlas:import -- --force            # reimport even if unchanged
+//   npm run atlas:import -- --approve          # apply a staged refresh
+//   npm run atlas:import -- --status           # report without changing anything
 //
-// Deliberately separate from `atlas:build`: building needs a ServUO tree, which
-// the website container does not have, while importing needs only the committed
-// JSON and a database. That split is what lets the atlas ship in the image and
-// be loaded on any deployment.
+// The server does this itself on every boot (see `shardAtlas.refreshOnBoot`), so
+// this is for operators who want to apply a map change without a restart, and
+// for approving a refresh that was staged because it would remove a facet.
 //
-// The atlas tables are import-owned. This TRUNCATEs and reloads all of them
-// inside ONE transaction, so a failed import leaves the previous atlas intact
-// rather than a half-loaded world — there is no partial state worth keeping,
-// since the artifact is the whole truth.
+// All the logic lives in `src/model/shardAtlas/shardAtlas.model.js`; this file
+// is argument parsing and output formatting.
 
-const fs = require('fs')
-const path = require('path')
-
-// Slugging must match the build's exactly, so it comes from the same module.
-const { slugify: slugOf } = require('../src/utils/spawnAtlasParse')
-
-// `src/utils/db` is required lazily, inside the functions that actually talk to
-// the database, rather than at module load. Requiring it opens a connection
-// pool as a side effect, and readPoint() — which the tests round-trip against
-// buildSpawnAtlas.encodePoint() — is pure. Loading it eagerly made the unit
-// tests hold a pool open against a dead port for ~11 seconds.
 const db = () => require('../src/utils/db')
 
-const DEFAULT_DIR = path.join(__dirname, '..', 'db', 'data')
-const BATCH = 500
-
-// ── Artifact reading ───────────────────────────────────────────────────────
-
-function readJson(file) {
-  return JSON.parse(fs.readFileSync(file, 'utf8'))
-}
-
-/**
- * Decode one point record back to the full shape.
- *
- * The exact mirror of the two space-saving encodings buildSpawnAtlas.js applies
- * on write — omitted-when-default fields, and `types` as [name, max] tuples.
- * If one side changes, this must change with it.
- */
-function readPoint(raw, facet) {
-  const region = raw.region ?? null
-  const landmark = raw.landmark ?? null
-  return {
-    facet,
-    name: raw.name ?? null,
-    x: raw.x ?? 0,
-    y: raw.y ?? 0,
-    width: raw.width ?? 0,
-    height: raw.height ?? 0,
-    range: raw.range ?? 0,
-    maxCount: raw.maxCount ?? 0,
-    minDelay: raw.minDelay ?? 0,
-    maxDelay: raw.maxDelay ?? 0,
-    todStart: raw.todStart ?? 0,
-    todEnd: raw.todEnd ?? 0,
-    todMode: raw.todMode ?? 0,
-    region,
-    landmark,
-    // Recomputed rather than stored — it is exactly this expression, and the
-    // build omits it for that reason.
-    label: region || landmark || 'Wilderness',
-    types: (raw.types ?? []).map((entry) =>
-      Array.isArray(entry) ? { type: entry[0], max: entry[1] } : entry,
-    ),
-  }
-}
-
-function loadArtifact(dir) {
-  const indexFile = path.join(dir, 'spawnAtlas.index.json')
-  const metaFile = path.join(dir, 'spawnAtlas.meta.json')
-  if (!fs.existsSync(indexFile)) {
-    throw new Error(
-      `No atlas artifact at ${indexFile}. Build one first:\n` +
-        '  npm run atlas:build -- --servuo <path to ServUO>',
-    )
-  }
-
-  const index = readJson(indexFile)
-  const meta = fs.existsSync(metaFile) ? readJson(metaFile) : {}
-
-  const points = []
-  for (const facet of index.facets ?? []) {
-    const file = path.join(dir, `spawnAtlas.${facet}.json`)
-    if (!fs.existsSync(file)) {
-      throw new Error(`Artifact is incomplete: ${path.basename(file)} is missing`)
-    }
-    const shard = readJson(file)
-    for (const raw of shard.points ?? []) points.push(readPoint(raw, shard.facet || facet))
-  }
-
-  return { index, meta, points }
-}
-
-/**
- * Optional operator-supplied art map, `{ "<slug>": "<file under uploads/atlas/>" }`.
- *
- * Never committed and never shipped — creature sprites come out of the
- * operator's own client `.mul`/`.uop` files, which are theirs, not ours. Absent
- * (the normal case) every `art` stays NULL and the UI renders text-only.
- * See docs/website/SPAWN_ATLAS.md.
- */
-function loadArtMap(dir) {
-  const file = path.join(dir, 'spawnAtlas.art.json')
-  if (!fs.existsSync(file)) return {}
-  const map = readJson(file)
-  return map && typeof map === 'object' ? map : {}
-}
-
-// ── Insert helpers ─────────────────────────────────────────────────────────
-
-async function insertBatched(conn, sql, rows) {
-  for (let i = 0; i < rows.length; i += BATCH) {
-    await conn.batch(sql, rows.slice(i, i + BATCH))
-  }
-  return rows.length
-}
-
-// ── Import ─────────────────────────────────────────────────────────────────
-
-async function importAtlas(dir) {
-  const { index, meta, points } = loadArtifact(dir)
-  const art = loadArtMap(dir)
-  const conn = await db().pool.getConnection()
-  const counts = {}
-
-  try {
-    await conn.beginTransaction()
-
-    // TRUNCATE is DDL in MariaDB and would commit the transaction implicitly,
-    // defeating the all-or-nothing guarantee. DELETE is transactional, which is
-    // what this needs; at ~7k rows the difference is not worth the atomicity.
-    for (const table of [
-      'shard_spawn_point_types',
-      'shard_spawn_points',
-      'shard_spawn_creatures',
-      'shard_regions',
-      'shard_landmarks',
-      'shard_champion_spawns',
-    ]) {
-      await conn.query(`DELETE FROM ${table}`)
-    }
-    await conn.query('ALTER TABLE shard_spawn_points AUTO_INCREMENT = 1')
-
-    counts.creatures = await insertBatched(
-      conn,
-      'INSERT INTO shard_spawn_creatures (slug, name, total, points, facets, art) VALUES (?,?,?,?,?,?)',
-      (index.creatures ?? []).map((c) => [
-        c.slug,
-        c.name,
-        c.total ?? 0,
-        c.points ?? 0,
-        JSON.stringify(c.facets ?? {}),
-        art[c.slug] ?? null,
-      ]),
-    )
-
-    counts.regions = await insertBatched(
-      conn,
-      'INSERT INTO shard_regions (facet, name, type, priority, parent, rects) VALUES (?,?,?,?,?,?)',
-      (index.regions ?? []).map((r) => [
-        r.facet,
-        r.name,
-        r.type || null,
-        r.priority ?? 0,
-        r.parent || null,
-        JSON.stringify(r.rects ?? []),
-      ]),
-    )
-
-    counts.landmarks = await insertBatched(
-      conn,
-      'INSERT INTO shard_landmarks (facet, name, grp, x, y, z) VALUES (?,?,?,?,?,?)',
-      (index.landmarks ?? []).map((l) => [
-        l.facet,
-        l.name,
-        l.group || null,
-        l.x ?? 0,
-        l.y ?? 0,
-        l.z ?? 0,
-      ]),
-    )
-
-    counts.champions = await insertBatched(
-      conn,
-      'INSERT INTO shard_champion_spawns ' +
-        '(slug, name, grp, type, random_type, facet, x, y, z, radius, label) ' +
-        'VALUES (?,?,?,?,?,?,?,?,?,?,?)',
-      (index.champions ?? []).map((c) => [
-        c.slug,
-        c.name,
-        c.group || null,
-        c.type || null,
-        c.randomType ? 1 : 0,
-        c.facet,
-        c.x ?? 0,
-        c.y ?? 0,
-        c.z ?? 0,
-        c.radius ?? 0,
-        c.label || null,
-      ]),
-    )
-
-    // Point ids are assigned here rather than left to AUTO_INCREMENT, because
-    // the join rows need to know them and `conn.batch()` reports no usable
-    // insertId for a multi-row insert. Assigning them explicitly is safe — the
-    // atlas tables are import-owned and this transaction just emptied them — and
-    // it makes the ids a pure function of artifact order instead of something
-    // derived from driver behaviour.
-    const pointRows = points.map((p, i) => [
-      i + 1,
-      p.facet,
-      p.name,
-      p.x,
-      p.y,
-      p.width,
-      p.height,
-      p.range,
-      p.maxCount,
-      p.minDelay,
-      p.maxDelay,
-      p.todStart,
-      p.todEnd,
-      p.todMode,
-      p.region,
-      p.landmark,
-      p.label,
-    ])
-    counts.points = await insertBatched(
-      conn,
-      'INSERT INTO shard_spawn_points ' +
-        '(id, facet, name, x, y, width, height, spawn_range, max_count, min_delay, max_delay, ' +
-        'tod_start, tod_end, tod_mode, region, landmark, label) ' +
-        'VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)',
-      pointRows,
-    )
-
-    const typeRows = []
-    points.forEach((point, i) => {
-      // A spawner may legitimately list the same type twice (six-type points
-      // repeating a creature). The primary key is (point_id, slug), so collapse
-      // duplicates to the larger max rather than letting the insert fail.
-      const bySlug = new Map()
-      for (const entry of point.types) {
-        const slug = slugOf(entry.type)
-        if (slug === '') continue
-        bySlug.set(slug, Math.max(bySlug.get(slug) ?? 0, entry.max ?? 1))
-      }
-      for (const [slug, max] of bySlug) typeRows.push([i + 1, slug, max])
-    })
-    counts.pointTypes = await insertBatched(
-      conn,
-      'INSERT INTO shard_spawn_point_types (point_id, slug, max_count) VALUES (?,?,?)',
-      typeRows,
-    )
-
-    await conn.query(
-      'INSERT INTO shard_atlas_meta (id, payload) VALUES (1, ?) ' +
-        'ON DUPLICATE KEY UPDATE payload = VALUES(payload), imported_at = CURRENT_TIMESTAMP',
-      [JSON.stringify({ ...meta, importedCounts: counts })],
-    )
-
-    await conn.commit()
-    return counts
-  } catch (err) {
-    await conn.rollback().catch(() => {})
-    throw err
-  } finally {
-    conn.release()
-  }
-}
-
-// ── CLI ────────────────────────────────────────────────────────────────────
-
 function parseArgs(argv) {
-  const args = { dir: DEFAULT_DIR }
+  const args = {}
   for (let i = 0; i < argv.length; i += 1) {
-    if (argv[i] === '--dir') args.dir = argv[++i]
-    else if (argv[i] === '--help' || argv[i] === '-h') args.help = true
+    const flag = argv[i]
+    if (flag === '--servuo') args.servuo = argv[++i]
+    else if (flag === '--force') args.force = true
+    else if (flag === '--approve') args.approve = true
+    else if (flag === '--reject') args.reject = true
+    else if (flag === '--status') args.status = true
+    else if (flag === '--help' || flag === '-h') args.help = true
   }
   return args
 }
 
+const USAGE = `
+Refresh the spawn atlas from a ServUO tree.
+
+  node scripts/importSpawnAtlas.js [options]
+
+  --servuo <path>  Use this tree for this run instead of the configured path.
+  --force          Reimport even when the source files are unchanged.
+  --approve        Apply a refresh that was staged for removing a facet.
+  --reject         Keep the current atlas and dismiss the staged refresh.
+  --status         Report atlas and source state; change nothing.
+
+With no options this imports only if the tree differs from what is loaded.
+`
+
+function describe(result) {
+  switch (result.status) {
+    case 'skipped':
+      return (
+        'No ServUO path configured — nothing to import.\n' +
+        'Set one with SERVUO_PATH, the admin panel, or --servuo <path>.\n'
+      )
+    case 'unavailable':
+      return `ServUO tree unavailable: ${result.reason}\n`
+    case 'unchanged':
+      return `Atlas is already up to date${result.reason ? ` (${result.reason})` : ''}.\n`
+    case 'needsReview': {
+      return (
+        'Refresh NOT applied — it would remove ' +
+        `${result.removedFacets.length} facet(s): ${result.removedFacets.join(', ')}.\n` +
+        'This is what a half-copied or mid-update tree looks like, so it has been\n' +
+        'staged for review. The current atlas is unchanged.\n' +
+        'Apply it with --approve, or dismiss it with --reject.\n'
+      )
+    }
+    case 'imported': {
+      const c = result.counts
+      const added = result.addedFacets?.length ? ` Added facets: ${result.addedFacets.join(', ')}.` : ''
+      const removed = result.removedFacets?.length
+        ? ` Removed facets: ${result.removedFacets.join(', ')}.`
+        : ''
+      return (
+        `Atlas imported: ${c.points} points, ${c.creatures} creatures, ` +
+        `${c.pointTypes} point/type rows, ${c.regions} regions, ` +
+        `${c.landmarks} landmarks, ${c.champions} champion altars.${added}${removed}\n`
+      )
+    }
+    case 'failed':
+      return `Atlas refresh failed: ${result.reason}\n`
+    default:
+      return `${JSON.stringify(result, null, 2)}\n`
+  }
+}
+
 async function main() {
   const args = parseArgs(process.argv.slice(2))
   if (args.help) {
-    process.stdout.write(
-      '\nLoad the committed spawn atlas artifact into the database.\n\n' +
-        '  node scripts/importSpawnAtlas.js [--dir <artifact dir>]\n\n' +
-        `  --dir   Artifact directory. Default: ${DEFAULT_DIR}\n\n`,
-    )
+    process.stdout.write(USAGE)
     return
   }
 
-  const counts = await importAtlas(args.dir)
-  require('../src/utils/logger')('atlas:import').info('spawn atlas imported', counts)
-  process.stdout.write(
-    `Atlas imported: ${counts.points} points, ${counts.creatures} creatures, ` +
-      `${counts.pointTypes} point/type rows, ${counts.regions} regions, ` +
-      `${counts.landmarks} landmarks, ${counts.champions} champion altars.\n`,
-  )
+  const shardAtlas = require('../src/model/shardAtlas/shardAtlas.model')
+
+  // `--servuo` is a per-run override and deliberately does NOT persist to the
+  // configured path; changing where the atlas permanently reads from is an
+  // admin action, not a side effect of a one-off import.
+  const override = { path: args.servuo ?? '' }
+
+  if (args.status) {
+    process.stdout.write(`${JSON.stringify(await shardAtlas.status(override), null, 2)}\n`)
+    return
+  }
+  if (args.reject) {
+    process.stdout.write(`${JSON.stringify(await shardAtlas.rejectPending(), null, 2)}\n`)
+    return
+  }
+
+  const result = args.approve
+    ? await shardAtlas.approvePending(override)
+    : await shardAtlas.refresh({ ...override, force: Boolean(args.force) })
+
+  process.stdout.write(describe(result))
+  if (result.status === 'failed') process.exitCode = 1
 }
 
 if (require.main === module) {
@@ -315,4 +124,4 @@ if (require.main === module) {
     .finally(() => db().close())
 }
 
-module.exports = { importAtlas, readPoint, loadArtifact }
+module.exports = { describe, parseArgs }