docs/website/SPAWN_ATLAS.md

# Spawn atlas

**Status:** Data pipeline landed on `edge` (website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112)); API and client pages follow in a second PR.
**Design:** [`docs/link/v3.md` §6](../link/v3.md) — Protocol 3.0 Part C.

The spawn atlas is a browsable catalogue of what the shard *contains*: which
creatures spawn, where, how many, and which champion altars are configured. It
answers "where do I find a lizardman?" with **"Shrines, Yew, Isamu-Jima"** rather
than with a list of raw coordinates.

## Two things that shape the whole design

**The shard's ServUO tree is the single source of truth.** Nothing is
precomputed and committed to the repository. A shard's maps change over its
lifetime — facets get added, replaced, or renamed — and a snapshot in the repo
would silently drift from the world players actually see. The atlas is therefore
re-derived from the tree **on every server boot**.

**Facets are not a fixed list.** Nothing in the codebase names Felucca, Trammel,
or any other stock facet. The facet set is whatever the shard's own files
declare, discovered at parse time. A shard running entirely custom maps gets
exactly the same treatment as a stock one, with no code change.

## What it is not

The atlas is **static shard content, not live shard state.**

- It does **not** come from the sidecar. Nothing here touches the bridge, and
  there is no event kind, no wire change and no `PROTOCOL_VERSION` bump for it.
  Part C is website-only.
- It stays fully populated while the shard is down.
- Its champion table (`shard_champion_spawns`) is the *configured roster* —
  "there is an Unholy Terror altar in Deceit". The live `champ.update` feed in
  `shard_champs` is the separate, sidecar-fed answer to "it is on level 3 right
  now". Both exist; do not conflate them.

Routes live at `/api/v1/public/atlas`, deliberately **not** under `/shard`,
because `/shard/*` means sidecar-dependent.

## Configuring the tree

The website needs to be able to *read* the ServUO tree — same host, a bind mount,
or a shared volume. Two ways to point at it, the setting winning over the
environment:

| Source | Notes |
|---|---|
| `spawn_atlas_servuo_path` setting | Admin-editable; changes take effect on the next refresh without a redeploy |
| `SERVUO_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |

With neither set the atlas is simply skipped — the site runs normally without
one.

## The boot path

On every start the server hashes the source files and compares them against what
is loaded. Unchanged (the normal case on a restart) costs one read pass, ~120 ms,
and no database write. A real change costs a ~400 ms parse and a reload.

Two contracts govern it:

**1. It never blocks startup.** No configured path, an unreadable mount, a
malformed file, a database error — every one is caught and logged, and the site
comes up serving whatever atlas it already had.

**2. A facet disappearing is never applied automatically.** Losing a facet looks
exactly like a half-copied or mid-update tree, and boot cannot tell that apart
from a real map change. That refresh is *staged* for a human instead. Everything
else — new facets, renamed regions, changed spawns — applies immediately, since
none of it can destroy something an operator would miss.

```
boot
 └─ path configured?            no ──▶ skip
     └─ tree readable?          no ──▶ warn, carry on
         └─ hashes changed?     no ──▶ done (nothing parsed)
             └─ parse
                 └─ a facet would be removed?
                      no ──▶ import
                     yes ──▶ stage for admin review; atlas unchanged
```

### Approving or rejecting a staged refresh

Only the *decision* is stored, never the parsed world — a few KB of source hashes
plus the facet diff. Approving **re-parses** the tree, so what lands matches the
tree at approval time rather than at boot, and a multi-megabyte blob never sits
in the database.

A rejection is remembered against those exact source hashes, so a declined
refresh does not re-prompt on every restart. Change the tree and the hashes
differ, which asks again.

From the admin panel (second PR), or from the CLI:

```bash
cd website/server
npm run atlas:import -- --status     # what is loaded, and what is pending
npm run atlas:import -- --approve    # apply the staged refresh
npm run atlas:import -- --reject     # keep the current atlas, dismiss it
```

## The CLI

The server refreshes itself on boot, so this is for applying a map change
*without* a restart, and for the approve/reject flow above.

```bash
npm run atlas:import                       # import if the tree differs
npm run atlas:import -- --servuo <path>    # override the path for this run
npm run atlas:import -- --force            # reimport even if unchanged
```

`--servuo` is a per-run override and deliberately does **not** persist — changing
where the atlas permanently reads from is an admin action, not a side effect of a
one-off import.

## Sources

| File | Count (stock ServUO 57.4) | Used for |
|---|---|---|
| `Spawns/*.xml` | 13 files, ~10.5 MB | Every spawner: location, size, delays, time-of-day, creature types |
| `Data/Regions.xml` | 129 KB, nested | Named regions and their rectangles |
| `Data/Locations/*.xml` | 6 files | Landmarks (dungeon levels, town markers) |
| `Config/ChampionSpawns.xml` | 4.8 KB | Configured champion altars |

**A stock tree has 13 spawn files but only 6 facets.** `Eodon.xml`,
`GravewaterLake.xml`, `TreasuresOfKotl.xml` and the other named-area files hold
TerMur/Trammel points. The facet always comes from each record's own `<Map>`,
never from the file name.

## How a coordinate becomes a place name

This is the transform the atlas exists for, in `resolveRegion()`:

1. The highest-`priority` named region whose rectangle contains the point. Ties
   break toward the **smallest** rect, so a specific room wins over the
   dungeon-wide rect enclosing it.
2. Otherwise the nearest landmark within the landmark radius (200 tiles by
   default), labelled by its **group** ("Covetous"), not its individual marker
   ("Level 1").
3. Otherwise `"Wilderness"`.

The radius cap in step 2 is what keeps step 3 reachable. Without it the nearest
landmark is always *some* landmark however far away, and open countryside gets
labelled with a dungeon on the far side of the map.

Against stock ServUO this resolves **83.2%** of points (5,369 of 6,455): 3,681 by
region, 1,688 by landmark, 1,086 Wilderness.

## Three quirks in the source data

Each of these is silent if unhandled — the atlas still builds, it is just wrong.

**Facet names disagree between sources.** `Data/Locations/*.xml` spells them
`Ter Mur` and `Tokuno Islands`, while `<Map>` and `<Facet name>` say `TerMur` and
`Tokuno`. Unreconciled, the landmark bucket is keyed differently from the points
looking it up, so the fallback never fires and every unregioned spawn on those
facets reads "Wilderness".

This is reconciled **by matching, not by a lookup table** — there is no list of
facet names anywhere. `facetKey()` collapses spelling differences (lowercase,
alphanumerics only), and `resolveFacetName()` matches a loose spelling against
the canonical set discovered from the shard's own spawn and region data, by exact
key then by prefix in either direction. A name matching nothing keeps its own
name: forcing a wrong match would file a real custom facet's landmarks under the
wrong facet, which is worse than leaving it alone.

**Spawn type tokens carry XmlSpawner directives.** The `<Objects2>` type is not
always a bare class name:

```
Fairy,{RND,4,8}      alchemist/z/-50      Agralem/Name/Agralem
GargishRouser,1      greatape,true        GargishRefugee/hue/34532
```

Taken literally these invent creatures that do not exist *and* split real ones in
two, because `Fairy` and `Fairy,{RND,4,8}` slug apart into separate entries. 71 of
845 were affected. Everything from the first `/` or `,` is stripped, leaving 800
real creatures.

**Case is inconsistent across files.** The same creature is `Lizardman` in one
file and `lizardman` in another. Slugging collapses them correctly, but the
display name is chosen deterministically — most common spelling wins, ties break
to the more capitalised form, then alphabetically — because otherwise it would
depend on file read order and change on an unrelated restart.

## Tables

All are **import-owned**: a refresh empties and reloads them in one transaction,
so a failed reload leaves the previous atlas intact rather than a half-loaded
world. Nothing else writes to them and nothing holds a foreign key to them — no
FKs at all, consistent with every other `shard_*` table. Full column listings in
[`BACKEND_DESIGN.md`](BACKEND_DESIGN.md).

| Table | Rows (stock) | Notes |
|---|---|---|
| `shard_spawn_creatures` | 800 | `slug` PK; `total` = sum of each type's own max; nullable `art` |
| `shard_spawn_points` | 6,455 | `spawn_range`, since `range` is reserved in MariaDB |
| `shard_spawn_point_types` | 23,927 | The many-to-many; one spawner commonly carries six types |
| `shard_regions` | 387 | Flattened out of the nesting; `rects` JSON |
| `shard_landmarks` | 558 | `grp`, since `group` is reserved in SQL |
| `shard_champion_spawns` | 25 | Configured altars, not the live feed |
| `shard_atlas_meta` | 1 | Singleton; source hashes, for the change check |
| `shard_atlas_pending` | 0–1 | Singleton; a staged refresh awaiting admin review |

`shard_spawn_creatures.name` carries a plain `INDEX`, deliberately **not
`FULLTEXT`**: ~800 rows makes a `LIKE` scan free, and FULLTEXT's minimum token
length would break searches for names like "orc".

The reload uses `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and would
implicitly commit, defeating the all-or-nothing guarantee. Point ids are assigned
explicitly rather than left to `AUTO_INCREMENT`, because the join rows need them
and `conn.batch()` reports no usable `insertId`.

## Artwork — operator-supplied, never shipped

**This project ships no creature art and no extraction tooling, and never will.**
UO sprites live in the operator's own client `.mul`/`.uop` files. They are the
operator's, not ours to redistribute.

The atlas is fully functional as text. `shard_spawn_creatures.art` is nullable
and is NULL on every fresh import; pages render without images, which is the
normal and supported state, not a degraded one.

An operator who wants art:

1. Extracts it from **their own** client files (UOFiddler, ClassicUO tooling, or
   any art extractor).
2. Drops the images under `server/uploads/atlas/`.
3. Copies `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json`
   and maps creature slugs to file names.
4. Restarts, or runs `npm run atlas:import -- --force`.

Both `spawnAtlas.art.json` and `server/uploads/` are gitignored, so neither the
map nor the images can be committed by accident.

## Code layout

| File | Role |
|---|---|
| `src/utils/spawnAtlasParse.js` | **Pure and fs-free** parsers, so CI covers them with no ServUO tree. Zero dependencies. |
| `src/utils/spawnAtlasSource.js` | The only thing that reads a ServUO tree; shared by the boot path and the CLI |
| `src/model/shardAtlas/shardAtlas.db.js` | The one-transaction replace |
| `src/model/shardAtlas/shardAtlas.model.js` | The refresh decision, staging, approve/reject |
| `scripts/importSpawnAtlas.js` | Thin CLI over the model |

Parsing notes:

- `Regions.xml`, `Locations/*.xml` and `ChampionSpawns.xml` genuinely nest, and
  get a small hand-rolled **subset** tokenizer — elements, attributes,
  self-closing tags, comments, the XML declaration, CDATA, and the five
  predefined entities plus numeric refs. It is not a general-purpose XML parser
  and must not be reused as one.
- The ~10.5 MB of `Spawns/*.xml` never touches that tokenizer. Those records are
  flat, so they get a streaming regex sweep instead; a DOM would allocate a node
  per element across ~40 fields on every record to keep 14 of them. **Do not put
  the Points files through a DOM parser.**
- `<Objects2>` is `Type:MX=n:SB=…` segments joined by `:OBJ=`. Split on `:OBJ=`
  *first* — a naive `split(':')` shreds it. A single Trammel point carries six
  types.