# Spawn atlas **Status:** Data pipeline landed on `edge` (website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112)); API and client pages follow in a second PR. **Design:** [`docs/link/v3.md` §6](../link/v3.md) — Protocol 3.0 Part C. The spawn atlas is a browsable catalogue of what the shard *contains*: which creatures spawn, where, how many, and which champion altars are configured. It answers "where do I find a lizardman?" with **"Shrines, Yew, Isamu-Jima"** rather than with a list of raw coordinates. ## Two things that shape the whole design **The shard's ServUO tree is the single source of truth.** Nothing is precomputed and committed to the repository. A shard's maps change over its lifetime — facets get added, replaced, or renamed — and a snapshot in the repo would silently drift from the world players actually see. The atlas is therefore re-derived from the tree **on every server boot**. **Facets are not a fixed list.** Nothing in the codebase names Felucca, Trammel, or any other stock facet. The facet set is whatever the shard's own files declare, discovered at parse time. A shard running entirely custom maps gets exactly the same treatment as a stock one, with no code change. ## What it is not The atlas is **static shard content, not live shard state.** - It does **not** come from the sidecar. Nothing here touches the bridge, and there is no event kind, no wire change and no `PROTOCOL_VERSION` bump for it. Part C is website-only. - It stays fully populated while the shard is down. - Its champion table (`shard_champion_spawns`) is the *configured roster* — "there is an Unholy Terror altar in Deceit". The live `champ.update` feed in `shard_champs` is the separate, sidecar-fed answer to "it is on level 3 right now". Both exist; do not conflate them. Routes live at `/api/v1/public/atlas`, deliberately **not** under `/shard`, because `/shard/*` means sidecar-dependent. ## Configuring the tree The website needs to be able to *read* the ServUO tree — same host, a bind mount, or a shared volume. Two ways to point at it, the setting winning over the environment: | Source | Notes | |---|---| | `spawn_atlas_servuo_path` setting | Admin-editable; changes take effect on the next refresh without a redeploy | | `SERVUO_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up | With neither set the atlas is simply skipped — the site runs normally without one. ## The boot path On every start the server hashes the source files and compares them against what is loaded. Unchanged (the normal case on a restart) costs one read pass, ~120 ms, and no database write. A real change costs a ~400 ms parse and a reload. Two contracts govern it: **1. It never blocks startup.** No configured path, an unreadable mount, a malformed file, a database error — every one is caught and logged, and the site comes up serving whatever atlas it already had. **2. A facet disappearing is never applied automatically.** Losing a facet looks exactly like a half-copied or mid-update tree, and boot cannot tell that apart from a real map change. That refresh is *staged* for a human instead. Everything else — new facets, renamed regions, changed spawns — applies immediately, since none of it can destroy something an operator would miss. ``` boot └─ path configured? no ──▶ skip └─ tree readable? no ──▶ warn, carry on └─ hashes changed? no ──▶ done (nothing parsed) └─ parse └─ a facet would be removed? no ──▶ import yes ──▶ stage for admin review; atlas unchanged ``` ### Approving or rejecting a staged refresh Only the *decision* is stored, never the parsed world — a few KB of source hashes plus the facet diff. Approving **re-parses** the tree, so what lands matches the tree at approval time rather than at boot, and a multi-megabyte blob never sits in the database. A rejection is remembered against those exact source hashes, so a declined refresh does not re-prompt on every restart. Change the tree and the hashes differ, which asks again. From the admin panel (second PR), or from the CLI: ```bash cd website/server npm run atlas:import -- --status # what is loaded, and what is pending npm run atlas:import -- --approve # apply the staged refresh npm run atlas:import -- --reject # keep the current atlas, dismiss it ``` ## The CLI The server refreshes itself on boot, so this is for applying a map change *without* a restart, and for the approve/reject flow above. ```bash npm run atlas:import # import if the tree differs npm run atlas:import -- --servuo # override the path for this run npm run atlas:import -- --force # reimport even if unchanged ``` `--servuo` is a per-run override and deliberately does **not** persist — changing where the atlas permanently reads from is an admin action, not a side effect of a one-off import. ## Sources | File | Count (stock ServUO 57.4) | Used for | |---|---|---| | `Spawns/*.xml` | 13 files, ~10.5 MB | Every spawner: location, size, delays, time-of-day, creature types | | `Data/Regions.xml` | 129 KB, nested | Named regions and their rectangles | | `Data/Locations/*.xml` | 6 files | Landmarks (dungeon levels, town markers) | | `Config/ChampionSpawns.xml` | 4.8 KB | Configured champion altars | **A stock tree has 13 spawn files but only 6 facets.** `Eodon.xml`, `GravewaterLake.xml`, `TreasuresOfKotl.xml` and the other named-area files hold TerMur/Trammel points. The facet always comes from each record's own ``, never from the file name. ## How a coordinate becomes a place name This is the transform the atlas exists for, in `resolveRegion()`: 1. The highest-`priority` named region whose rectangle contains the point. Ties break toward the **smallest** rect, so a specific room wins over the dungeon-wide rect enclosing it. 2. Otherwise the nearest landmark within the landmark radius (200 tiles by default), labelled by its **group** ("Covetous"), not its individual marker ("Level 1"). 3. Otherwise `"Wilderness"`. The radius cap in step 2 is what keeps step 3 reachable. Without it the nearest landmark is always *some* landmark however far away, and open countryside gets labelled with a dungeon on the far side of the map. Against stock ServUO this resolves **83.2%** of points (5,369 of 6,455): 3,681 by region, 1,688 by landmark, 1,086 Wilderness. ## Three quirks in the source data Each of these is silent if unhandled — the atlas still builds, it is just wrong. **Facet names disagree between sources.** `Data/Locations/*.xml` spells them `Ter Mur` and `Tokuno Islands`, while `` and `` say `TerMur` and `Tokuno`. Unreconciled, the landmark bucket is keyed differently from the points looking it up, so the fallback never fires and every unregioned spawn on those facets reads "Wilderness". This is reconciled **by matching, not by a lookup table** — there is no list of facet names anywhere. `facetKey()` collapses spelling differences (lowercase, alphanumerics only), and `resolveFacetName()` matches a loose spelling against the canonical set discovered from the shard's own spawn and region data, by exact key then by prefix in either direction. A name matching nothing keeps its own name: forcing a wrong match would file a real custom facet's landmarks under the wrong facet, which is worse than leaving it alone. **Spawn type tokens carry XmlSpawner directives.** The `` type is not always a bare class name: ``` Fairy,{RND,4,8} alchemist/z/-50 Agralem/Name/Agralem GargishRouser,1 greatape,true GargishRefugee/hue/34532 ``` Taken literally these invent creatures that do not exist *and* split real ones in two, because `Fairy` and `Fairy,{RND,4,8}` slug apart into separate entries. 71 of 845 were affected. Everything from the first `/` or `,` is stripped, leaving 800 real creatures. **Case is inconsistent across files.** The same creature is `Lizardman` in one file and `lizardman` in another. Slugging collapses them correctly, but the display name is chosen deterministically — most common spelling wins, ties break to the more capitalised form, then alphabetically — because otherwise it would depend on file read order and change on an unrelated restart. ## Tables All are **import-owned**: a refresh empties and reloads them in one transaction, so a failed reload leaves the previous atlas intact rather than a half-loaded world. Nothing else writes to them and nothing holds a foreign key to them — no FKs at all, consistent with every other `shard_*` table. Full column listings in [`BACKEND_DESIGN.md`](BACKEND_DESIGN.md). | Table | Rows (stock) | Notes | |---|---|---| | `shard_spawn_creatures` | 800 | `slug` PK; `total` = sum of each type's own max; nullable `art` | | `shard_spawn_points` | 6,455 | `spawn_range`, since `range` is reserved in MariaDB | | `shard_spawn_point_types` | 23,927 | The many-to-many; one spawner commonly carries six types | | `shard_regions` | 387 | Flattened out of the nesting; `rects` JSON | | `shard_landmarks` | 558 | `grp`, since `group` is reserved in SQL | | `shard_champion_spawns` | 25 | Configured altars, not the live feed | | `shard_atlas_meta` | 1 | Singleton; source hashes, for the change check | | `shard_atlas_pending` | 0–1 | Singleton; a staged refresh awaiting admin review | `shard_spawn_creatures.name` carries a plain `INDEX`, deliberately **not `FULLTEXT`**: ~800 rows makes a `LIKE` scan free, and FULLTEXT's minimum token length would break searches for names like "orc". The reload uses `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and would implicitly commit, defeating the all-or-nothing guarantee. Point ids are assigned explicitly rather than left to `AUTO_INCREMENT`, because the join rows need them and `conn.batch()` reports no usable `insertId`. ## Artwork — operator-supplied, never shipped **This project ships no creature art and no extraction tooling, and never will.** UO sprites live in the operator's own client `.mul`/`.uop` files. They are the operator's, not ours to redistribute. The atlas is fully functional as text. `shard_spawn_creatures.art` is nullable and is NULL on every fresh import; pages render without images, which is the normal and supported state, not a degraded one. An operator who wants art: 1. Extracts it from **their own** client files (UOFiddler, ClassicUO tooling, or any art extractor). 2. Drops the images under `server/uploads/atlas/`. 3. Copies `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json` and maps creature slugs to file names. 4. Restarts, or runs `npm run atlas:import -- --force`. Both `spawnAtlas.art.json` and `server/uploads/` are gitignored, so neither the map nor the images can be committed by accident. ## Code layout | File | Role | |---|---| | `src/utils/spawnAtlasParse.js` | **Pure and fs-free** parsers, so CI covers them with no ServUO tree. Zero dependencies. | | `src/utils/spawnAtlasSource.js` | The only thing that reads a ServUO tree; shared by the boot path and the CLI | | `src/model/shardAtlas/shardAtlas.db.js` | The one-transaction replace | | `src/model/shardAtlas/shardAtlas.model.js` | The refresh decision, staging, approve/reject | | `scripts/importSpawnAtlas.js` | Thin CLI over the model | Parsing notes: - `Regions.xml`, `Locations/*.xml` and `ChampionSpawns.xml` genuinely nest, and get a small hand-rolled **subset** tokenizer — elements, attributes, self-closing tags, comments, the XML declaration, CDATA, and the five predefined entities plus numeric refs. It is not a general-purpose XML parser and must not be reused as one. - The ~10.5 MB of `Spawns/*.xml` never touches that tokenizer. Those records are flat, so they get a streaming regex sweep instead; a DOM would allocate a node per element across ~40 fields on every record to keep 14 of them. **Do not put the Points files through a DOM parser.** - `` is `Type:MX=n:SB=…` segments joined by `:OBJ=`. Split on `:OBJ=` *first* — a naive `split(':')` shreds it. A single Trammel point carries six types.