docs/website/SPAWN_ATLAS.md

Spawn atlas

Status: Complete on edge — data pipeline in website #112, API + pages in website #113. Design: docs/link/v3.md §6 — Protocol 3.0 Part C.

The spawn atlas is a browsable catalogue of what the shard contains: which creatures spawn, where, how many, and which champion altars are configured. It answers "where do I find a lizardman?" with "Shrines, Yew, Isamu-Jima" rather than with a list of raw coordinates.

Two things that shape the whole design

The shard's ServUO tree is the single source of truth. Nothing is precomputed and committed to the repository. A shard's maps change over its lifetime — facets get added, replaced, or renamed — and a snapshot in the repo would silently drift from the world players actually see. The atlas is therefore re-derived from the tree on every server boot.

Facets are not a fixed list. Nothing in the codebase names Felucca, Trammel, or any other stock facet. The facet set is whatever the shard's own files declare, discovered at parse time. A shard running entirely custom maps gets exactly the same treatment as a stock one, with no code change.

What it is not

The atlas is static shard content, not live shard state.

• It does not come from the sidecar. Nothing here touches the bridge, and there is no event kind, no wire change and no PROTOCOL_VERSION bump for it. Part C is website-only.
• It stays fully populated while the shard is down.
• Its champion table (shard_champion_spawns) is the configured roster — "there is an Unholy Terror altar in Deceit". The live champ.update feed in shard_champs is the separate, sidecar-fed answer to "it is on level 3 right now". Both exist; do not conflate them.

Routes live at /api/v1/public/atlas, deliberately not under /shard, because /shard/* means sidecar-dependent.

Configuring the tree

The website needs to be able to read the ServUO tree — same host, a bind mount, or a shared volume. Two ways to point at it, the setting winning over the environment:

Source	Notes
spawn_atlas_servuo_path setting	Admin-editable; changes take effect on the next refresh without a redeploy
SERVUO_PATH env var	The deploy-time default, since the path usually describes a mount the deployment sets up

With neither set the atlas is simply skipped — the site runs normally without one.

The boot path

On every start the server hashes the source files and compares them against what is loaded. Unchanged (the normal case on a restart) costs one read pass, ~120 ms, and no database write. A real change costs a ~400 ms parse and a reload.

Two contracts govern it:

1. It never blocks startup. No configured path, an unreadable mount, a malformed file, a database error — every one is caught and logged, and the site comes up serving whatever atlas it already had.

2. A facet disappearing is never applied automatically. Losing a facet looks exactly like a half-copied or mid-update tree, and boot cannot tell that apart from a real map change. That refresh is staged for a human instead. Everything else — new facets, renamed regions, changed spawns — applies immediately, since none of it can destroy something an operator would miss.

boot
 └─ path configured?            no ──▶ skip
     └─ tree readable?          no ──▶ warn, carry on
         └─ hashes changed?     no ──▶ done (nothing parsed)
             └─ parse
                 └─ a facet would be removed?
                      no ──▶ import
                     yes ──▶ stage for admin review; atlas unchanged

Approving or rejecting a staged refresh

Only the decision is stored, never the parsed world — a few KB of source hashes plus the facet diff. Approving re-parses the tree, so what lands matches the tree at approval time rather than at boot, and a multi-megabyte blob never sits in the database.

A rejection is remembered against those exact source hashes, so a declined refresh does not re-prompt on every restart. Change the tree and the hashes differ, which asks again.

From Admin → Spawn Atlas, or from the CLI:

cd website/server
npm run atlas:import -- --status     # what is loaded, and what is pending
npm run atlas:import -- --approve    # apply the staged refresh
npm run atlas:import -- --reject     # keep the current atlas, dismiss it

The CLI

The server refreshes itself on boot, so this is for applying a map change without a restart, and for the approve/reject flow above.

npm run atlas:import                       # import if the tree differs
npm run atlas:import -- --servuo <path>    # override the path for this run
npm run atlas:import -- --force            # reimport even if unchanged

--servuo is a per-run override and deliberately does not persist — changing where the atlas permanently reads from is an admin action, not a side effect of a one-off import.

Sources

File	Count (stock ServUO 57.4)	Used for
Spawns/*.xml	13 files, ~10.5 MB	Every spawner: location, size, delays, time-of-day, creature types
Data/Regions.xml	129 KB, nested	Named regions and their rectangles
Data/Locations/*.xml	6 files	Landmarks (dungeon levels, town markers)
Config/ChampionSpawns.xml	4.8 KB	Configured champion altars

A stock tree has 13 spawn files but only 6 facets. Eodon.xml, GravewaterLake.xml, TreasuresOfKotl.xml and the other named-area files hold TerMur/Trammel points. The facet always comes from each record's own <Map>, never from the file name.

How a coordinate becomes a place name

This is the transform the atlas exists for, in resolveRegion():

1. The highest-priority named region whose rectangle contains the point. Ties break toward the smallest rect, so a specific room wins over the dungeon-wide rect enclosing it.
2. Otherwise the nearest landmark within the landmark radius (200 tiles by default), labelled by its group ("Covetous"), not its individual marker ("Level 1").
3. Otherwise "Wilderness".

The radius cap in step 2 is what keeps step 3 reachable. Without it the nearest landmark is always some landmark however far away, and open countryside gets labelled with a dungeon on the far side of the map.

Against stock ServUO this resolves 83.2% of points (5,369 of 6,455): 3,681 by region, 1,688 by landmark, 1,086 Wilderness.

Three quirks in the source data

Each of these is silent if unhandled — the atlas still builds, it is just wrong.

Facet names disagree between sources. Data/Locations/*.xml spells them Ter Mur and Tokuno Islands, while <Map> and <Facet name> say TerMur and Tokuno. Unreconciled, the landmark bucket is keyed differently from the points looking it up, so the fallback never fires and every unregioned spawn on those facets reads "Wilderness".

This is reconciled by matching, not by a lookup table — there is no list of facet names anywhere. facetKey() collapses spelling differences (lowercase, alphanumerics only), and resolveFacetName() matches a loose spelling against the canonical set discovered from the shard's own spawn and region data, by exact key then by prefix in either direction. A name matching nothing keeps its own name: forcing a wrong match would file a real custom facet's landmarks under the wrong facet, which is worse than leaving it alone.

Spawn type tokens carry XmlSpawner directives. The <Objects2> type is not always a bare class name:

Fairy,{RND,4,8}      alchemist/z/-50      Agralem/Name/Agralem
GargishRouser,1      greatape,true        GargishRefugee/hue/34532

Taken literally these invent creatures that do not exist and split real ones in two, because Fairy and Fairy,{RND,4,8} slug apart into separate entries. 71 of 845 were affected. Everything from the first / or , is stripped, leaving 800 real creatures.

Case is inconsistent across files. The same creature is Lizardman in one file and lizardman in another. Slugging collapses them correctly, but the display name is chosen deterministically — most common spelling wins, ties break to the more capitalised form, then alphabetically — because otherwise it would depend on file read order and change on an unrelated restart.

Tables

All are import-owned: a refresh empties and reloads them in one transaction, so a failed reload leaves the previous atlas intact rather than a half-loaded world. Nothing else writes to them and nothing holds a foreign key to them — no FKs at all, consistent with every other shard_* table. Full column listings in ../modules/uo/SCHEMA.md — these are module-uo's tables, not core's.

Table	Rows (stock)	Notes
shard_spawn_creatures	800	slug PK; total = sum of each type's own max; nullable art
shard_spawn_points	6,455	spawn_range, since range is reserved in MariaDB
shard_spawn_point_types	23,927	The many-to-many; one spawner commonly carries six types
shard_regions	387	Flattened out of the nesting; rects JSON
shard_landmarks	558	grp, since group is reserved in SQL
shard_champion_spawns	25	Configured altars, not the live feed
shard_atlas_meta	1	Singleton; source hashes, for the change check
shard_atlas_pending	0–1	Singleton; a staged refresh awaiting admin review

shard_spawn_creatures.name carries a plain INDEX, deliberately not FULLTEXT: ~800 rows makes a LIKE scan free, and FULLTEXT's minimum token length would break searches for names like "orc".

The reload uses DELETE, not TRUNCATE — TRUNCATE is DDL in MariaDB and would implicitly commit, defeating the all-or-nothing guarantee. Point ids are assigned explicitly rather than left to AUTO_INCREMENT, because the join rows need them and conn.batch() reports no usable insertId.

Artwork — the shard extracts it now (Protocol 8)

This project ships no creature art and no extraction tooling, and never will. UO sprites live in the operator's own client .mul/.uop files. They are the operator's, not ours to redistribute.

What changed in protocol 8 is not that rule — it is who does the extracting. The shard already has those files (a ServUO server cannot boot without a UO client), so as of v8.md phase 3 it decodes them itself and hands the pictures to the website over the bridge. Nobody installs UOFiddler and nobody copies images to a web host.

Admin → Shard → Import. The import walks the shard's asset manifest, fetches only the sprites whose hash changed, writes them under uploads/atlas/, asks the shard for a body id per creature (§8 — the shard constructs the creature and reads Body.BodyID, which is the only thing that is right for a shard's own custom creatures) and points each shard_spawn_creatures.art at its picture. Boot never calls the shard for this: the files change when an operator patches their client, which is an event they know about and the site does not.

On this machine's stock client that is 787 creature portraits, about a megabyte in total.

NULL stays a first-class state, and always will be. An install with no shard link has never imported one; a Linux shard host without libgdiplus cannot render a sprite at all (a named NO_IMAGING status, not an error); and even after a complete import, two thirds of the playable ghost and gargoyle bodies have no art in the client. Pages render without images, which is normal and supported, not degraded.

The operator's own artwork still wins

An operator who has drawn their own portraits keeps them. The map is unchanged:

1. Drop the images under server/uploads/atlas/.
2. Copy server/db/data/spawnAtlas.art.example.json to spawnAtlas.art.json and map creature slugs to file names.
3. Restart, or run the import.

spawnAtlas.art.json is applied over anything imported, per slug, so a sprite rip never replaces a hand-drawn portrait on the next Update. Both it and server/uploads/ are gitignored, so neither the map nor the images can be committed by accident.

Why the imported art is not stored on the creature row

shard_spawn_creatures is emptied and refilled by every atlas refresh, and a refresh happens on every boot. So the body ids and the imported files live in shard_creature_bodies and shard_assets, outside that blast radius, and the atlas import re-derives art from them on the way past. Storing it on the row would mean an ordinary re-parse of the ServUO tree silently deleting every portrait — with the next asset Update finding the client files unchanged, reporting "nothing to do", and never putting them back.

Code layout

File	Role
src/utils/spawnAtlasParse.js	Pure and fs-free parsers, so CI covers them with no ServUO tree. Zero dependencies.
src/utils/spawnAtlasSource.js	The only thing that reads a ServUO tree; shared by the boot path and the CLI
src/model/shardAtlas/shardAtlas.db.js	The one-transaction replace
src/model/shardAtlas/shardAtlas.model.js	The refresh decision, staging, approve/reject
scripts/importSpawnAtlas.js	Thin CLI over the model

Parsing notes:

• Regions.xml, Locations/*.xml and ChampionSpawns.xml genuinely nest, and get a small hand-rolled subset tokenizer — elements, attributes, self-closing tags, comments, the XML declaration, CDATA, and the five predefined entities plus numeric refs. It is not a general-purpose XML parser and must not be reused as one.
• The ~10.5 MB of Spawns/*.xml never touches that tokenizer. Those records are flat, so they get a streaming regex sweep instead; a DOM would allocate a node per element across ~40 fields on every record to keep 14 of them. Do not put the Points files through a DOM parser.
• <Objects2> is Type:MX=n:SB=… segments joined by :OBJ=. Split on :OBJ= first — a naive split(':') shreds it. A single Trammel point carries six types.
• Respawn delays are stored in two different units, per record. XmlSpawner writes MinDelay/MaxDelay in minutes, and switches to seconds only when a spawner's delay does not divide into whole minutes — flagging that with DelayInSec on the same record. A 5 therefore means five minutes on one spawner and five seconds on the next, and both are plausible respawn times, so a reader assuming either unit is silently wrong about the other. Stock ServUO 57.4 has ~170 second-flagged spawners out of 6,455. The parser normalises everything to seconds; the API and UI carry seconds throughout.

The parser version

spawnAtlasSource.js exports PARSER_VERSION, stored in shard_atlas_meta alongside the source hashes and bumped whenever the parser derives different data from identical files — a fixed misreading, a new field, a changed unit.

A refresh re-derives when the tree changed or the parser did. Hashing the tree alone would be a trap: an install whose maps never change would keep serving whatever an older build derived, indefinitely, and a deploy that corrects the parse would never reach the data. A version mismatch counts as drift, so the correction lands on the next boot without an operator having to know it happened.

The API

Everything is served from MariaDB. Nothing on this path touches the sidecar, so the pages stay complete while the shard is down — which is why the routes sit at /api/v1/public/atlas and not under /public/shard, where a prefix means "sidecar-dependent". Unlike /shard/*, they are siteMode-gated, like /posts and /wiki: a bestiary is site content and follows site content's rules.

Every route carries requireFeature('atlas') — 404 when an admin has disabled the feature (its pages must not reveal that it exists) and 403 when the caller sits below its configured audience. The default is anonymous, so the gates are inert until an admin changes something. Responses are field-projected like every other shard read; atlas declares no sensitive fields today, and the projection call is there so the first one that does is covered by construction rather than by a retrofit (v3.md §3.6.1).

Route	Answers
GET /atlas/creatures?q=&facet=&limit=&offset=	The bestiary, most numerous first, paginated with an unpaginated total
GET /atlas/creatures/:slug?facet=&points=	One creature: places, spawners, alsoHere
GET /atlas/regions?facet=&q=	Named regions and their rectangles
GET /atlas/landmarks?facet=&q=	Points of interest, labelled by group
GET /atlas/champions?facet=	The configured altar roster
GET /atlas/meta	Facets, counts and when the atlas was parsed

Two shapes worth knowing:

• places is the aggregate the atlas exists for. "Lizardman → Shrines, Isamu-Jima, Yew", grouped in SQL rather than by summing 6,455 point rows in Node. spawners is the raw list underneath it, bounded, with spawnersTruncated saying when it was cut.
• points is a COUNT, spawners is the LIST. The two are named apart deliberately: the same key meaning a number on the search route and an array on the detail route is the kind of thing a client only discovers in production.

GET /atlas/meta reports the game world only. The ServUO path, the per-file hashes and any pending refresh describe the operator's filesystem, and live on the admin route instead.

A facet is never validated against a list — nothing in the codebase names one. ?facet= is length-bounded and matched exactly, so an unknown name returns an empty result rather than an error. The filter is an EXISTS over the points and deliberately not a JSON path or JSON_SEARCH built from caller input: that function treats % and _ as wildcards, which would make ?facet=% match everything.

The admin panel

Admin → Spawn Atlas (/admin/shard-atlas, admin-only — it reads a path on the server's filesystem and replaces every atlas table, which is closer to a deploy action than to moderation).

Route	Does
GET /admin/shard/atlas	Status: path, readable, drift, counts, facets, pending
POST /admin/shard/atlas/import	Import now; { force: true } ignores the hash gate
POST /admin/shard/atlas/approve	Apply a staged refresh, facet loss and all
POST /admin/shard/atlas/reject	Keep the current atlas; remember the decision
PUT /admin/shard/atlas/path	Point the atlas at a different tree

Three behaviours that are deliberate:

• An unreadable tree is a 200, not a 500. refresh() reports outcomes rather than throwing, because the boot path must never be stopped by a bad tree, and that contract is preserved at the API. The panel says "The tree could not be read: …"; a 500 would say only that something broke.
• Setting the path does not import. Moving the mount and reloading the world are separate decisions, and an operator fixing a typo should not have a multi-thousand-row replace happen under them. The response carries fresh status so the panel can offer the import as the next step.
• Every action is written to the admin activity log (shard.atlas.import / .approve / .reject / .path).