feat(atlas): parse a ServUO tree into a committed spawn atlas artifact
Protocol 3.0 order 3 (Part C), first of two website PRs. This half is the data
pipeline only — parsers, the build/import CLI, and the tables. No routes and no
client, so nothing is user-visible yet; the API and pages follow in PR 2.
Part C is website-only: no plugin, no sidecar, no new event kinds, no wire
change.
## Parsing
`src/utils/spawnAtlasParse.js` is pure and fs-free so CI covers it with no
ServUO tree. Zero new dependencies — `Regions.xml` genuinely nests, so it gets a
small hand-rolled subset tokenizer rather than a new XML package. The 10.5 MB of
`Spawns/*.xml` never touches it: those records are flat and get a streaming
regex sweep instead.
The high-value transform is point-in-rect placement — highest region priority
wins, ties break to the smaller rect, then a nearest-landmark fallback within
200 tiles, else "Wilderness". That is what turns "lizardman at 5411,1234" into
"Despise, Felucca", and it resolves 83.2% of points (5,369 of 6,455).
Three things the real data forced, none of which were in the design:
- **Only 6 facets, not 13.** `Eodon.xml`, `GravewaterLake.xml` and the other
named-area files carry TerMur/Trammel points, so the facet comes from each
record's own `<Map>` and the artifact shards 6 ways.
- **Facet names disagree across sources.** `Data/Locations/*.xml` spells them
`Ter Mur` and `Tokuno Islands`; `<Map>` and `<Facet name>` say `TerMur` and
`Tokuno`. Unreconciled this is silent — the landmark fallback simply never
fires on those facets and every unregioned spawn there reads "Wilderness".
- **Spawn type tokens carry XmlSpawner directives**: `Fairy,{RND,4,8}`,
`alchemist/z/-50`, `Agralem/Name/Agralem`. Taken literally these invent
creatures that do not exist AND split real ones in two, since `Fairy` and
`Fairy,{RND,4,8}` slug apart. 71 of 845 entries were affected; stripping at
the first `/` or `,` leaves 800 clean ones.
## Artifact
`npm run atlas:build -- --servuo <path>` writes `db/data/spawnAtlas.*.json`:
6 facet shards + a compact index + a small indented `meta`. 1.41 MB committed,
down from 4.40 MB by dropping `facet` per record, omitting defaulted fields, and
tuple-encoding the ~24,000 type entries. `encodePoint()` and the importer's
`readPoint()` are exact inverses and are round-tripped in tests.
Display spelling is chosen deterministically (most common, ties to the
capitalised form) because the spawn files are inconsistent about case and the
name would otherwise depend on file read order — a spurious diff on every
unrelated rebuild.
## Import
`npm run atlas:import` needs no ServUO tree, which is the whole reason build and
import are separate: the container has the artifact but not the tree. It
reloads all six tables in one transaction (DELETE, not TRUNCATE, which is DDL
and would implicitly commit), so a failed import leaves the previous atlas
intact.
## No artwork, by design
The repo ships no creature art and no extraction tooling. Sprites live in the
operator's own client `.mul`/`.uop` files and are theirs, not ours to
redistribute. `shard_spawn_creatures.art` is nullable and NULL on every fresh
import; an operator who wants art extracts it themselves, drops it under
`server/uploads/atlas/` (already gitignored) and maps slugs in a gitignored
`spawnAtlas.art.json`. Text-only is the normal, fully supported state.
## Verification
- **544 server tests pass**, 57 new across `spawnAtlas.parse.test.js` (the
`:OBJ=` split, directive stripping, nested-region priority inheritance,
half-open rects, the facet reconciliation, tokenizer edge cases) and
`spawnAtlas.build.test.js` (aggregation, deterministic naming, and the
encode/decode round trip).
- Built and imported for real against the local MariaDB and the ServUO tree at
`C:\Users\colby\Desktop\ServUO`: 6,455 points, 800 creatures, 23,927
point/type rows, 387 regions, 558 landmarks, 25 champion altars.
- "Where does a lizardman spawn?" answers Shrines / Isamu-Jima / Yew across
Felucca, Trammel and Tokuno.
No routes changed, so the OpenAPI spec and route manifest are untouched.
---
- [x] AI-assisted: written with **Claude Code** (Claude Opus 5), reviewed before opening.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP
This commit is contained in:
@@ -1022,6 +1022,132 @@ CREATE TABLE IF NOT EXISTS announce_jobs (
|
||||
INDEX idx_announce_due_discord (discord_status, discord_next_attempt_at)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
|
||||
-- ── Spawn atlas (Protocol 3.0 Part C) ───────────────────────────────────────
|
||||
-- Static shard CONTENT, not live shard state: what spawns where, which regions
|
||||
-- and landmarks exist, and which champion altars are configured. Nothing here
|
||||
-- comes from the sidecar — it is imported from a committed artifact built off a
|
||||
-- ServUO tree by `npm run atlas:build` (see docs/website/SPAWN_ATLAS.md), so
|
||||
-- these tables stay populated whether the shard is up or not.
|
||||
--
|
||||
-- Every table is import-owned: `npm run atlas:import` TRUNCATEs and reloads them
|
||||
-- in one transaction. Nothing else may write here, and nothing else may hold a
|
||||
-- foreign key to them. No FKs at all, consistent with every other shard_* table.
|
||||
|
||||
-- One row per spawnable type, aggregated across the world. `total` is the sum of
|
||||
-- each type's own MX across every point that spawns it (how many exist at once);
|
||||
-- `facets` is a per-facet point count, so the facet filter and "where does this
|
||||
-- live" both answer without touching shard_spawn_points.
|
||||
CREATE TABLE IF NOT EXISTS shard_spawn_creatures (
|
||||
slug VARCHAR(120) NOT NULL PRIMARY KEY, -- slugified class name; the /atlas/:slug key
|
||||
name VARCHAR(120) NOT NULL, -- display spelling chosen by the build
|
||||
total INT NOT NULL DEFAULT 0,
|
||||
points INT NOT NULL DEFAULT 0,
|
||||
facets JSON NULL, -- { "Felucca": 171, "Trammel": 160, ... }
|
||||
-- Operator-supplied artwork, always NULL on a fresh import. The repo ships no
|
||||
-- creature art: sprites live in the operator's own client .mul/.uop files and
|
||||
-- are theirs to extract and place under uploads/atlas/. The UI renders without
|
||||
-- art when this is NULL, which is the normal case.
|
||||
art VARCHAR(255) NULL,
|
||||
-- Plain INDEX, deliberately NOT FULLTEXT: ~800 rows makes a LIKE scan free,
|
||||
-- and FULLTEXT's min-token-length would break searches for names like "orc".
|
||||
INDEX idx_shard_spawn_creatures_name (name)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
|
||||
-- One row per spawner. `region`/`landmark` are the resolved place name — the
|
||||
-- point-in-rect transform that turns "5411,1234" into "Despise" — and `label` is
|
||||
-- the resolved display string (region, else landmark, else 'Wilderness').
|
||||
CREATE TABLE IF NOT EXISTS shard_spawn_points (
|
||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
facet VARCHAR(40) NOT NULL,
|
||||
name VARCHAR(120) NULL, -- the ServUO spawner's own name
|
||||
x INT NOT NULL,
|
||||
y INT NOT NULL,
|
||||
width INT NOT NULL DEFAULT 0,
|
||||
height INT NOT NULL DEFAULT 0,
|
||||
spawn_range INT NOT NULL DEFAULT 0, -- `range` is reserved in MariaDB
|
||||
max_count INT NOT NULL DEFAULT 0,
|
||||
min_delay INT NOT NULL DEFAULT 0,
|
||||
max_delay INT NOT NULL DEFAULT 0,
|
||||
tod_start INT NOT NULL DEFAULT 0, -- meaningless unless tod_mode <> 0
|
||||
tod_end INT NOT NULL DEFAULT 0,
|
||||
tod_mode INT NOT NULL DEFAULT 0,
|
||||
region VARCHAR(120) NULL,
|
||||
landmark VARCHAR(120) NULL,
|
||||
label VARCHAR(120) NOT NULL DEFAULT 'Wilderness',
|
||||
INDEX idx_shard_spawn_points_facet (facet),
|
||||
INDEX idx_shard_spawn_points_label (label)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
|
||||
-- The many-to-many between the two above: one spawner commonly carries several
|
||||
-- types (a single Trammel point spawns six), each with its own max. This is how
|
||||
-- /atlas/creatures/:slug finds the places a creature appears.
|
||||
CREATE TABLE IF NOT EXISTS shard_spawn_point_types (
|
||||
point_id INT NOT NULL,
|
||||
slug VARCHAR(120) NOT NULL, -- → shard_spawn_creatures.slug (no FK)
|
||||
max_count INT NOT NULL DEFAULT 1,
|
||||
PRIMARY KEY (point_id, slug),
|
||||
INDEX idx_shard_spawn_point_types_slug (slug)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
|
||||
-- Named regions from Data/Regions.xml, flattened out of their nesting. `rects`
|
||||
-- holds the region's rectangles; `priority` and rect area are what resolved each
|
||||
-- spawn point at build time, kept here so the admin drift check can re-derive.
|
||||
CREATE TABLE IF NOT EXISTS shard_regions (
|
||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
facet VARCHAR(40) NOT NULL,
|
||||
name VARCHAR(120) NOT NULL,
|
||||
type VARCHAR(80) NULL, -- ServUO region class
|
||||
priority INT NOT NULL DEFAULT 0,
|
||||
parent VARCHAR(120) NULL, -- enclosing named region, if any
|
||||
rects JSON NULL,
|
||||
INDEX idx_shard_regions_facet (facet),
|
||||
INDEX idx_shard_regions_name (name)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
|
||||
-- Points of interest from Data/Locations/*.xml. `grp` is the innermost enclosing
|
||||
-- parent ("Covetous"), which is the label worth showing — "Covetous" reads
|
||||
-- better than the individual marker "Level 1". (`group` is reserved in SQL.)
|
||||
CREATE TABLE IF NOT EXISTS shard_landmarks (
|
||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
facet VARCHAR(40) NOT NULL,
|
||||
name VARCHAR(120) NOT NULL,
|
||||
grp VARCHAR(120) NULL,
|
||||
x INT NOT NULL,
|
||||
y INT NOT NULL,
|
||||
z INT NOT NULL DEFAULT 0,
|
||||
INDEX idx_shard_landmarks_facet (facet),
|
||||
INDEX idx_shard_landmarks_name (name)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
|
||||
-- Configured champion altars from Config/ChampionSpawns.xml. This is static
|
||||
-- roster data ("there is an Unholy Terror altar in Deceit") and is distinct from
|
||||
-- the live champ.update feed in shard_champs ("it is on level 3 right now").
|
||||
CREATE TABLE IF NOT EXISTS shard_champion_spawns (
|
||||
slug VARCHAR(160) NOT NULL PRIMARY KEY, -- facet-name, e.g. "felucca-deceit"
|
||||
name VARCHAR(120) NOT NULL,
|
||||
grp VARCHAR(80) NULL, -- spawn group; one active per group
|
||||
type VARCHAR(80) NULL, -- '' when randomised per activation
|
||||
random_type TINYINT(1) NOT NULL DEFAULT 0,
|
||||
facet VARCHAR(40) NOT NULL,
|
||||
x INT NOT NULL,
|
||||
y INT NOT NULL,
|
||||
z INT NOT NULL DEFAULT 0,
|
||||
radius INT NOT NULL DEFAULT 0,
|
||||
label VARCHAR(120) NULL, -- resolved place name
|
||||
INDEX idx_shard_champion_spawns_facet (facet)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
|
||||
-- Singleton (id = 1) describing the artifact currently loaded: when it was
|
||||
-- built, its counts, and a sha256 per ServUO source file. The admin drift check
|
||||
-- compares this against db/data/spawnAtlas.meta.json to report when the database
|
||||
-- is behind the committed artifact.
|
||||
CREATE TABLE IF NOT EXISTS shard_atlas_meta (
|
||||
id TINYINT NOT NULL PRIMARY KEY DEFAULT 1,
|
||||
payload JSON NOT NULL,
|
||||
imported_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
CONSTRAINT chk_shard_atlas_meta_singleton CHECK (id = 1)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
|
||||
-- Migrations for databases created before the wiki upgrade. Each statement uses
|
||||
-- IF NOT EXISTS so re-running on every boot is a harmless no-op. New installs get
|
||||
-- these columns from the CREATE TABLE above; existing installs get them here.
|
||||
|
||||
Reference in New Issue
Block a user