RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(website): the atlas reads the shard's tree on every boot, not a snapshot

Browse Source 
Follows the redesign in website #112. Two decisions from the original §6 were
rejected in review and replaced; the docs now describe what was actually built.

**The committed artifact is gone.** A shard's maps change over its life, so a
snapshot in the repo silently drifts from the world players actually see. The
ServUO tree is the single source of truth and the atlas is re-derived on every
server boot, hash-gated so an unchanged tree costs one read pass and no write.

**Nothing may name a facet.** The first implementation carried a lookup table of
the six stock UO facets. A shard may add facets, replace them outright, or rename
them when its maps are updated, and a built-in list mishandles all three
silently. Reconciliation is now by matching against the facet set discovered from
the shard's own data, with an unmatched name keeping its own rather than being
forced into a wrong bucket.

## Changes

- **`website/SPAWN_ATLAS.md`** rewritten: the two ideas that shape the design,
  how to configure the tree path, the boot flow as a decision tree, the
  approve/reject flow, and the code layout. The artwork policy is unchanged and
  still explicit — no art ever ships, operators extract their own from their own
  client files.
- **`link/v3.md` §6.1 (new)** records the two rejected decisions plus the two
  boot-path contracts. The old "what real data changed" list becomes §6.2. §6's
  now-superseded passages — the artifact bullet, the payload budget, the operator
  re-run story — are marked rather than deleted, so the reasoning stays legible.
- **`website/BACKEND_DESIGN.md`** documents `shard_atlas_pending` and the two
  contracts that make it safe: a facet removal is staged for a human, and the
  boot refresh can never block startup.

The two contracts are the part worth reviewing. Losing a facet is
indistinguishable at boot from a half-copied or mid-update tree, so it is staged
rather than applied; and no failure mode of the atlas — missing path, unreadable
mount, malformed file, database error — is allowed to stop the site coming up.

---

- [x] AI-assisted: written with **Claude Code** (Claude Opus 5), reviewed before opening.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP
This commit is contained in:
claude
2026-07-28 16:44:39 -05:00
parent 3fb3f63f25
commit ff1c2064a5
3 changed files with 214 additions and 126 deletions
Download Patch File Download Diff File Expand all files Collapse all files

226
website/SPAWN_ATLAS.md
View File

@@ -8,6 +8,19 @@ creatures spawn, where, how many, and which champion altars are configured. It
answers "where do I find a lizardman?" with **"Shrines, Yew, Isamu-Jima"** rather
than with a list of raw coordinates.
## Two things that shape the whole design
**The shard's ServUO tree is the single source of truth.** Nothing is
precomputed and committed to the repository. A shard's maps change over its
lifetime — facets get added, replaced, or renamed — and a snapshot in the repo
would silently drift from the world players actually see. The atlas is therefore
re-derived from the tree **on every server boot**.
**Facets are not a fixed list.** Nothing in the codebase names Felucca, Trammel,
or any other stock facet. The facet set is whatever the shard's own files
declare, discovered at parse time. A shard running entirely custom maps gets
exactly the same treatment as a stock one, with no code change.
## What it is not
The atlas is **static shard content, not live shard state.**
@@ -24,44 +37,83 @@ The atlas is **static shard content, not live shard state.**
Routes live at `/api/v1/public/atlas`, deliberately **not** under `/shard`,
because `/shard/*` means sidecar-dependent.
## The two commands
## Configuring the tree
Building needs a ServUO tree. Importing does not. That split is the whole design:
the website container ships the committed artifact but has no ServUO tree, so it
can import but never build.
The website needs to be able to *read* the ServUO tree — same host, a bind mount,
or a shared volume. Two ways to point at it, the setting winning over the
environment:
```bash
# On a machine that has the ServUO tree (writes server/db/data/spawnAtlas.*.json)
cd website/server
npm run atlas:build -- --servuo /path/to/ServUO
| Source | Notes |
|---|---|
| `spawn_atlas_servuo_path` setting | Admin-editable; changes take effect on the next refresh without a redeploy |
| `SERVUO_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |
# Anywhere, including the deployed container
npm run atlas:import
With neither set the atlas is simply skipped — the site runs normally without
one.
## The boot path
On every start the server hashes the source files and compares them against what
is loaded. Unchanged (the normal case on a restart) costs one read pass, ~120 ms,
and no database write. A real change costs a ~400 ms parse and a reload.
Two contracts govern it:
**1. It never blocks startup.** No configured path, an unreadable mount, a
malformed file, a database error — every one is caught and logged, and the site
comes up serving whatever atlas it already had.
**2. A facet disappearing is never applied automatically.** Losing a facet looks
exactly like a half-copied or mid-update tree, and boot cannot tell that apart
from a real map change. That refresh is *staged* for a human instead. Everything
else — new facets, renamed regions, changed spawns — applies immediately, since
none of it can destroy something an operator would miss.
```
boot
└─ path configured? no ──▶ skip
└─ tree readable? no ──▶ warn, carry on
└─ hashes changed? no ──▶ done (nothing parsed)
└─ parse
└─ a facet would be removed?
no ──▶ import
yes ──▶ stage for admin review; atlas unchanged
```
`atlas:build` flags:
### Approving or rejecting a staged refresh
| Flag | Default | Meaning |
|---|---|---|
| `--servuo` | *(required)* | ServUO server root — the directory holding `Spawns/`, `Data/`, `Config/` |
| `--out` | `server/db/data` | Where to write the artifact |
| `--landmark-radius` | `200` | Max tile distance for the landmark fallback |
Only the *decision* is stored, never the parsed world — a few KB of source hashes
plus the facet diff. Approving **re-parses** the tree, so what lands matches the
tree at approval time rather than at boot, and a multi-megabyte blob never sits
in the database.
`atlas:import` takes `--dir` (default `server/db/data`).
A rejection is remembered against those exact source hashes, so a declined
refresh does not re-prompt on every restart. Change the tree and the hashes
differ, which asks again.
## Operator re-run story
From the admin panel (second PR), or from the CLI:
Spawns changed → rebuild → commit → deploy → import.
```bash
cd website/server
npm run atlas:import -- --status # what is loaded, and what is pending
npm run atlas:import -- --approve # apply the staged refresh
npm run atlas:import -- --reject # keep the current atlas, dismiss it
```
1. `npm run atlas:build -- --servuo <path>` on a machine with the tree.
2. Commit the regenerated `server/db/data/spawnAtlas.*.json`.
3. Deploy.
4. `npm run atlas:import`, or `POST /api/v1/admin/shard/atlas/import`.
## The CLI
`shard_atlas_meta` stores a sha256 per source file, so
`GET /api/v1/admin/shard/atlas/status` reports when the database is behind the
committed artifact. **Build stays CLI-only** — there is no admin button that
reads a ServUO tree.
The server refreshes itself on boot, so this is for applying a map change
*without* a restart, and for the approve/reject flow above.
```bash
npm run atlas:import # import if the tree differs
npm run atlas:import -- --servuo <path> # override the path for this run
npm run atlas:import -- --force # reimport even if unchanged
```
`--servuo` is a per-run override and deliberately does **not** persist — changing
where the atlas permanently reads from is an admin action, not a side effect of a
one-off import.
## Sources
@@ -72,11 +124,10 @@ reads a ServUO tree.
| `Data/Locations/*.xml` | 6 files | Landmarks (dungeon levels, town markers) |
| `Config/ChampionSpawns.xml` | 4.8 KB | Configured champion altars |
**There are 13 spawn files but only 6 facets.** `Eodon.xml`,
**A stock tree has 13 spawn files but only 6 facets.** `Eodon.xml`,
`GravewaterLake.xml`, `TreasuresOfKotl.xml` and the other named-area files hold
TerMur/Trammel points. The facet always comes from each record's own `<Map>`,
never from the file name, and the artifact shards by facet — Felucca, Trammel,
Ilshenar, Malas, Tokuno, TerMur.
never from the file name.
## How a coordinate becomes a place name
@@ -85,16 +136,17 @@ This is the transform the atlas exists for, in `resolveRegion()`:
1. The highest-`priority` named region whose rectangle contains the point. Ties
break toward the **smallest** rect, so a specific room wins over the
dungeon-wide rect enclosing it.
2. Otherwise the nearest landmark within `--landmark-radius` tiles, labelled by
its **group** ("Covetous"), not its individual marker ("Level 1").
2. Otherwise the nearest landmark within the landmark radius (200 tiles by
default), labelled by its **group** ("Covetous"), not its individual marker
("Level 1").
3. Otherwise `"Wilderness"`.
The radius cap in step 2 is what keeps step 3 reachable. Without it the nearest
landmark is always *some* landmark however far away, and open countryside gets
labelled with a dungeon on the far side of the map.
Against stock ServUO this resolves **83.2%** of points (5,369 of 6,455): 3,689 by
region, 1,690 by landmark, 1,086 Wilderness.
Against stock ServUO this resolves **83.2%** of points (5,369 of 6,455): 3,681 by
region, 1,688 by landmark, 1,086 Wilderness.
## Three quirks in the source data
@@ -102,11 +154,17 @@ Each of these is silent if unhandled — the atlas still builds, it is just wron
**Facet names disagree between sources.** `Data/Locations/*.xml` spells them
`Ter Mur` and `Tokuno Islands`, while `<Map>` and `<Facet name>` say `TerMur` and
`Tokuno`. Unreconciled, the landmark bucket for those two facets is keyed
differently from the points looking it up, so the fallback never fires and every
unregioned spawn in Ter Mur and Tokuno reads "Wilderness". All facet names are
canonicalised through `normalizeFacet()`; unknown facets pass through unchanged
so a custom shard facet still gets an atlas.
`Tokuno`. Unreconciled, the landmark bucket is keyed differently from the points
looking it up, so the fallback never fires and every unregioned spawn on those
facets reads "Wilderness".
This is reconciled **by matching, not by a lookup table** — there is no list of
facet names anywhere. `facetKey()` collapses spelling differences (lowercase,
alphanumerics only), and `resolveFacetName()` matches a loose spelling against
the canonical set discovered from the shard's own spawn and region data, by exact
key then by prefix in either direction. A name matching nothing keeps its own
name: forcing a wrong match would file a real custom facet's landmarks under the
wrong facet, which is worse than leaving it alone.
**Spawn type tokens carry XmlSpawner directives.** The `<Objects2>` type is not
always a bare class name:
@@ -125,38 +183,35 @@ real creatures.
file and `lizardman` in another. Slugging collapses them correctly, but the
display name is chosen deterministically — most common spelling wins, ties break
to the more capitalised form, then alphabetically — because otherwise it would
depend on file read order and produce a spurious artifact diff on every unrelated
rebuild.
depend on file read order and change on an unrelated restart.
## Artifact format
## Tables
`server/db/data/`, all committed:
All are **import-owned**: a refresh empties and reloads them in one transaction,
so a failed reload leaves the previous atlas intact rather than a half-loaded
world. Nothing else writes to them and nothing holds a foreign key to them — no
FKs at all, consistent with every other `shard_*` table. Full column listings in
[`BACKEND_DESIGN.md`](BACKEND_DESIGN.md).
| File | Contents |
|---|---|
| `spawnAtlas.meta.json` | Indented. Build time, counts, sha256 per source file |
| `spawnAtlas.index.json` | Compact. Facets, creatures, regions, landmarks, champions |
| `spawnAtlas.<facet>.json` × 6 | Compact. That facet's spawn points |
| Table | Rows (stock) | Notes |
|---|---|---|
| `shard_spawn_creatures` | 800 | `slug` PK; `total` = sum of each type's own max; nullable `art` |
| `shard_spawn_points` | 6,455 | `spawn_range`, since `range` is reserved in MariaDB |
| `shard_spawn_point_types` | 23,927 | The many-to-many; one spawner commonly carries six types |
| `shard_regions` | 387 | Flattened out of the nesting; `rects` JSON |
| `shard_landmarks` | 558 | `grp`, since `group` is reserved in SQL |
| `shard_champion_spawns` | 25 | Configured altars, not the live feed |
| `shard_atlas_meta` | 1 | Singleton; source hashes, for the change check |
| `shard_atlas_pending` | 0–1 | Singleton; a staged refresh awaiting admin review |
**1.41 MB total**, down from 4.40 MB. The design budgeted "well under 1 MB",
which turned out optimistic for 6,455 points; three encodings closed most of the
gap:
`shard_spawn_creatures.name` carries a plain `INDEX`, deliberately **not
`FULLTEXT`**: ~800 rows makes a `LIKE` scan free, and FULLTEXT's minimum token
length would break searches for names like "orc".
- `facet` is dropped per record — the shard file names it once at the top.
- Fields at their default are omitted rather than written as `0`. Most spawners
are a single point with no time-of-day gating, so `width`, `height`, `range`
and the three `tod*` fields are zero on the large majority of records.
- `types` are `[name, max]` tuples. There are ~24,000 type entries and
`{"type":"Orc","max":1}` spends 15 bytes apiece restating two key names that
never vary.
`label` is not stored at all — it is exactly `region || landmark || "Wilderness"`
and the importer recomputes it.
`buildSpawnAtlas.encodePoint()` and `importSpawnAtlas.readPoint()` are exact
inverses, round-tripped in `test/spawnAtlas.build.test.js`. **Change one, change
both.** The artifact never reaches the browser; the browser sees only paginated
API responses.
The reload uses `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and would
implicitly commit, defeating the all-or-nothing guarantee. Point ids are assigned
explicitly rather than left to `AUTO_INCREMENT`, because the join rows need them
and `conn.batch()` reports no usable `insertId`.
## Artwork — operator-supplied, never shipped
@@ -175,41 +230,22 @@ An operator who wants art:
2. Drops the images under `server/uploads/atlas/`.
3. Copies `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json`
and maps creature slugs to file names.
4. Re-runs `npm run atlas:import`.
4. Restarts, or runs `npm run atlas:import -- --force`.
Both `spawnAtlas.art.json` and `server/uploads/` are gitignored, so neither the
map nor the images can be committed by accident.
## Tables
## Code layout
All six are **import-owned**: `atlas:import` empties and reloads them in one
transaction. Nothing else writes to them and nothing holds a foreign key to them
— no FKs at all, consistent with every other `shard_*` table. Full column
listings in [`BACKEND_DESIGN.md`](BACKEND_DESIGN.md).
| File | Role |
|---|---|
| `src/utils/spawnAtlasParse.js` | **Pure and fs-free** parsers, so CI covers them with no ServUO tree. Zero dependencies. |
| `src/utils/spawnAtlasSource.js` | The only thing that reads a ServUO tree; shared by the boot path and the CLI |
| `src/model/shardAtlas/shardAtlas.db.js` | The one-transaction replace |
| `src/model/shardAtlas/shardAtlas.model.js` | The refresh decision, staging, approve/reject |
| `scripts/importSpawnAtlas.js` | Thin CLI over the model |
| Table | Rows (stock) | Notes |
|---|---|---|
| `shard_spawn_creatures` | 800 | `slug` PK; `total` = sum of each type's own max; nullable `art` |
| `shard_spawn_points` | 6,455 | `spawn_range`, since `range` is reserved in MariaDB |
| `shard_spawn_point_types` | 23,927 | The many-to-many; one spawner commonly carries six types |
| `shard_regions` | 387 | Flattened out of the nesting; `rects` JSON |
| `shard_landmarks` | 558 | `grp`, since `group` is reserved in SQL |
| `shard_champion_spawns` | 25 | Configured altars, not the live feed |
| `shard_atlas_meta` | 1 | Singleton (`id = 1`); source hashes for the drift check |
`shard_spawn_creatures.name` carries a plain `INDEX`, deliberately **not
`FULLTEXT`**: ~800 rows makes a `LIKE` scan free, and FULLTEXT's minimum token
length would break searches for names like "orc".
The importer uses `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and
would implicitly commit, defeating the all-or-nothing reload. Point ids are
assigned explicitly rather than left to `AUTO_INCREMENT`, because the join rows
need to know them and `conn.batch()` reports no usable `insertId`.
## Parsing notes
`server/src/utils/spawnAtlasParse.js` is **pure and fs-free**, so CI covers it
with no ServUO tree. It adds **zero dependencies**.
Parsing notes:
- `Regions.xml`, `Locations/*.xml` and `ChampionSpawns.xml` genuinely nest, and
get a small hand-rolled **subset** tokenizer — elements, attributes,