Compare commits
4 Commits
e9ecdc0ecb
...
docs/spawn
| Author | SHA1 | Date | |
|---|---|---|---|
| 1b7da860b5 | |||
| ff1c2064a5 | |||
| 10ae129b94 | |||
| 3fb3f63f25 |
124
link/v3.md
124
link/v3.md
@@ -13,7 +13,7 @@ Each part is marked off here as it lands on `edge`. §9 carries the same state p
|
||||
|---|---|---|---|
|
||||
| 1 | **A** — visibility framework + actor-leak fix (§3) | ✅ **Done** | website [#109](https://gitea.whitlocktech.com/RunicGateway/website/pulls/109) + [#110](https://gitea.whitlocktech.com/RunicGateway/website/pulls/110), docs [#64](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/64) + [#65](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/65) |
|
||||
| 2 | **B/1** — `world.ruleset` (§5) | ✅ **Done** | servuo-plugins [#3](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/3), link [#17](https://gitea.whitlocktech.com/RunicGateway/link/pulls/17), website [#111](https://gitea.whitlocktech.com/RunicGateway/website/pulls/111), docs [#66](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/66) |
|
||||
| 3 | **C** — spawn atlas (§6) | ⬜ **Next** | — |
|
||||
| 3 | **C** — spawn atlas (§6) | 🟡 **Data pipeline done** | website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112) (parsers + CLI + tables); API/client PR next |
|
||||
| 4 | **B/2** — `points.board` (§7) | ⬜ Not started | — |
|
||||
| 5 | **B/3** — `vendor.listing` (§8) | ⬜ Not started | — |
|
||||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3 (§4) | ⬜ Not started | — |
|
||||
@@ -329,19 +329,31 @@ frame during verification.
|
||||
|
||||
**No plugin, no sidecar, no `Bridge.cfg` knob, no new kinds.** Not part of the v3 wire change.
|
||||
|
||||
**Decision: committed generated artifact + idempotent DB import**, split in two because the build
|
||||
needs the ServUO tree (which the website container does not have) and the import does not. Not
|
||||
runtime import (10.5 MB of XML per boot), not a browser-served blob.
|
||||
> **Status:** data pipeline landed on `edge` — website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112)
|
||||
> (parsers, build/import CLI, tables, artifact). API + client pages are the second website PR.
|
||||
> Part C ships as **two** website PRs, not one: the parsing half is where the correctness risk
|
||||
> lives, and burying it under routes and React would have meant reviewing it in a 10k-line diff.
|
||||
> Full operator documentation: [`docs/website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md).
|
||||
>
|
||||
> **§6 below is the original design and is partly superseded.** §6.1 records two decisions that were
|
||||
> rejected in review and replaced (the committed artifact, and the fixed facet list); §6.2 records
|
||||
> the corrections the real ServUO data forced. Read both before trusting §6.
|
||||
|
||||
**Decision (revised at implementation time): the shard's ServUO tree is the single source of truth,
|
||||
re-derived on every server boot.** The original plan here was a committed generated artifact plus an
|
||||
idempotent import. That was rejected in review for two reasons, recorded in §6.1: a snapshot in the
|
||||
repo goes stale as a shard's maps change, and the design leaned on a fixed facet list that no shard
|
||||
is obliged to keep. Still not a browser-served blob; still parsed server-side only.
|
||||
|
||||
New in `website/server/`:
|
||||
|
||||
- `src/utils/spawnAtlasParse.js` — **pure functions, no fs**, so they are unit-testable in CI without
|
||||
a ServUO tree: `parseObjects2()`, `parsePoints()`, `parseRegions()`, `parseLocations()`,
|
||||
`resolveRegion()`.
|
||||
- `scripts/buildSpawnAtlas.js` (`--servuo <path> --out db/data/`) and `scripts/importSpawnAtlas.js`
|
||||
(TRUNCATE + batched INSERT in one transaction); `package.json` scripts `atlas:build`, `atlas:import`.
|
||||
- `db/data/spawnAtlas.<facet>.json` ×13 + `spawnAtlas.index.json` (creatures, champions, regions,
|
||||
landmarks, meta with per-source-file hashes).
|
||||
- ~~`scripts/buildSpawnAtlas.js` and a committed `db/data/spawnAtlas.*.json` artifact~~ — dropped,
|
||||
see §6.1 R1. Replaced by `src/utils/spawnAtlasSource.js` (the only thing that reads a ServUO tree,
|
||||
shared by the boot path and the CLI) and a `scripts/importSpawnAtlas.js` that is a thin CLI over
|
||||
the model. `package.json` gains `atlas:import` only.
|
||||
- `src/model/shardAtlas/{shardAtlas.db.js,shardAtlas.model.js}` following the `shardState` split.
|
||||
- `src/router/v1/public/atlas.{router,controller}.js`; `test/spawnAtlas.parse.test.js`.
|
||||
|
||||
@@ -372,16 +384,92 @@ stays CLI-only.**
|
||||
|
||||
Client: `routes/public/Atlas.jsx` (`/site/atlas`) and `AtlasCreature.jsx` (`/site/atlas/:slug`).
|
||||
|
||||
**Payload risk** — a monolithic artifact would be 2–3 MB of committed JSON. Shard per facet and drop
|
||||
every `<Points>` field the site cannot use (`UniqueId`, all trigger/refractory/proximity/sequential
|
||||
fields, sound ids), keeping Name/Map/X/Y/W/H/Range/MaxCount/MinDelay/MaxDelay/TOD*/types — well under
|
||||
1 MB. The artifact never reaches the browser; the browser sees only paginated API responses.
|
||||
**Payload risk** — *superseded by §6.1 R1; nothing is committed.* The field selection it describes
|
||||
still applies at parse time: every `<Points>` field the site cannot use (`UniqueId`, all
|
||||
trigger/refractory/proximity/sequential fields, sound ids) is dropped, keeping
|
||||
Name/Map/X/Y/W/H/Range/MaxCount/MinDelay/MaxDelay/TOD*/types. Parsed data never reaches the browser;
|
||||
the browser sees only paginated API responses.
|
||||
|
||||
**Operator re-run story** — spawns changed → `npm run atlas:build -- --servuo <path>` on a machine
|
||||
with the tree → commit the regenerated `db/data/spawnAtlas.*.json` → deploy → `npm run atlas:import`
|
||||
(or `POST /admin/shard/atlas/import`). `shard_atlas_meta.source` holds per-file hashes, so
|
||||
`GET /admin/shard/atlas/status` reports when the DB is behind the artifact. Full detail in
|
||||
`docs/website/SPAWN_ATLAS.md`.
|
||||
**Operator re-run story** — *revised by §6.1 R1.* Spawns changed → restart, or
|
||||
`npm run atlas:import` / `POST /admin/shard/atlas/import` to apply without one. `shard_atlas_meta`
|
||||
holds a sha256 per source file, so the server can tell on boot whether anything changed, and
|
||||
`GET /admin/shard/atlas/status` reports drift. If the change would remove a facet it is staged for
|
||||
approval rather than applied (§6.1 R3). Full detail in `docs/website/SPAWN_ATLAS.md`.
|
||||
|
||||
### 6.1 What implementation changed
|
||||
|
||||
Two design decisions in §6 were rejected in review and replaced; the rest are corrections the real
|
||||
ServUO data forced. Kept as a diff rather than edited in place, because each is a trap the next
|
||||
person would otherwise re-enter.
|
||||
|
||||
**R1. The committed artifact is gone — the tree is re-parsed on every boot.** §6 proposed building a
|
||||
generated artifact, committing it, and importing it. Two problems. A shard's maps change over its
|
||||
life, so a snapshot in the repo silently drifts from the world players actually see; and the build/
|
||||
import split existed only to work around the website container not having a tree, which is a
|
||||
deployment question (mount it) rather than a reason to freeze data. The server now hashes the source
|
||||
files on boot and re-derives the atlas when they differ. `scripts/buildSpawnAtlas.js`, the 1.41 MB
|
||||
artifact, and the whole encode/decode seam it needed are deleted.
|
||||
|
||||
**R2. Nothing may name a facet.** The first implementation carried a lookup table of the six stock
|
||||
UO facets to reconcile the spelling drift between sources. A shard may add facets, replace them
|
||||
outright, or rename them when its maps are updated, and a built-in list mishandles all three
|
||||
silently. Reconciliation is now by *matching* against the facet set discovered from the shard's own
|
||||
spawn and region data — exact key, then prefix in either direction — with an unmatched name keeping
|
||||
its own rather than being forced into a wrong bucket.
|
||||
|
||||
**R3. Two contracts on the boot path.** It never blocks startup: no path, an unreadable mount, a
|
||||
malformed file or a database error is caught and logged, and the site comes up serving whatever
|
||||
atlas it had. And a refresh that would REMOVE a facet is never applied automatically — facet loss
|
||||
is indistinguishable at boot from a half-copied or mid-update tree, so it is staged in
|
||||
`shard_atlas_pending` for an admin to approve or reject. Only the decision is stored (source hashes
|
||||
+ the facet diff, a few KB); approving re-parses, so what lands matches the tree at approval time.
|
||||
A rejection is remembered against those hashes so it does not re-prompt every restart.
|
||||
|
||||
### 6.2 What the build against real data changed
|
||||
|
||||
Six corrections to the design above, from running it against stock ServUO 57.4. Kept as a diff
|
||||
rather than edited in place, because each one is a trap the next person would otherwise re-enter.
|
||||
|
||||
**1. Six facets, not thirteen.** The design said `spawnAtlas.<facet>.json ×13`, assuming one facet
|
||||
per spawn file. There are 13 files but only **6** facets — `Eodon.xml`, `GravewaterLake.xml`,
|
||||
`TreasuresOfKotl.xml` and the other named-area files carry TerMur/Trammel points. The facet comes
|
||||
from each record's own `<Map>`, never the file name, and the artifact shards 6 ways.
|
||||
|
||||
**2. The XML dependency call: hand-rolled, zero deps.** §6 left `fast-xml-parser` vs a ~120-line
|
||||
tokenizer open. Resolved as the tokenizer — a deliberate *subset* parser covering only what these
|
||||
files use. The server keeps zero XML dependencies at any tier.
|
||||
|
||||
**3. Facet names disagree between sources — a silent failure.** `Data/Locations/*.xml` spells them
|
||||
`Ter Mur` and `Tokuno Islands`; `<Map>` and `<Facet name>` say `TerMur` and `Tokuno`. Unreconciled,
|
||||
the landmark bucket is keyed differently from the points looking it up, so the fallback never fires
|
||||
and **every unregioned spawn in Ter Mur and Tokuno reads "Wilderness"** — a plausible-looking atlas
|
||||
that is quietly wrong for two facets. All facet names now pass through `normalizeFacet()`.
|
||||
|
||||
**4. Spawn type tokens carry XmlSpawner directives.** `<Objects2>` types are not always bare class
|
||||
names: `Fairy,{RND,4,8}`, `alchemist/z/-50`, `Agralem/Name/Agralem`, `greatape,true`. Taken literally
|
||||
they invent creatures that do not exist *and* split real ones in two, since `Fairy` and
|
||||
`Fairy,{RND,4,8}` slug apart. 71 of 845 entries were affected; stripping at the first `/` or `,`
|
||||
leaves **800** real creatures. (The design's "~1,500 creature rows" estimate was high; 800 only
|
||||
reinforces the plain-`INDEX`-not-`FULLTEXT` call.)
|
||||
|
||||
**5. The artifact would have been 1.41 MB, not "well under 1 MB" — and is now moot.** Dropping the
|
||||
unused `<Points>` fields as the design directed still left 4.40 MB; three further encodings brought
|
||||
it to 1.41 MB, and getting under 1 MB would have meant dropping the spawner `name`. The size budget
|
||||
in §6 was simply optimistic for 6,455 points. Superseded by §6.1 R1: there is no artifact, so there
|
||||
is no payload to budget and no encode/decode seam to keep in sync.
|
||||
|
||||
**6. `DELETE`, not `TRUNCATE`.** The design said "TRUNCATE + batched INSERT in one transaction",
|
||||
which does not hold: `TRUNCATE` is DDL in MariaDB and implicitly commits, so a mid-import failure
|
||||
would leave the atlas half-loaded. `DELETE` is transactional, and at ~7k rows the cost is
|
||||
irrelevant. Point ids are also assigned explicitly rather than by `AUTO_INCREMENT`, because the
|
||||
join rows need them and `conn.batch()` reports no usable `insertId`.
|
||||
|
||||
**Measured result:** 6,455 points, 800 creatures, 23,927 point/type rows, 387 regions, 558
|
||||
landmarks, 25 champion altars. The placement transform resolves **83.2%** of points (3,689 by
|
||||
region, 1,690 by landmark, 1,086 Wilderness).
|
||||
|
||||
**One thing the design got exactly right:** the point-in-rect transform really is the reason to
|
||||
build this. "Where does a lizardman spawn?" answers *Shrines, Isamu-Jima, Yew* across three facets.
|
||||
|
||||
---
|
||||
|
||||
@@ -577,7 +665,7 @@ inherently up to one full cycle old, and the UI must say so.
|
||||
|---|---|---|---|---|
|
||||
| 1 | **A** — visibility framework + actor-leak fix | website, docs | none | ✅ Done |
|
||||
| 2 | **B/1** — `world.ruleset` (§5) | all four | new kind | ✅ Done |
|
||||
| 3 | **C** — spawn atlas (§6) | website, docs | none | ⬜ **Next** |
|
||||
| 3 | **C** — spawn atlas (§6) | website, docs | none | 🟡 Pipeline done, API/client next |
|
||||
| 4 | **B/2** — `points.board` (§7) | all four | new kind + `char.profile` field | ⬜ |
|
||||
| 5 | **B/3** — `vendor.listing` (§8) | all four | new kinds | ⬜ |
|
||||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3, `edge` → `main` | all four | the bump | ⬜ |
|
||||
|
||||
@@ -388,6 +388,67 @@ feature is ignored (a stale row must not resurrect a removed feature), an invali
|
||||
the default rather than failing open, and a rule touching a locked field (`acct` / `webId`) is
|
||||
discarded. See §6.5.
|
||||
|
||||
### shard_spawn_* / shard_regions / shard_landmarks / shard_champion_spawns / shard_atlas_meta — the spawn atlas (Protocol 3.0)
|
||||
|
||||
Static shard **content**, not live shard state. Nothing here comes from the sidecar: the atlas is
|
||||
derived from the shard's own ServUO tree, re-read on **every server boot** and hash-gated so an
|
||||
unchanged tree costs one read pass and no write. Nothing is precomputed and committed — a shard's
|
||||
maps change over its life, and a snapshot in the repo would silently drift from the world players
|
||||
actually see. These tables stay populated whether the shard is up or not. Full operator detail in
|
||||
[`SPAWN_ATLAS.md`](SPAWN_ATLAS.md); the design is `docs/link/v3.md` §6.
|
||||
|
||||
**No facet name appears anywhere in the code.** A shard may add facets, replace them, or rename them
|
||||
when its maps are updated; the facet set is discovered from the tree, and the loose spellings in
|
||||
`Data/Locations` are matched against it rather than looked up in a table.
|
||||
|
||||
| Table | Key columns |
|
||||
|---|---|
|
||||
| `shard_spawn_creatures` | `slug` PK, `name`, `total`, `points`, `facets` JSON, `art` NULL |
|
||||
| `shard_spawn_points` | `id` PK, `facet`, `name`, `x`, `y`, `width`, `height`, `spawn_range`, `max_count`, `min_delay`, `max_delay`, `tod_start/end/mode`, `region`, `landmark`, `label` |
|
||||
| `shard_spawn_point_types` | `(point_id, slug)` PK, `max_count` |
|
||||
| `shard_regions` | `facet`, `name`, `type`, `priority`, `parent`, `rects` JSON |
|
||||
| `shard_landmarks` | `facet`, `name`, `grp`, `x`, `y`, `z` |
|
||||
| `shard_champion_spawns` | `slug` PK, `name`, `grp`, `type`, `random_type`, `facet`, `x`, `y`, `z`, `radius`, `label` |
|
||||
| `shard_atlas_meta` | Singleton (`id = 1`), `payload` JSON (counts + a sha256 per source file), `imported_at` |
|
||||
| `shard_atlas_pending` | Singleton (`id = 1`), `status` (`pending`/`rejected`), `payload` JSON, `detected_at` |
|
||||
|
||||
The first seven are **import-owned**: a refresh empties and reloads every one inside a single
|
||||
transaction, so a failed reload leaves the previous atlas intact rather than a half-loaded world.
|
||||
Nothing else writes to them, and nothing holds a foreign key to them — no FKs at all, consistent with
|
||||
every other `shard_*` table.
|
||||
|
||||
**`shard_atlas_pending` is the security-relevant one.** A refresh that would REMOVE a facet is never
|
||||
applied automatically: facet loss is indistinguishable at boot from a half-copied or mid-update tree,
|
||||
so it is staged here for an admin to approve or reject, and **startup is never blocked by it**. Only
|
||||
the decision is stored — source hashes plus the facet diff, a few KB — and approving re-parses the
|
||||
tree, so a multi-megabyte blob never lands in the database and what gets applied matches the tree at
|
||||
approval time. A rejection is remembered against those exact hashes so a declined refresh does not
|
||||
re-prompt on every restart. Everything else (new facets, renamed regions, changed spawns) applies
|
||||
immediately, since none of it can destroy data an operator would miss.
|
||||
|
||||
The boot refresh is **best-effort by contract**: no configured path, an unreadable mount, a malformed
|
||||
file or a database error is caught and logged, and the site comes up serving whatever atlas it had.
|
||||
The tree path comes from the `spawn_atlas_servuo_path` setting, falling back to `SERVUO_PATH`.
|
||||
|
||||
Four column choices worth stating, because each one is a trap:
|
||||
|
||||
- **`spawn_range`, not `range`**, and **`grp`, not `group`** — both are reserved words.
|
||||
- **`DELETE`, not `TRUNCATE`.** `TRUNCATE` is DDL in MariaDB and implicitly commits, which would
|
||||
defeat the all-or-nothing reload. At ~7k rows the difference does not matter.
|
||||
- **Point ids are assigned explicitly**, not left to `AUTO_INCREMENT`: the `shard_spawn_point_types`
|
||||
rows need to know them, and `conn.batch()` reports no usable `insertId` for a multi-row insert.
|
||||
- **Plain `INDEX` on `name`, deliberately not `FULLTEXT`.** ~800 creature rows makes a `LIKE` scan
|
||||
free, and FULLTEXT's minimum token length would break searches for names like "orc".
|
||||
|
||||
`shard_champion_spawns` is the *configured* altar roster ("there is an Unholy Terror altar in
|
||||
Deceit"). The live `champ.update` feed in `shard_champs` is the separate answer to "it is on level 3
|
||||
right now". Both exist; they are not the same data.
|
||||
|
||||
**`shard_spawn_creatures.art` is always NULL on a fresh import.** The project ships no creature
|
||||
artwork: sprites live in the operator's own client `.mul`/`.uop` files and are theirs, not ours to
|
||||
redistribute. An operator supplies art via a gitignored map plus images under the (already
|
||||
gitignored) `server/uploads/atlas/`. Text-only is the normal, supported state.
|
||||
|
||||
---
|
||||
|
||||
## 4. API contract
|
||||
|
||||
261
website/SPAWN_ATLAS.md
Normal file
261
website/SPAWN_ATLAS.md
Normal file
@@ -0,0 +1,261 @@
|
||||
# Spawn atlas
|
||||
|
||||
**Status:** Data pipeline landed on `edge` (website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112)); API and client pages follow in a second PR.
|
||||
**Design:** [`docs/link/v3.md` §6](../link/v3.md) — Protocol 3.0 Part C.
|
||||
|
||||
The spawn atlas is a browsable catalogue of what the shard *contains*: which
|
||||
creatures spawn, where, how many, and which champion altars are configured. It
|
||||
answers "where do I find a lizardman?" with **"Shrines, Yew, Isamu-Jima"** rather
|
||||
than with a list of raw coordinates.
|
||||
|
||||
## Two things that shape the whole design
|
||||
|
||||
**The shard's ServUO tree is the single source of truth.** Nothing is
|
||||
precomputed and committed to the repository. A shard's maps change over its
|
||||
lifetime — facets get added, replaced, or renamed — and a snapshot in the repo
|
||||
would silently drift from the world players actually see. The atlas is therefore
|
||||
re-derived from the tree **on every server boot**.
|
||||
|
||||
**Facets are not a fixed list.** Nothing in the codebase names Felucca, Trammel,
|
||||
or any other stock facet. The facet set is whatever the shard's own files
|
||||
declare, discovered at parse time. A shard running entirely custom maps gets
|
||||
exactly the same treatment as a stock one, with no code change.
|
||||
|
||||
## What it is not
|
||||
|
||||
The atlas is **static shard content, not live shard state.**
|
||||
|
||||
- It does **not** come from the sidecar. Nothing here touches the bridge, and
|
||||
there is no event kind, no wire change and no `PROTOCOL_VERSION` bump for it.
|
||||
Part C is website-only.
|
||||
- It stays fully populated while the shard is down.
|
||||
- Its champion table (`shard_champion_spawns`) is the *configured roster* —
|
||||
"there is an Unholy Terror altar in Deceit". The live `champ.update` feed in
|
||||
`shard_champs` is the separate, sidecar-fed answer to "it is on level 3 right
|
||||
now". Both exist; do not conflate them.
|
||||
|
||||
Routes live at `/api/v1/public/atlas`, deliberately **not** under `/shard`,
|
||||
because `/shard/*` means sidecar-dependent.
|
||||
|
||||
## Configuring the tree
|
||||
|
||||
The website needs to be able to *read* the ServUO tree — same host, a bind mount,
|
||||
or a shared volume. Two ways to point at it, the setting winning over the
|
||||
environment:
|
||||
|
||||
| Source | Notes |
|
||||
|---|---|
|
||||
| `spawn_atlas_servuo_path` setting | Admin-editable; changes take effect on the next refresh without a redeploy |
|
||||
| `SERVUO_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |
|
||||
|
||||
With neither set the atlas is simply skipped — the site runs normally without
|
||||
one.
|
||||
|
||||
## The boot path
|
||||
|
||||
On every start the server hashes the source files and compares them against what
|
||||
is loaded. Unchanged (the normal case on a restart) costs one read pass, ~120 ms,
|
||||
and no database write. A real change costs a ~400 ms parse and a reload.
|
||||
|
||||
Two contracts govern it:
|
||||
|
||||
**1. It never blocks startup.** No configured path, an unreadable mount, a
|
||||
malformed file, a database error — every one is caught and logged, and the site
|
||||
comes up serving whatever atlas it already had.
|
||||
|
||||
**2. A facet disappearing is never applied automatically.** Losing a facet looks
|
||||
exactly like a half-copied or mid-update tree, and boot cannot tell that apart
|
||||
from a real map change. That refresh is *staged* for a human instead. Everything
|
||||
else — new facets, renamed regions, changed spawns — applies immediately, since
|
||||
none of it can destroy something an operator would miss.
|
||||
|
||||
```
|
||||
boot
|
||||
└─ path configured? no ──▶ skip
|
||||
└─ tree readable? no ──▶ warn, carry on
|
||||
└─ hashes changed? no ──▶ done (nothing parsed)
|
||||
└─ parse
|
||||
└─ a facet would be removed?
|
||||
no ──▶ import
|
||||
yes ──▶ stage for admin review; atlas unchanged
|
||||
```
|
||||
|
||||
### Approving or rejecting a staged refresh
|
||||
|
||||
Only the *decision* is stored, never the parsed world — a few KB of source hashes
|
||||
plus the facet diff. Approving **re-parses** the tree, so what lands matches the
|
||||
tree at approval time rather than at boot, and a multi-megabyte blob never sits
|
||||
in the database.
|
||||
|
||||
A rejection is remembered against those exact source hashes, so a declined
|
||||
refresh does not re-prompt on every restart. Change the tree and the hashes
|
||||
differ, which asks again.
|
||||
|
||||
From the admin panel (second PR), or from the CLI:
|
||||
|
||||
```bash
|
||||
cd website/server
|
||||
npm run atlas:import -- --status # what is loaded, and what is pending
|
||||
npm run atlas:import -- --approve # apply the staged refresh
|
||||
npm run atlas:import -- --reject # keep the current atlas, dismiss it
|
||||
```
|
||||
|
||||
## The CLI
|
||||
|
||||
The server refreshes itself on boot, so this is for applying a map change
|
||||
*without* a restart, and for the approve/reject flow above.
|
||||
|
||||
```bash
|
||||
npm run atlas:import # import if the tree differs
|
||||
npm run atlas:import -- --servuo <path> # override the path for this run
|
||||
npm run atlas:import -- --force # reimport even if unchanged
|
||||
```
|
||||
|
||||
`--servuo` is a per-run override and deliberately does **not** persist — changing
|
||||
where the atlas permanently reads from is an admin action, not a side effect of a
|
||||
one-off import.
|
||||
|
||||
## Sources
|
||||
|
||||
| File | Count (stock ServUO 57.4) | Used for |
|
||||
|---|---|---|
|
||||
| `Spawns/*.xml` | 13 files, ~10.5 MB | Every spawner: location, size, delays, time-of-day, creature types |
|
||||
| `Data/Regions.xml` | 129 KB, nested | Named regions and their rectangles |
|
||||
| `Data/Locations/*.xml` | 6 files | Landmarks (dungeon levels, town markers) |
|
||||
| `Config/ChampionSpawns.xml` | 4.8 KB | Configured champion altars |
|
||||
|
||||
**A stock tree has 13 spawn files but only 6 facets.** `Eodon.xml`,
|
||||
`GravewaterLake.xml`, `TreasuresOfKotl.xml` and the other named-area files hold
|
||||
TerMur/Trammel points. The facet always comes from each record's own `<Map>`,
|
||||
never from the file name.
|
||||
|
||||
## How a coordinate becomes a place name
|
||||
|
||||
This is the transform the atlas exists for, in `resolveRegion()`:
|
||||
|
||||
1. The highest-`priority` named region whose rectangle contains the point. Ties
|
||||
break toward the **smallest** rect, so a specific room wins over the
|
||||
dungeon-wide rect enclosing it.
|
||||
2. Otherwise the nearest landmark within the landmark radius (200 tiles by
|
||||
default), labelled by its **group** ("Covetous"), not its individual marker
|
||||
("Level 1").
|
||||
3. Otherwise `"Wilderness"`.
|
||||
|
||||
The radius cap in step 2 is what keeps step 3 reachable. Without it the nearest
|
||||
landmark is always *some* landmark however far away, and open countryside gets
|
||||
labelled with a dungeon on the far side of the map.
|
||||
|
||||
Against stock ServUO this resolves **83.2%** of points (5,369 of 6,455): 3,681 by
|
||||
region, 1,688 by landmark, 1,086 Wilderness.
|
||||
|
||||
## Three quirks in the source data
|
||||
|
||||
Each of these is silent if unhandled — the atlas still builds, it is just wrong.
|
||||
|
||||
**Facet names disagree between sources.** `Data/Locations/*.xml` spells them
|
||||
`Ter Mur` and `Tokuno Islands`, while `<Map>` and `<Facet name>` say `TerMur` and
|
||||
`Tokuno`. Unreconciled, the landmark bucket is keyed differently from the points
|
||||
looking it up, so the fallback never fires and every unregioned spawn on those
|
||||
facets reads "Wilderness".
|
||||
|
||||
This is reconciled **by matching, not by a lookup table** — there is no list of
|
||||
facet names anywhere. `facetKey()` collapses spelling differences (lowercase,
|
||||
alphanumerics only), and `resolveFacetName()` matches a loose spelling against
|
||||
the canonical set discovered from the shard's own spawn and region data, by exact
|
||||
key then by prefix in either direction. A name matching nothing keeps its own
|
||||
name: forcing a wrong match would file a real custom facet's landmarks under the
|
||||
wrong facet, which is worse than leaving it alone.
|
||||
|
||||
**Spawn type tokens carry XmlSpawner directives.** The `<Objects2>` type is not
|
||||
always a bare class name:
|
||||
|
||||
```
|
||||
Fairy,{RND,4,8} alchemist/z/-50 Agralem/Name/Agralem
|
||||
GargishRouser,1 greatape,true GargishRefugee/hue/34532
|
||||
```
|
||||
|
||||
Taken literally these invent creatures that do not exist *and* split real ones in
|
||||
two, because `Fairy` and `Fairy,{RND,4,8}` slug apart into separate entries. 71 of
|
||||
845 were affected. Everything from the first `/` or `,` is stripped, leaving 800
|
||||
real creatures.
|
||||
|
||||
**Case is inconsistent across files.** The same creature is `Lizardman` in one
|
||||
file and `lizardman` in another. Slugging collapses them correctly, but the
|
||||
display name is chosen deterministically — most common spelling wins, ties break
|
||||
to the more capitalised form, then alphabetically — because otherwise it would
|
||||
depend on file read order and change on an unrelated restart.
|
||||
|
||||
## Tables
|
||||
|
||||
All are **import-owned**: a refresh empties and reloads them in one transaction,
|
||||
so a failed reload leaves the previous atlas intact rather than a half-loaded
|
||||
world. Nothing else writes to them and nothing holds a foreign key to them — no
|
||||
FKs at all, consistent with every other `shard_*` table. Full column listings in
|
||||
[`BACKEND_DESIGN.md`](BACKEND_DESIGN.md).
|
||||
|
||||
| Table | Rows (stock) | Notes |
|
||||
|---|---|---|
|
||||
| `shard_spawn_creatures` | 800 | `slug` PK; `total` = sum of each type's own max; nullable `art` |
|
||||
| `shard_spawn_points` | 6,455 | `spawn_range`, since `range` is reserved in MariaDB |
|
||||
| `shard_spawn_point_types` | 23,927 | The many-to-many; one spawner commonly carries six types |
|
||||
| `shard_regions` | 387 | Flattened out of the nesting; `rects` JSON |
|
||||
| `shard_landmarks` | 558 | `grp`, since `group` is reserved in SQL |
|
||||
| `shard_champion_spawns` | 25 | Configured altars, not the live feed |
|
||||
| `shard_atlas_meta` | 1 | Singleton; source hashes, for the change check |
|
||||
| `shard_atlas_pending` | 0–1 | Singleton; a staged refresh awaiting admin review |
|
||||
|
||||
`shard_spawn_creatures.name` carries a plain `INDEX`, deliberately **not
|
||||
`FULLTEXT`**: ~800 rows makes a `LIKE` scan free, and FULLTEXT's minimum token
|
||||
length would break searches for names like "orc".
|
||||
|
||||
The reload uses `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and would
|
||||
implicitly commit, defeating the all-or-nothing guarantee. Point ids are assigned
|
||||
explicitly rather than left to `AUTO_INCREMENT`, because the join rows need them
|
||||
and `conn.batch()` reports no usable `insertId`.
|
||||
|
||||
## Artwork — operator-supplied, never shipped
|
||||
|
||||
**This project ships no creature art and no extraction tooling, and never will.**
|
||||
UO sprites live in the operator's own client `.mul`/`.uop` files. They are the
|
||||
operator's, not ours to redistribute.
|
||||
|
||||
The atlas is fully functional as text. `shard_spawn_creatures.art` is nullable
|
||||
and is NULL on every fresh import; pages render without images, which is the
|
||||
normal and supported state, not a degraded one.
|
||||
|
||||
An operator who wants art:
|
||||
|
||||
1. Extracts it from **their own** client files (UOFiddler, ClassicUO tooling, or
|
||||
any art extractor).
|
||||
2. Drops the images under `server/uploads/atlas/`.
|
||||
3. Copies `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json`
|
||||
and maps creature slugs to file names.
|
||||
4. Restarts, or runs `npm run atlas:import -- --force`.
|
||||
|
||||
Both `spawnAtlas.art.json` and `server/uploads/` are gitignored, so neither the
|
||||
map nor the images can be committed by accident.
|
||||
|
||||
## Code layout
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `src/utils/spawnAtlasParse.js` | **Pure and fs-free** parsers, so CI covers them with no ServUO tree. Zero dependencies. |
|
||||
| `src/utils/spawnAtlasSource.js` | The only thing that reads a ServUO tree; shared by the boot path and the CLI |
|
||||
| `src/model/shardAtlas/shardAtlas.db.js` | The one-transaction replace |
|
||||
| `src/model/shardAtlas/shardAtlas.model.js` | The refresh decision, staging, approve/reject |
|
||||
| `scripts/importSpawnAtlas.js` | Thin CLI over the model |
|
||||
|
||||
Parsing notes:
|
||||
|
||||
- `Regions.xml`, `Locations/*.xml` and `ChampionSpawns.xml` genuinely nest, and
|
||||
get a small hand-rolled **subset** tokenizer — elements, attributes,
|
||||
self-closing tags, comments, the XML declaration, CDATA, and the five
|
||||
predefined entities plus numeric refs. It is not a general-purpose XML parser
|
||||
and must not be reused as one.
|
||||
- The ~10.5 MB of `Spawns/*.xml` never touches that tokenizer. Those records are
|
||||
flat, so they get a streaming regex sweep instead; a DOM would allocate a node
|
||||
per element across ~40 fields on every record to keep 14 of them. **Do not put
|
||||
the Points files through a DOM parser.**
|
||||
- `<Objects2>` is `Type:MX=n:SB=…` segments joined by `:OBJ=`. Split on `:OBJ=`
|
||||
*first* — a naive `split(':')` shreds it. A single Trammel point carries six
|
||||
types.
|
||||
Reference in New Issue
Block a user