docs(website): the atlas reads the shard's tree on every boot, not a snapshot

Follows the redesign in website #112. Two decisions from the original §6 were
rejected in review and replaced; the docs now describe what was actually built.

**The committed artifact is gone.** A shard's maps change over its life, so a
snapshot in the repo silently drifts from the world players actually see. The
ServUO tree is the single source of truth and the atlas is re-derived on every
server boot, hash-gated so an unchanged tree costs one read pass and no write.

**Nothing may name a facet.** The first implementation carried a lookup table of
the six stock UO facets. A shard may add facets, replace them outright, or rename
them when its maps are updated, and a built-in list mishandles all three
silently. Reconciliation is now by matching against the facet set discovered from
the shard's own data, with an unmatched name keeping its own rather than being
forced into a wrong bucket.

## Changes

- **`website/SPAWN_ATLAS.md`** rewritten: the two ideas that shape the design,
  how to configure the tree path, the boot flow as a decision tree, the
  approve/reject flow, and the code layout. The artwork policy is unchanged and
  still explicit — no art ever ships, operators extract their own from their own
  client files.
- **`link/v3.md` §6.1 (new)** records the two rejected decisions plus the two
  boot-path contracts. The old "what real data changed" list becomes §6.2. §6's
  now-superseded passages — the artifact bullet, the payload budget, the operator
  re-run story — are marked rather than deleted, so the reasoning stays legible.
- **`website/BACKEND_DESIGN.md`** documents `shard_atlas_pending` and the two
  contracts that make it safe: a facet removal is staged for a human, and the
  boot refresh can never block startup.

The two contracts are the part worth reviewing. Losing a facet is
indistinguishable at boot from a half-copied or mid-update tree, so it is staged
rather than applied; and no failure mode of the atlas — missing path, unreadable
mount, malformed file, database error — is allowed to stop the site coming up.

---

- [x] AI-assisted: written with **Claude Code** (Claude Opus 5), reviewed before opening.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP
This commit is contained in:
2026-07-28 16:44:39 -05:00
parent 3fb3f63f25
commit ff1c2064a5
3 changed files with 214 additions and 126 deletions

View File

@@ -335,22 +335,25 @@ frame during verification.
> lives, and burying it under routes and React would have meant reviewing it in a 10k-line diff.
> Full operator documentation: [`docs/website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md).
>
> §6.1 below records where the shipped implementation differs from this design. The differences are
> all things the real ServUO data forced, not changes of mind.
> **§6 below is the original design and is partly superseded.** §6.1 records two decisions that were
> rejected in review and replaced (the committed artifact, and the fixed facet list); §6.2 records
> the corrections the real ServUO data forced. Read both before trusting §6.
**Decision: committed generated artifact + idempotent DB import**, split in two because the build
needs the ServUO tree (which the website container does not have) and the import does not. Not
runtime import (10.5 MB of XML per boot), not a browser-served blob.
**Decision (revised at implementation time): the shard's ServUO tree is the single source of truth,
re-derived on every server boot.** The original plan here was a committed generated artifact plus an
idempotent import. That was rejected in review for two reasons, recorded in §6.1: a snapshot in the
repo goes stale as a shard's maps change, and the design leaned on a fixed facet list that no shard
is obliged to keep. Still not a browser-served blob; still parsed server-side only.
New in `website/server/`:
- `src/utils/spawnAtlasParse.js`**pure functions, no fs**, so they are unit-testable in CI without
a ServUO tree: `parseObjects2()`, `parsePoints()`, `parseRegions()`, `parseLocations()`,
`resolveRegion()`.
- `scripts/buildSpawnAtlas.js` (`--servuo <path> --out db/data/`) and `scripts/importSpawnAtlas.js`
(TRUNCATE + batched INSERT in one transaction); `package.json` scripts `atlas:build`, `atlas:import`.
- `db/data/spawnAtlas.<facet>.json` ×13 + `spawnAtlas.index.json` (creatures, champions, regions,
landmarks, meta with per-source-file hashes).
- ~~`scripts/buildSpawnAtlas.js` and a committed `db/data/spawnAtlas.*.json` artifact~~ — dropped,
see §6.1 R1. Replaced by `src/utils/spawnAtlasSource.js` (the only thing that reads a ServUO tree,
shared by the boot path and the CLI) and a `scripts/importSpawnAtlas.js` that is a thin CLI over
the model. `package.json` gains `atlas:import` only.
- `src/model/shardAtlas/{shardAtlas.db.js,shardAtlas.model.js}` following the `shardState` split.
- `src/router/v1/public/atlas.{router,controller}.js`; `test/spawnAtlas.parse.test.js`.
@@ -381,18 +384,48 @@ stays CLI-only.**
Client: `routes/public/Atlas.jsx` (`/site/atlas`) and `AtlasCreature.jsx` (`/site/atlas/:slug`).
**Payload risk**a monolithic artifact would be 23 MB of committed JSON. Shard per facet and drop
every `<Points>` field the site cannot use (`UniqueId`, all trigger/refractory/proximity/sequential
fields, sound ids), keeping Name/Map/X/Y/W/H/Range/MaxCount/MinDelay/MaxDelay/TOD*/types — well under
1 MB. The artifact never reaches the browser; the browser sees only paginated API responses.
**Payload risk***superseded by §6.1 R1; nothing is committed.* The field selection it describes
still applies at parse time: every `<Points>` field the site cannot use (`UniqueId`, all
trigger/refractory/proximity/sequential fields, sound ids) is dropped, keeping
Name/Map/X/Y/W/H/Range/MaxCount/MinDelay/MaxDelay/TOD*/types. Parsed data never reaches the browser;
the browser sees only paginated API responses.
**Operator re-run story**spawns changed → `npm run atlas:build -- --servuo <path>` on a machine
with the tree → commit the regenerated `db/data/spawnAtlas.*.json` → deploy `npm run atlas:import`
(or `POST /admin/shard/atlas/import`). `shard_atlas_meta.source` holds per-file hashes, so
`GET /admin/shard/atlas/status` reports when the DB is behind the artifact. Full detail in
`docs/website/SPAWN_ATLAS.md`.
**Operator re-run story***revised by §6.1 R1.* Spawns changed → restart, or
`npm run atlas:import` / `POST /admin/shard/atlas/import` to apply without one. `shard_atlas_meta`
holds a sha256 per source file, so the server can tell on boot whether anything changed, and
`GET /admin/shard/atlas/status` reports drift. If the change would remove a facet it is staged for
approval rather than applied (§6.1 R3). Full detail in `docs/website/SPAWN_ATLAS.md`.
### 6.1 What the build against real data changed
### 6.1 What implementation changed
Two design decisions in §6 were rejected in review and replaced; the rest are corrections the real
ServUO data forced. Kept as a diff rather than edited in place, because each is a trap the next
person would otherwise re-enter.
**R1. The committed artifact is gone — the tree is re-parsed on every boot.** §6 proposed building a
generated artifact, committing it, and importing it. Two problems. A shard's maps change over its
life, so a snapshot in the repo silently drifts from the world players actually see; and the build/
import split existed only to work around the website container not having a tree, which is a
deployment question (mount it) rather than a reason to freeze data. The server now hashes the source
files on boot and re-derives the atlas when they differ. `scripts/buildSpawnAtlas.js`, the 1.41 MB
artifact, and the whole encode/decode seam it needed are deleted.
**R2. Nothing may name a facet.** The first implementation carried a lookup table of the six stock
UO facets to reconcile the spelling drift between sources. A shard may add facets, replace them
outright, or rename them when its maps are updated, and a built-in list mishandles all three
silently. Reconciliation is now by *matching* against the facet set discovered from the shard's own
spawn and region data — exact key, then prefix in either direction — with an unmatched name keeping
its own rather than being forced into a wrong bucket.
**R3. Two contracts on the boot path.** It never blocks startup: no path, an unreadable mount, a
malformed file or a database error is caught and logged, and the site comes up serving whatever
atlas it had. And a refresh that would REMOVE a facet is never applied automatically — facet loss
is indistinguishable at boot from a half-copied or mid-update tree, so it is staged in
`shard_atlas_pending` for an admin to approve or reject. Only the decision is stored (source hashes
+ the facet diff, a few KB); approving re-parses, so what lands matches the tree at approval time.
A rejection is remembered against those hashes so it does not re-prompt every restart.
### 6.2 What the build against real data changed
Six corrections to the design above, from running it against stock ServUO 57.4. Kept as a diff
rather than edited in place, because each one is a trap the next person would otherwise re-enter.
@@ -419,12 +452,11 @@ they invent creatures that do not exist *and* split real ones in two, since `Fai
leaves **800** real creatures. (The design's "~1,500 creature rows" estimate was high; 800 only
reinforces the plain-`INDEX`-not-`FULLTEXT` call.)
**5. The artifact is 1.41 MB, not "well under 1 MB".** Dropping the unused `<Points>` fields as the
design directed still left 4.40 MB. Three further encodings `facet` dropped per record,
default-valued fields omitted rather than written as `0`, and `types` as `[name, max]` tuples
(~24,000 entries × 15 bytes of repeated key names) — brought it to 1.41 MB. Getting under 1 MB would
mean dropping the spawner `name`, which is the only human handle on a specific spawner and worth
keeping. `encodePoint()` and `readPoint()` are exact inverses and are round-tripped in tests.
**5. The artifact would have been 1.41 MB, not "well under 1 MB" — and is now moot.** Dropping the
unused `<Points>` fields as the design directed still left 4.40 MB; three further encodings brought
it to 1.41 MB, and getting under 1 MB would have meant dropping the spawner `name`. The size budget
in §6 was simply optimistic for 6,455 points. Superseded by §6.1 R1: there is no artifact, so there
is no payload to budget and no encode/decode seam to keep in sync.
**6. `DELETE`, not `TRUNCATE`.** The design said "TRUNCATE + batched INSERT in one transaction",
which does not hold: `TRUNCATE` is DDL in MariaDB and implicitly commits, so a mid-import failure