feat(atlas): the spawn atlas reads the shard, not the shard's filesystem (Phase 7) #40

Merged
whitlocktech merged 1 commits from feat/asset-bridge-p7 into edge 2026-09-14 07:37:20 +00:00
Member

Phase 7 of the Asset Bridge — docs/link/v8.md §10. Pairs with servuo-plugins#34. link needed nothing.

What changes

spawnAtlasSource.js gains a second backend behind its existing interface. The bridge wins whenever uo-link is configured and enabled (the rule §17.7 settled for the cliloc table); a local ServUO tree is the source where there is no shard link; an explicit --servuo <path> is an instruction and overrules both.

The parsers do not move. spawnAtlasParse.js is still pure, still fs-free, still CI-covered without a ServUO tree anywhere near it. buildFromFiles is now where the parse starts, and both readers feed it the same shape — so there is no second build to be subtly different.

Boot does not call the shard

Same answer and same reasoning as the cliloc table: a local tree hashes in ~120 ms and skips, while a round trip in the boot sequence would answer "no" on every restart that did not follow a map edit. Editing spawn files is an operator action, so importing is one — Admin → Spawn Atlas → Import now, or the CLI.

What that costs is real, so it is said out loud rather than left to be discovered: an install on the bridge has no automatic refresh at all, and an edited spawn file stays invisible until someone presses the button. The panel says so, the CLI says so, and the skip is logged rather than silent. (The alternative — one manifest round trip per boot, ~70 ms and no file bytes — was on the table and was declined in favour of consistency.)

Three checks in treeBridge.js that are not decoration

Each is a way this ends in a tree that looks imported. XML is forgiving about what it skips, so a mis-assembled spawn file parses cleanly and simply has fewer spawns in it.

  • Every chunk re-declares its address — label, index, offset — and carries the hash of its own uncompressed bytes. Chunks are placed by declared index, not arrival order (there is a test that reverses the page).
  • The whole file is hashed after reassembly against its manifest row.
  • The catalog must not move mid-walk, or the import is refused rather than stitched out of two trees.

Two things only the live walk found

  • PARSER_VERSION 4 → 5. The parse is order-sensitive in one place: the decoration index keeps the first item id it sees for a type. The two readers agreed on a stock tree by coincidence — the filesystem reader walks each directory with localeCompare, the shard sorts whole relative paths, and those diverge the moment a directory mixes cases. buildFromFiles now sorts by label, ordinally, once, whatever order the files arrived in. Identical input, a different answer for a handful of types — exactly what the version number exists to push through the hash gate.

    The parity test asserted deepEqual, which ignores object key order — so it passed while the real trees differed in meta.source's ordering. It now asserts serialised equality too.

  • The source fingerprint is taken over raw bytes at both ends. Hashing the decoded text hashes a UTF-8 re-encoding: identical for valid UTF-8, different for a file that is not, because an undecodable byte becomes U+FFFD and never comes back. One Latin-1 character in a creature name would have made the drift gate report a change on every import, forever, with the tree untouched. (All-ASCII trees fingerprint identically, so no existing install sees a spurious re-import from this.)

A 200 from assets.sources stopped meaning what it meant

A shard may now serve its configuration tree while declining to serve its UO client, so that call answers when either plane is on. Both client-file readers therefore check assetsEnabled and raise DISABLED — otherwise an empty file list reads as "your client has no cliloc.enu", and sends an operator to their client install looking for a setting that lives on their shard.

Measured end to end

Live ServUO shard → real sidecar → this reader:

Files / bytes 141 / 11,895,427
Chunks / pages 158 / 3
On the wire 1.33 MB
Full import 512 ms
Verification all 141 byte-identical to disk
Atlas parity identical — 6,455 points, 800 creatures, 387 regions, 558 landmarks, 25 champions, 309 decoration types, built from the bridge and from the disk
Drift check 141 rows, ~70 ms, no file bytes

Tests

server 713 pass, client 42 pass, check:imports / check:bundle / check:swagger / check:externals all OK, client build clean. 22 new tests across treeBridge.test.js (the walk, its five refusals, the zero-byte file, the tree-only shard, and the disk-vs-bridge parity) and atlasSourceSelection.test.js (which source runs, and that boot makes no shard call).

One pre-existing assertion changed wording: checkPage's message gained a noun so the tree walk does not report "assets".

🤖 Generated with Claude Code

https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4

Phase 7 of the Asset Bridge — `docs/link/v8.md` §10. Pairs with **servuo-plugins#34**. `link` needed nothing. ## What changes `spawnAtlasSource.js` gains a second backend behind its existing interface. **The bridge wins whenever uo-link is configured and enabled** (the rule §17.7 settled for the cliloc table); a local ServUO tree is the source where there is no shard link; an explicit `--servuo <path>` is an instruction and overrules both. **The parsers do not move.** `spawnAtlasParse.js` is still pure, still fs-free, still CI-covered without a ServUO tree anywhere near it. `buildFromFiles` is now where the parse starts, and both readers feed it the same shape — so there is no second build to be subtly different. ## Boot does not call the shard Same answer and same reasoning as the cliloc table: a local tree hashes in ~120 ms and skips, while a round trip in the boot sequence would answer "no" on every restart that did not follow a map edit. Editing spawn files is an operator action, so importing is one — **Admin → Spawn Atlas → Import now**, or the CLI. **What that costs is real**, so it is said out loud rather than left to be discovered: an install on the bridge has *no automatic refresh at all*, and an edited spawn file stays invisible until someone presses the button. The panel says so, the CLI says so, and the skip is logged rather than silent. (The alternative — one manifest round trip per boot, ~70 ms and no file bytes — was on the table and was declined in favour of consistency.) ## Three checks in `treeBridge.js` that are not decoration Each is a way this ends in a tree that *looks* imported. XML is forgiving about what it skips, so a mis-assembled spawn file parses cleanly and simply has fewer spawns in it. - **Every chunk re-declares its address** — label, index, offset — and carries the hash of its own uncompressed bytes. Chunks are placed by **declared index**, not arrival order (there is a test that reverses the page). - **The whole file is hashed after reassembly** against its manifest row. - **The `catalog` must not move mid-walk**, or the import is refused rather than stitched out of two trees. ## Two things only the live walk found - **`PARSER_VERSION` 4 → 5.** The parse is order-sensitive in one place: the decoration index keeps the **first** item id it sees for a type. The two readers agreed on a stock tree *by coincidence* — the filesystem reader walks each directory with `localeCompare`, the shard sorts whole relative paths, and those diverge the moment a directory mixes cases. `buildFromFiles` now sorts by label, ordinally, once, whatever order the files arrived in. Identical input, a different answer for a handful of types — exactly what the version number exists to push through the hash gate. The parity test asserted `deepEqual`, **which ignores object key order** — so it passed while the real trees differed in `meta.source`'s ordering. It now asserts serialised equality too. - **The source fingerprint is taken over raw bytes** at both ends. Hashing the decoded text hashes a UTF-8 *re-encoding*: identical for valid UTF-8, different for a file that is not, because an undecodable byte becomes U+FFFD and never comes back. One Latin-1 character in a creature name would have made the drift gate report a change on every import, forever, with the tree untouched. (All-ASCII trees fingerprint identically, so no existing install sees a spurious re-import from this.) ## A 200 from `assets.sources` stopped meaning what it meant A shard may now serve its configuration tree while declining to serve its UO client, so that call answers when *either* plane is on. Both client-file readers therefore check `assetsEnabled` and raise `DISABLED` — otherwise an empty file list reads as *"your client has no cliloc.enu"*, and sends an operator to their client install looking for a setting that lives on their shard. ## Measured end to end Live ServUO shard → real sidecar → this reader: | | | |---|---| | Files / bytes | 141 / 11,895,427 | | Chunks / pages | 158 / 3 | | On the wire | **1.33 MB** | | Full import | **512 ms** | | Verification | all 141 byte-identical to disk | | **Atlas parity** | **identical** — 6,455 points, 800 creatures, 387 regions, 558 landmarks, 25 champions, 309 decoration types, built from the bridge and from the disk | | Drift check | 141 rows, ~70 ms, no file bytes | ## Tests `server` **713 pass**, `client` 42 pass, `check:imports` / `check:bundle` / `check:swagger` / `check:externals` all OK, client build clean. 22 new tests across `treeBridge.test.js` (the walk, its five refusals, the zero-byte file, the tree-only shard, and the disk-vs-bridge parity) and `atlasSourceSelection.test.js` (which source runs, and that boot makes no shard call). One pre-existing assertion changed wording: `checkPage`'s message gained a noun so the tree walk does not report "assets". 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
wtclaude added 1 commit 2026-09-14 07:02:36 +00:00
feat(atlas): the spawn atlas reads the shard, not the shard's filesystem (Phase 7)
All checks were successful
PR Checks / server-tests (pull_request) Successful in 23s
PR Checks / frozen-manifest (pull_request) Successful in 1m8s
PR Checks / client-build (pull_request) Successful in 7m59s
d4d5989926
`spawnAtlasSource.js` gains a second backend behind its existing interface
(docs/link/v8.md 10). Where a shard is linked and enabled the tree arrives over
the sidecar; where there is none, a local ServUO tree is read exactly as before.
An explicit --servuo path is an instruction and overrules both.

The parsers do not move. spawnAtlasParse.js is still pure, still fs-free and
still CI-covered without a ServUO tree anywhere near it; `buildFromFiles` is now
where the parse starts, and both readers feed it the same shape.

treeBridge.js walks the manifest and then the chunks. Three of its checks are
not decoration -- each is a way this ends in a tree that LOOKS imported, and XML
is forgiving enough that a mis-assembled spawn file parses cleanly and simply
has fewer spawns in it:

  - every chunk re-declares its address and carries the hash of its own
    uncompressed bytes, and chunks are placed by declared index rather than
    arrival order
  - the whole file is hashed after reassembly against its manifest row
  - the catalog must not move mid-walk, or the import is refused rather than
    stitched out of two trees

Boot does not call the shard. The same answer 17.7 gave the cliloc table, and
the same reasoning: a local tree hashes in ~120 ms and skips, while a round trip
in the boot sequence would answer "no" on every restart that did not follow a
map edit. Editing spawn files is an operator action, so importing is one --
Admin -> Spawn Atlas -> Import. What that costs is real and is said out loud in
the panel, the CLI and the log: an install on the bridge has NO automatic
refresh at all.

Two things the live walk found that the unit tests could not:

  - PARSER_VERSION 4 -> 5. The parse is order-sensitive in one place -- the
    decoration index keeps the FIRST item id it sees for a type -- and the two
    readers agreed on a stock tree by coincidence, since the filesystem reader
    walks each directory with localeCompare while the shard sorts whole relative
    paths. buildFromFiles now sorts by label, ordinally, once, whatever order
    the files arrived in. Identical input, a different answer for a handful of
    types: exactly what the version number exists to push through the hash gate.
    The parity test asserted deepEqual, which ignores key order; it now asserts
    serialised equality too.
  - The source fingerprint is taken over RAW BYTES at both ends. Hashing decoded
    text hashes a UTF-8 re-encoding -- identical for valid UTF-8, different for a
    file that is not, because an undecodable byte becomes U+FFFD and never comes
    back. One Latin-1 character in a creature name would have made the drift gate
    report a change on every import, forever, with the tree untouched.

A 200 from assets.sources also stopped meaning "the client files are on offer":
a shard may now serve its configuration tree while declining to serve its UO
client. Both client-file readers check `assetsEnabled` and say DISABLED, instead
of reading an empty file list as "your client has no cliloc.enu" and sending an
operator to their client install for a setting that lives on their shard.

Measured end to end against a live shard and the real sidecar: 141 files,
11.9 MB, 158 chunks, 3 pages, 1.33 MB on the wire, 512 ms; every file
byte-identical to disk; and the atlas built over the bridge identical to the one
built off it -- 6,455 points, 800 creatures, 387 regions, 558 landmarks,
25 champions, 309 decoration types.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
whitlocktech merged commit 7dbdaa14ad into edge 2026-09-14 07:37:20 +00:00
whitlocktech deleted branch feat/asset-bridge-p7 2026-09-14 07:37:22 +00:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: RunicGateway/Module-uo#40
No description provided.