feat(atlas): the spawn atlas reads the shard, not the shard's filesystem (Phase 7)
All checks were successful
PR Checks / server-tests (pull_request) Successful in 23s
PR Checks / frozen-manifest (pull_request) Successful in 1m8s
PR Checks / client-build (pull_request) Successful in 7m59s

`spawnAtlasSource.js` gains a second backend behind its existing interface
(docs/link/v8.md 10). Where a shard is linked and enabled the tree arrives over
the sidecar; where there is none, a local ServUO tree is read exactly as before.
An explicit --servuo path is an instruction and overrules both.

The parsers do not move. spawnAtlasParse.js is still pure, still fs-free and
still CI-covered without a ServUO tree anywhere near it; `buildFromFiles` is now
where the parse starts, and both readers feed it the same shape.

treeBridge.js walks the manifest and then the chunks. Three of its checks are
not decoration -- each is a way this ends in a tree that LOOKS imported, and XML
is forgiving enough that a mis-assembled spawn file parses cleanly and simply
has fewer spawns in it:

  - every chunk re-declares its address and carries the hash of its own
    uncompressed bytes, and chunks are placed by declared index rather than
    arrival order
  - the whole file is hashed after reassembly against its manifest row
  - the catalog must not move mid-walk, or the import is refused rather than
    stitched out of two trees

Boot does not call the shard. The same answer 17.7 gave the cliloc table, and
the same reasoning: a local tree hashes in ~120 ms and skips, while a round trip
in the boot sequence would answer "no" on every restart that did not follow a
map edit. Editing spawn files is an operator action, so importing is one --
Admin -> Spawn Atlas -> Import. What that costs is real and is said out loud in
the panel, the CLI and the log: an install on the bridge has NO automatic
refresh at all.

Two things the live walk found that the unit tests could not:

  - PARSER_VERSION 4 -> 5. The parse is order-sensitive in one place -- the
    decoration index keeps the FIRST item id it sees for a type -- and the two
    readers agreed on a stock tree by coincidence, since the filesystem reader
    walks each directory with localeCompare while the shard sorts whole relative
    paths. buildFromFiles now sorts by label, ordinally, once, whatever order
    the files arrived in. Identical input, a different answer for a handful of
    types: exactly what the version number exists to push through the hash gate.
    The parity test asserted deepEqual, which ignores key order; it now asserts
    serialised equality too.
  - The source fingerprint is taken over RAW BYTES at both ends. Hashing decoded
    text hashes a UTF-8 re-encoding -- identical for valid UTF-8, different for a
    file that is not, because an undecodable byte becomes U+FFFD and never comes
    back. One Latin-1 character in a creature name would have made the drift gate
    report a change on every import, forever, with the tree untouched.

A 200 from assets.sources also stopped meaning "the client files are on offer":
a shard may now serve its configuration tree while declining to serve its UO
client. Both client-file readers check `assetsEnabled` and say DISABLED, instead
of reading an empty file list as "your client has no cliloc.enu" and sending an
operator to their client install for a setting that lives on their shard.

Measured end to end against a live shard and the real sidecar: 141 files,
11.9 MB, 158 chunks, 3 pages, 1.33 MB on the wire, 512 ms; every file
byte-identical to disk; and the atlas built over the bridge identical to the one
built off it -- 6,455 points, 800 creatures, 387 regions, 558 landmarks,
25 champions, 309 decoration types.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
2026-09-14 02:00:43 -05:00
parent 679762b643
commit d4d5989926
13 changed files with 1432 additions and 73 deletions

View File

@@ -5,13 +5,13 @@ const db = require('./shardAtlas.db')
const core = require('../../core')
const { settings } = core
const { slugify } = require('../../utils/spawnAtlasParse')
const {
AtlasSourceError,
PARSER_VERSION,
buildAtlas,
hashSources,
sameSources,
} = require('../../utils/spawnAtlasSource')
const { AtlasSourceError, PARSER_VERSION, sameSources } = require('../../utils/spawnAtlasSource')
// The two readers are reached through the namespace rather than destructured,
// because a test stubs them ON the module object and a binding taken at require
// time would keep calling the real one — quietly, and while reporting success.
const spawnAtlasSource = require('../../utils/spawnAtlasSource')
const { TreeBridgeError } = require('../../utils/treeBridge')
const uoLinkConfig = require('../uoLinkConfig/uoLinkConfig.model')
const log = require('../../core').logger('shardAtlas')
// The spawn atlas, refreshed from the shard's own ServUO tree.
@@ -37,6 +37,45 @@ const log = require('../../core').logger('shardAtlas')
const SETTING_KEY = 'spawn_atlas_servuo_path'
/**
* Is there a shard to ask?
*
* Both halves matter. `baseUrl` alone is an install that has been configured and
* then switched off, and calling it would spend a 12 s timeout to learn what the
* row already says. Never throws: an unreadable config means "no shard", and a
* local tree is a working answer.
*/
async function shardLinked() {
try {
const config = await uoLinkConfig.getSafe()
return Boolean(config?.enabled && config?.baseUrl)
} catch {
return false
}
}
/**
* Which end this atlas is built from (docs/link/v8.md §10, §17.7).
*
* **The bridge wins whenever uo-link is configured and enabled**, the same rule
* the cliloc table follows and for the same reason: there is no version of "which
* source?" an operator benefits from answering, so there is no setting asking it.
* A local tree remains the source where there is no shard link — development,
* same-host installs — plus the one-off explicit path an admin can type, which is
* an instruction rather than a default and therefore overrules this.
*/
async function sourceFor(pathOverride = '') {
const explicit = String(pathOverride || '').trim()
if (explicit !== '') return { kind: 'fs', root: explicit }
if (await shardLinked()) return { kind: 'bridge', root: '' }
return { kind: 'fs', root: await getServuoPath() }
}
/** How a source reads in a log line or an admin panel. */
const describe = (source) => (source.kind === 'bridge' ? 'the shard bridge' : source.root)
/**
* Where the ServUO tree lives.
*
@@ -175,18 +214,29 @@ const currentParser = (meta) => meta?.parserVersion === PARSER_VERSION
async function refresh({ force = false, approve = false, path: pathOverride = '' } = {}) {
// An explicit override wins outright — it is a one-off "use this tree", and it
// must not be silently overruled by the configured path the way an env default
// would be.
const root = pathOverride.trim() !== '' ? pathOverride.trim() : await getServuoPath()
if (root === '') return { status: 'skipped', reason: 'no ServUO path configured' }
// would be, nor by the bridge.
const source = await sourceFor(pathOverride)
const root = source.root
const where = describe(source)
if (source.kind === 'fs' && root === '') {
return { status: 'skipped', reason: 'no ServUO path configured' }
}
let hashes
try {
hashes = hashSources(root)
hashes = await spawnAtlasSource.hashFrom(source)
} catch (err) {
if (err instanceof AtlasSourceError) {
return { status: 'unavailable', reason: err.message, code: err.code, path: root }
if (err instanceof AtlasSourceError || err instanceof TreeBridgeError) {
return {
status: 'unavailable',
source: source.kind,
reason: err.message,
code: err.code,
path: where,
}
}
return { status: 'failed', reason: err.message, path: root }
return { status: 'failed', source: source.kind, reason: err.message, path: where }
}
const meta = await db.getMeta().catch(() => null)
@@ -199,7 +249,7 @@ async function refresh({ force = false, approve = false, path: pathOverride = ''
// whatever an older build derived — a corrected parse would ship and never
// reach the data.
if (!force && sameSources(hashes, loaded) && currentParser(meta)) {
return { status: 'unchanged', path: root }
return { status: 'unchanged', source: source.kind, path: where }
}
// A rejected refresh must not re-prompt on every boot. It stays rejected until
@@ -207,14 +257,28 @@ async function refresh({ force = false, approve = false, path: pathOverride = ''
// decision.
const pending = await db.getPending().catch(() => null)
if (!approve && !force && pending?.status === 'rejected' && sameSources(hashes, pending.hashes)) {
return { status: 'unchanged', path: root, reason: 'refresh previously rejected' }
return {
status: 'unchanged',
source: source.kind,
path: where,
reason: 'refresh previously rejected',
}
}
let atlas
try {
atlas = buildAtlas(root)
atlas = await spawnAtlasSource.buildFrom(source)
} catch (err) {
return { status: 'failed', reason: err.message, path: root }
if (err instanceof AtlasSourceError || err instanceof TreeBridgeError) {
return {
status: 'unavailable',
source: source.kind,
reason: err.message,
code: err.code,
path: where,
}
}
return { status: 'failed', source: source.kind, reason: err.message, path: where }
}
const currentFacets = await db.getFacets().catch(() => [])
@@ -227,7 +291,8 @@ async function refresh({ force = false, approve = false, path: pathOverride = ''
if (removedFacets.length > 0 && !approve) {
const summary = {
hashes,
path: root,
source: source.kind,
path: where,
currentFacets,
incomingFacets,
removedFacets,
@@ -242,9 +307,16 @@ async function refresh({ force = false, approve = false, path: pathOverride = ''
try {
const counts = await applyAtlas(atlas)
return { status: 'imported', path: root, counts, addedFacets, removedFacets }
return {
status: 'imported',
source: source.kind,
path: where,
counts,
addedFacets,
removedFacets,
}
} catch (err) {
return { status: 'failed', reason: err.message, path: root }
return { status: 'failed', source: source.kind, reason: err.message, path: where }
}
}
@@ -267,7 +339,9 @@ async function rejectPending() {
/** Everything the admin panel needs to describe atlas state. */
async function status({ path: pathOverride = '' } = {}) {
const root = pathOverride.trim() !== '' ? pathOverride.trim() : await getServuoPath()
const source = await sourceFor(pathOverride)
const root = source.root
const configured = source.kind === 'bridge' || root !== ''
const [meta, pending, facets] = await Promise.all([
db.getMeta().catch(() => null),
db.getPending().catch(() => null),
@@ -276,9 +350,13 @@ async function status({ path: pathOverride = '' } = {}) {
let treeReadable = false
let drift = null
if (root !== '') {
if (configured) {
try {
const hashes = hashSources(root)
// On the bridge this is the MANIFEST, not the tree: 141 rows and ~32 KB,
// with no file bytes crossing the wire to answer "has anything changed".
// It is still a shard round trip on an admin page load, which is why it is
// here and not on the boot path (§17.7).
const hashes = await spawnAtlasSource.hashFrom(source)
treeReadable = true
const loaded = meta?.source
? Object.fromEntries(Object.entries(meta.source).map(([l, v]) => [l, v.sha256]))
@@ -292,8 +370,9 @@ async function status({ path: pathOverride = '' } = {}) {
}
return {
configured: root !== '',
path: root,
configured,
source: source.kind,
path: describe(source),
treeReadable,
drift,
facets,
@@ -309,6 +388,18 @@ async function status({ path: pathOverride = '' } = {}) {
*/
async function refreshOnBoot() {
try {
// **On the bridge it imports nothing**, deliberately, and by the same
// reasoning as the cliloc table (§17.7). A local tree hashes in ~120 ms and
// skips; asking the shard would put a sidecar round trip in the boot sequence
// to answer a question whose answer is "no" on every restart that did not
// follow a map edit — and editing spawn files is an operator action, so
// importing became one: Admin → Shard → Import. Whatever atlas is loaded
// keeps serving until then.
if ((await sourceFor()).kind === 'bridge') {
log.info('spawn atlas comes from the shard; import is admin-triggered (Admin → Shard)')
return { status: 'skipped', source: 'bridge', reason: 'the shard is the atlas source' }
}
const result = await refresh()
switch (result.status) {
case 'imported':