Files
Module-uo/server/test/assetBridge.test.js
Claude d4d5989926
All checks were successful
PR Checks / server-tests (pull_request) Successful in 23s
PR Checks / frozen-manifest (pull_request) Successful in 1m8s
PR Checks / client-build (pull_request) Successful in 7m59s
feat(atlas): the spawn atlas reads the shard, not the shard's filesystem (Phase 7)
`spawnAtlasSource.js` gains a second backend behind its existing interface
(docs/link/v8.md 10). Where a shard is linked and enabled the tree arrives over
the sidecar; where there is none, a local ServUO tree is read exactly as before.
An explicit --servuo path is an instruction and overrules both.

The parsers do not move. spawnAtlasParse.js is still pure, still fs-free and
still CI-covered without a ServUO tree anywhere near it; `buildFromFiles` is now
where the parse starts, and both readers feed it the same shape.

treeBridge.js walks the manifest and then the chunks. Three of its checks are
not decoration -- each is a way this ends in a tree that LOOKS imported, and XML
is forgiving enough that a mis-assembled spawn file parses cleanly and simply
has fewer spawns in it:

  - every chunk re-declares its address and carries the hash of its own
    uncompressed bytes, and chunks are placed by declared index rather than
    arrival order
  - the whole file is hashed after reassembly against its manifest row
  - the catalog must not move mid-walk, or the import is refused rather than
    stitched out of two trees

Boot does not call the shard. The same answer 17.7 gave the cliloc table, and
the same reasoning: a local tree hashes in ~120 ms and skips, while a round trip
in the boot sequence would answer "no" on every restart that did not follow a
map edit. Editing spawn files is an operator action, so importing is one --
Admin -> Spawn Atlas -> Import. What that costs is real and is said out loud in
the panel, the CLI and the log: an install on the bridge has NO automatic
refresh at all.

Two things the live walk found that the unit tests could not:

  - PARSER_VERSION 4 -> 5. The parse is order-sensitive in one place -- the
    decoration index keeps the FIRST item id it sees for a type -- and the two
    readers agreed on a stock tree by coincidence, since the filesystem reader
    walks each directory with localeCompare while the shard sorts whole relative
    paths. buildFromFiles now sorts by label, ordinally, once, whatever order
    the files arrived in. Identical input, a different answer for a handful of
    types: exactly what the version number exists to push through the hash gate.
    The parity test asserted deepEqual, which ignores key order; it now asserts
    serialised equality too.
  - The source fingerprint is taken over RAW BYTES at both ends. Hashing decoded
    text hashes a UTF-8 re-encoding -- identical for valid UTF-8, different for a
    file that is not, because an undecodable byte becomes U+FFFD and never comes
    back. One Latin-1 character in a creature name would have made the drift gate
    report a change on every import, forever, with the tree untouched.

A 200 from assets.sources also stopped meaning "the client files are on offer":
a shard may now serve its configuration tree while declining to serve its UO
client. Both client-file readers check `assetsEnabled` and say DISABLED, instead
of reading an empty file list as "your client has no cliloc.enu" and sending an
operator to their client install for a setting that lives on their shard.

Measured end to end against a live shard and the real sidecar: 141 files,
11.9 MB, 158 chunks, 3 pages, 1.33 MB on the wire, 512 ms; every file
byte-identical to disk; and the atlas built over the bridge identical to the one
built off it -- 6,455 points, 800 creatures, 387 regions, 558 landmarks,
25 champions, 309 decoration types.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 02:00:43 -05:00

408 lines
14 KiB
JavaScript

const { test } = require('node:test')
const assert = require('node:assert/strict')
const uoLinkClient = require('../utils/uoLinkClient')
const bridge = require('../utils/assetBridge')
// The three walks over the asset plane, driven against a stubbed sidecar client
// (docs/link/v8.md §5, §6, §8 — protocol 8, phase 3).
//
// Two families of failure are asserted here and they are not the same shape.
//
// **The envelope failures** are ways the shard can be wrong that leave this side
// holding a catalogue it believes is complete. They are invisible downstream: a
// catalogue missing its last three hundred bodies renders as a site where some
// creatures have pictures and some do not, which is exactly what NO catalogue
// looks like. Each corresponds to a field §3.4 puts on the wire specifically so
// this side can tell the difference.
//
// **The absence failures** are the opposite mistake, and phase 3's more likely
// one: treating a body this client has no art for as an error. Two thirds of the
// playable ghost and gargoyle bodies are in that state on a stock client, and an
// import that failed — or even warned loudly — on them would teach an operator to
// ignore the panel.
const saved = {}
function stub({ sources, manifest = [], fetch = [], bodies = [] } = {}) {
saved.getAssetSources = uoLinkClient.getAssetSources
saved.getAssetManifest = uoLinkClient.getAssetManifest
saved.fetchAssets = uoLinkClient.fetchAssets
saved.resolveBodies = uoLinkClient.resolveBodies
const calls = { manifest: [], fetch: [], bodies: [] }
uoLinkClient.getAssetSources = async () => sources
uoLinkClient.getAssetManifest = async ({ family, cursor } = {}) => {
calls.manifest.push({ family: family ?? null, cursor: cursor ?? null })
const next = manifest.shift()
if (!next) throw new Error('the walk asked for more manifest pages than the test supplied')
return next
}
uoLinkClient.fetchAssets = async ({ keys, catalog, cursor } = {}) => {
calls.fetch.push({ keys, catalog: catalog ?? null, cursor: cursor ?? null })
const next = fetch.shift()
if (!next) throw new Error('the walk asked for more fetch pages than the test supplied')
return next
}
uoLinkClient.resolveBodies = async (types) => {
calls.bodies.push(types)
const next = bodies.shift()
if (!next) throw new Error('the walk asked for more body chunks than the test supplied')
return next
}
return calls
}
function restore() {
for (const [name, fn] of Object.entries(saved)) {
if (fn) uoLinkClient[name] = fn
}
}
const ok = (data) => ({ ok: true, status: 200, data })
const fail = (status, data) => ({ ok: false, status, data })
const CATALOG = 'a3f9c21d4b8e0771'
const manifestPage = (rows, extra = {}) =>
ok({
kind: 'assets.manifest.ok',
family: 'body',
catalog: CATALOG,
extractorVersion: 1,
playerBodies: [400, 401, 402, 403],
scanned: rows.length,
rows,
more: false,
cut: 'end',
...extra,
})
const fetchPage = (rows, extra = {}) =>
ok({
kind: 'assets.fetch.ok',
family: 'body',
catalog: CATALOG,
rows,
more: false,
cut: 'end',
...extra,
})
const row = (body, sha = 'aa') => ({
key: `body/${body}/a0`,
sha256: sha,
bytes: 900,
width: 24,
height: 63,
body,
direction: 1,
})
const png = Buffer.from([0x89, 0x50, 0x4e, 0x47]).toString('base64')
const sourcesReply = (extra = {}) =>
ok({
kind: 'assets.sources.ok',
extractorVersion: 1,
imaging: { ok: true },
hashing: false,
complete: true,
files: [
{ name: 'anim.idx', size: 10, mtime: 1, sha256: 'a' },
{ name: 'anim.mul', size: 20, mtime: 2, sha256: 'b' },
{ name: 'body.def', size: 30, mtime: 3, sha256: 'c' },
// Not a source this family reads: `art.mul` decides item pictures, not
// creature ones, and folding it in would make every item-art change look
// like a reason to re-import the whole body catalogue.
{ name: 'art.mul', size: 148000000, mtime: 4, sha256: 'd' },
],
...extra,
})
// ── the source gate (§6 stage 1) ──────────────────────────────────────────
test('the source fingerprint keeps only the files the body catalogue reads', async (t) => {
stub({ sources: sourcesReply() })
t.after(restore)
const fingerprint = await bridge.sourceFingerprint()
assert.deepEqual(Object.keys(fingerprint.files).sort(), ['anim.idx', 'anim.mul', 'body.def'])
assert.equal(fingerprint.extractorVersion, 1)
})
test('a bumped extractor version is drift even when every client file is identical', () => {
const files = { 'anim.mul': { size: 1, mtime: 2, sha256: 'x' } }
assert.equal(
bridge.sameSources({ files, extractorVersion: 1 }, { files, extractorVersion: 1 }),
true,
)
// §7: a corrected frame offset changes every derived byte while every source
// file stays byte-identical. If this returned true the fix would never reach
// an install whose client never moves.
assert.equal(
bridge.sameSources({ files, extractorVersion: 2 }, { files, extractorVersion: 1 }),
false,
)
})
test('a client that GAINED an anim file is drift, not a match', () => {
const before = { files: { 'anim.mul': { size: 1, mtime: 2, sha256: 'x' } }, extractorVersion: 1 }
const after = {
files: {
'anim.mul': { size: 1, mtime: 2, sha256: 'x' },
// A client that grows an anim5.mul is a client whose gargoyles suddenly
// resolve. Comparing only the files present in both would call that
// unchanged and never import them.
'anim5.mul': { size: 9, mtime: 9, sha256: 'y' },
},
extractorVersion: 1,
}
assert.equal(bridge.sameSources(before, after), false)
})
test('a null hash falls back to size and mtime rather than reading as changed', () => {
// The shard hashes 195 MB anim files off the request path, so a null sha256 is
// "not computed yet". Treating it as a difference would re-import the whole
// catalogue on every restart until the background pass finished.
const a = { files: { 'anim.mul': { size: 5, mtime: 7, sha256: null } }, extractorVersion: 1 }
const b = { files: { 'anim.mul': { size: 5, mtime: 7, sha256: 'later' } }, extractorVersion: 1 }
assert.equal(bridge.sameSources(a, b), true)
})
// ── the manifest walk (§6 stage 2) ────────────────────────────────────────
test('the manifest walks every page and stops only on cut: end', async (t) => {
const calls = stub({
manifest: [
manifestPage([row(12), row(34)], { more: true, cursor: 'b:34', cut: 'limit' }),
manifestPage([row(400)]),
],
})
t.after(restore)
const result = await bridge.readManifest()
assert.equal(result.rows.length, 3)
assert.equal(result.catalog, CATALOG)
assert.deepEqual(result.playerBodies, [400, 401, 402, 403])
assert.deepEqual(
calls.manifest.map((c) => c.cursor),
[null, 'b:34'],
)
})
test('a short page that did not end the catalogue is refused', async (t) => {
// `cut: 'limit'` with `more: false` is the shard saying it stopped for its own
// reason. Importing what arrived would silently drop every body after it, and
// the result is indistinguishable from a client with fewer creatures.
stub({ manifest: [manifestPage([row(12)], { more: false, cut: 'limit' })] })
t.after(restore)
await assert.rejects(() => bridge.readManifest(), /stopped sending asset rows/)
})
test('a cursor that does not advance is refused rather than looped on', async (t) => {
stub({
manifest: [
manifestPage([row(12)], { more: true, cursor: 'b:12', cut: 'budget' }),
manifestPage([row(13)], { more: true, cursor: 'b:12', cut: 'budget' }),
],
})
t.after(restore)
await assert.rejects(() => bridge.readManifest(), /without advancing its cursor/)
})
test('the client files changing mid-walk aborts the whole import', async (t) => {
// The catalogue id is derived from the client files themselves, so a change
// between two pages means half of what we hold describes files that no longer
// exist — and nothing later can tell which half.
stub({
manifest: [
manifestPage([row(12)], { more: true, cursor: 'b:12', cut: 'limit' }),
manifestPage([row(34)], { catalog: 'something-else' }),
],
})
t.after(restore)
await assert.rejects(() => bridge.readManifest(), /changed while the manifest was being read/)
})
// ── the fetch (§5) ────────────────────────────────────────────────────────
test('a fetch passes the catalogue id and decodes the PNG', async (t) => {
const calls = stub({
fetch: [fetchPage([{ key: 'body/12/a0', status: 'ok', sha256: 'aa', bytes: 4, width: 24, height: 63, body: 12, direction: 1, png }])],
})
t.after(restore)
const { assets } = await bridge.fetchAssets({ keys: ['body/12/a0'], catalog: CATALOG })
assert.equal(calls.fetch[0].catalog, CATALOG)
assert.equal(assets.get('body/12/a0').png.length, 4)
assert.equal(assets.get('body/12/a0').width, 24)
})
test('a body catalogued at a later action keeps that action in its row', async (t) => {
// §11.2, phase 6. 73 of a stock client's bodies have no art at action 0 and are
// catalogued at the first action that does — body 820's is 23, and it is a
// horse. The action travels with the row because the atlas join needs it in
// SQL; re-deriving it from the key would put a second parser of §5's scheme in
// the schema.
stub({
manifest: [
manifestPage([
{ ...row(12), action: 0 },
{ key: 'body/820/a23', sha256: 'bb', bytes: 900, width: 68, height: 69, body: 820, action: 23, direction: 1 },
]),
],
})
t.after(restore)
const { rows } = await bridge.readManifest({})
assert.deepEqual(
rows.map((r) => [r.key, r.action]),
[
['body/12/a0', 0],
['body/820/a23', 23],
],
)
})
test('an overlay older than phase 6 reads as action 0 rather than as unknown', async (t) => {
// A phase-3 through phase-5 overlay omits `action` entirely, and every key it
// ever produced ended in `a0`. Reading that as null would make the atlas join
// COALESCE it back to 0 anyway; reading it as 0 here says so once.
stub({ manifest: [manifestPage([row(12)])] })
t.after(restore)
const { rows } = await bridge.readManifest({})
assert.equal(rows[0].action, 0)
})
test('an absent asset is a counted row, not a failed fetch', async (t) => {
// The whole reason this is not an error: two thirds of the playable ghost and
// gargoyle bodies have no art on a stock client (§5.2), and an import that
// failed on them could never succeed.
stub({
fetch: [
fetchPage([
{ key: 'body/12/a0', status: 'ok', sha256: 'aa', bytes: 4, png },
{ key: 'body/666/a0', status: 'absent' },
{ key: 'body/400/a2/f3', status: 'unsupported' },
]),
],
})
t.after(restore)
const { assets, missing } = await bridge.fetchAssets({
keys: ['body/12/a0', 'body/666/a0', 'body/400/a2/f3'],
catalog: CATALOG,
})
assert.equal(assets.size, 1)
// Counted apart, because they mean different things: `absent` is a gap in the
// operator's client and `unsupported` is a bug on this side.
assert.equal(missing.absent, 1)
assert.equal(missing.unsupported, 1)
})
test('a busy shard is retried rather than failing the walk', async (t) => {
saved.fetchAssets = uoLinkClient.fetchAssets
t.after(restore)
let attempts = 0
uoLinkClient.fetchAssets = async () => {
attempts++
if (attempts < 3) return fail(425, { reason: 'busy' })
return fetchPage([{ key: 'body/12/a0', status: 'ok', sha256: 'aa', bytes: 4, png }])
}
const { assets } = await bridge.fetchAssets({ keys: ['body/12/a0'], catalog: CATALOG })
assert.equal(attempts, 3)
assert.equal(assets.size, 1)
})
test('a shard host with no libgdiplus is named, not reported as a dead shard', async (t) => {
saved.getAssetManifest = uoLinkClient.getAssetManifest
t.after(restore)
uoLinkClient.getAssetManifest = async () =>
fail(503, { reason: "this shard host cannot render images - Mono's System.Drawing needs libgdiplus" })
await assert.rejects(
() => bridge.readManifest(),
(err) => err.code === 'NO_IMAGING',
)
})
// ── the body pass (§8) ────────────────────────────────────────────────────
test('body resolution chunks to the shard cap and records every outcome', async (t) => {
const creatures = []
for (let i = 0; i < bridge.BODY_CHUNK + 5; i++) {
creatures.push({ slug: `c-${i}`, name: `Creature${i}` })
}
const reply = (types) =>
ok({
kind: 'assets.bodies.ok',
rows: types.map((type, i) => (i === 0 ? { type, status: 'unknown' } : { type, status: 'ok', body: 100 + i })),
more: false,
cut: 'end',
})
const calls = stub({ bodies: [] })
t.after(restore)
uoLinkClient.resolveBodies = async (types) => {
calls.bodies.push(types)
return reply(types)
}
const rows = await bridge.resolveBodies({ creatures })
// Two chunks, and neither over the cap: the shard REFUSES an over-long list
// rather than truncating it, so a chunk size above its cap does not degrade —
// every request fails.
assert.equal(calls.bodies.length, 2)
assert.ok(calls.bodies.every((chunk) => chunk.length <= bridge.BODY_CHUNK))
assert.equal(rows.length, creatures.length)
// The negative answers are kept. Without them the next pass asks again, and
// the pass costs a real constructor per name on the shard's Core thread.
assert.equal(rows.filter((r) => r.status === 'unknown').length, 2)
})
test('two slugs sharing a class name are asked once and both get the answer', async (t) => {
const calls = stub({
bodies: [
ok({ kind: 'assets.bodies.ok', rows: [{ type: 'GiantSpider', status: 'ok', body: 28 }], more: false, cut: 'end' }),
],
})
t.after(restore)
const rows = await bridge.resolveBodies({
creatures: [
{ slug: 'giant-spider', name: 'GiantSpider' },
{ slug: 'giantspider', name: 'GiantSpider' },
],
})
assert.deepEqual(calls.bodies[0], ['GiantSpider'])
assert.equal(rows.length, 2)
assert.ok(rows.every((r) => r.body === 28))
})