Files
Module-uo/server/test/assetBridge.test.js
wtclaude a194ec68e0
All checks were successful
PR Checks / client-build (pull_request) Successful in 18s
PR Checks / server-tests (pull_request) Successful in 24s
PR Checks / frozen-manifest (pull_request) Successful in 49s
feat(assets): creature artwork from the shard's own client (Phase 3)
Until now the only way a creature got a picture on this site was for an
operator to open UOFiddler on a desktop, export sprites by hand, copy them to
the web host and write a spawnAtlas.art.json naming each one. Almost nobody
did, so shard_spawn_creatures.art was NULL on every install.

The shard has had those files the whole time. Admin -> Shard -> Import now
walks its asset manifest, fetches only the sprites whose hash changed, writes
them under uploads/atlas/, asks the shard for a body id per atlas creature
(§8: it CONSTRUCTS the creature and reads Body.BodyID, which is the only thing
that is right for a shard's own custom creatures) and points each creature at
its picture. On a stock client that is 787 portraits, about a megabyte.

**The one thing v8.md §12 got wrong, and it is not cosmetic.** It says
`shard_spawn_creatures.art` "starts being filled by the import". That table is
emptied and refilled by replaceAtlas on EVERY atlas refresh, and a refresh runs
on every boot -- so a filename stored there would be destroyed by an ordinary
re-parse of the ServUO tree, with the next Update finding the client files
unchanged, reporting "nothing to do", and never restoring it. Nothing would
report a fault; the pictures would just be gone.

So the assets and the body map live in their own tables outside that blast
radius, and applyAtlas re-derives `art` on the way past as
`{ ...derived, ...operatorMap }` -- which is also the one place "the operator's
own artwork wins" is enforced, on every rebuild rather than only at import.

Smaller decisions worth not rediscovering:

- The derivation joins on the catalogue KEY, not on the body id. The simpler
  join is correct today and stops being correct the moment phase 6 adds
  body/400/a2/f0, at which point one slug matches dozens of rows.
- Filenames are content-addressed. A stable name overwritten in place leaves
  every browser and CDN serving the previous client's sprite, with the database
  row perfectly correct.
- An unchanged key whose FILE is missing is fetched again. The row and the disk
  can disagree (a wiped uploads volume, a restore from a dump), and a broken
  image on a creature page is worse than one re-fetched sprite.
- A key the shard cannot render is not a failure. Two thirds of the playable
  ghost and gargoyle bodies have no art on a stock client, and an import that
  reported eight failures every time would teach an operator to ignore the panel.
- A key that VANISHED from the manifest needs review before anything changes:
  an unmounted client volume and a deliberate downgrade look identical here.

23 new tests; 674 server and 42 client tests pass. The SQL was also run against
a real MariaDB, which is what proved the CONCAT join and the singleton CHECK.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-10 18:40:46 -05:00

369 lines
13 KiB
JavaScript

const { test } = require('node:test')
const assert = require('node:assert/strict')
const uoLinkClient = require('../utils/uoLinkClient')
const bridge = require('../utils/assetBridge')
// The three walks over the asset plane, driven against a stubbed sidecar client
// (docs/link/v8.md §5, §6, §8 — protocol 8, phase 3).
//
// Two families of failure are asserted here and they are not the same shape.
//
// **The envelope failures** are ways the shard can be wrong that leave this side
// holding a catalogue it believes is complete. They are invisible downstream: a
// catalogue missing its last three hundred bodies renders as a site where some
// creatures have pictures and some do not, which is exactly what NO catalogue
// looks like. Each corresponds to a field §3.4 puts on the wire specifically so
// this side can tell the difference.
//
// **The absence failures** are the opposite mistake, and phase 3's more likely
// one: treating a body this client has no art for as an error. Two thirds of the
// playable ghost and gargoyle bodies are in that state on a stock client, and an
// import that failed — or even warned loudly — on them would teach an operator to
// ignore the panel.
const saved = {}
function stub({ sources, manifest = [], fetch = [], bodies = [] } = {}) {
saved.getAssetSources = uoLinkClient.getAssetSources
saved.getAssetManifest = uoLinkClient.getAssetManifest
saved.fetchAssets = uoLinkClient.fetchAssets
saved.resolveBodies = uoLinkClient.resolveBodies
const calls = { manifest: [], fetch: [], bodies: [] }
uoLinkClient.getAssetSources = async () => sources
uoLinkClient.getAssetManifest = async ({ family, cursor } = {}) => {
calls.manifest.push({ family: family ?? null, cursor: cursor ?? null })
const next = manifest.shift()
if (!next) throw new Error('the walk asked for more manifest pages than the test supplied')
return next
}
uoLinkClient.fetchAssets = async ({ keys, catalog, cursor } = {}) => {
calls.fetch.push({ keys, catalog: catalog ?? null, cursor: cursor ?? null })
const next = fetch.shift()
if (!next) throw new Error('the walk asked for more fetch pages than the test supplied')
return next
}
uoLinkClient.resolveBodies = async (types) => {
calls.bodies.push(types)
const next = bodies.shift()
if (!next) throw new Error('the walk asked for more body chunks than the test supplied')
return next
}
return calls
}
function restore() {
for (const [name, fn] of Object.entries(saved)) {
if (fn) uoLinkClient[name] = fn
}
}
const ok = (data) => ({ ok: true, status: 200, data })
const fail = (status, data) => ({ ok: false, status, data })
const CATALOG = 'a3f9c21d4b8e0771'
const manifestPage = (rows, extra = {}) =>
ok({
kind: 'assets.manifest.ok',
family: 'body',
catalog: CATALOG,
extractorVersion: 1,
playerBodies: [400, 401, 402, 403],
scanned: rows.length,
rows,
more: false,
cut: 'end',
...extra,
})
const fetchPage = (rows, extra = {}) =>
ok({
kind: 'assets.fetch.ok',
family: 'body',
catalog: CATALOG,
rows,
more: false,
cut: 'end',
...extra,
})
const row = (body, sha = 'aa') => ({
key: `body/${body}/a0`,
sha256: sha,
bytes: 900,
width: 24,
height: 63,
body,
direction: 1,
})
const png = Buffer.from([0x89, 0x50, 0x4e, 0x47]).toString('base64')
const sourcesReply = (extra = {}) =>
ok({
kind: 'assets.sources.ok',
extractorVersion: 1,
imaging: { ok: true },
hashing: false,
complete: true,
files: [
{ name: 'anim.idx', size: 10, mtime: 1, sha256: 'a' },
{ name: 'anim.mul', size: 20, mtime: 2, sha256: 'b' },
{ name: 'body.def', size: 30, mtime: 3, sha256: 'c' },
// Not a source this family reads: `art.mul` decides item pictures, not
// creature ones, and folding it in would make every item-art change look
// like a reason to re-import the whole body catalogue.
{ name: 'art.mul', size: 148000000, mtime: 4, sha256: 'd' },
],
...extra,
})
// ── the source gate (§6 stage 1) ──────────────────────────────────────────
test('the source fingerprint keeps only the files the body catalogue reads', async (t) => {
stub({ sources: sourcesReply() })
t.after(restore)
const fingerprint = await bridge.sourceFingerprint()
assert.deepEqual(Object.keys(fingerprint.files).sort(), ['anim.idx', 'anim.mul', 'body.def'])
assert.equal(fingerprint.extractorVersion, 1)
})
test('a bumped extractor version is drift even when every client file is identical', () => {
const files = { 'anim.mul': { size: 1, mtime: 2, sha256: 'x' } }
assert.equal(
bridge.sameSources({ files, extractorVersion: 1 }, { files, extractorVersion: 1 }),
true,
)
// §7: a corrected frame offset changes every derived byte while every source
// file stays byte-identical. If this returned true the fix would never reach
// an install whose client never moves.
assert.equal(
bridge.sameSources({ files, extractorVersion: 2 }, { files, extractorVersion: 1 }),
false,
)
})
test('a client that GAINED an anim file is drift, not a match', () => {
const before = { files: { 'anim.mul': { size: 1, mtime: 2, sha256: 'x' } }, extractorVersion: 1 }
const after = {
files: {
'anim.mul': { size: 1, mtime: 2, sha256: 'x' },
// A client that grows an anim5.mul is a client whose gargoyles suddenly
// resolve. Comparing only the files present in both would call that
// unchanged and never import them.
'anim5.mul': { size: 9, mtime: 9, sha256: 'y' },
},
extractorVersion: 1,
}
assert.equal(bridge.sameSources(before, after), false)
})
test('a null hash falls back to size and mtime rather than reading as changed', () => {
// The shard hashes 195 MB anim files off the request path, so a null sha256 is
// "not computed yet". Treating it as a difference would re-import the whole
// catalogue on every restart until the background pass finished.
const a = { files: { 'anim.mul': { size: 5, mtime: 7, sha256: null } }, extractorVersion: 1 }
const b = { files: { 'anim.mul': { size: 5, mtime: 7, sha256: 'later' } }, extractorVersion: 1 }
assert.equal(bridge.sameSources(a, b), true)
})
// ── the manifest walk (§6 stage 2) ────────────────────────────────────────
test('the manifest walks every page and stops only on cut: end', async (t) => {
const calls = stub({
manifest: [
manifestPage([row(12), row(34)], { more: true, cursor: 'b:34', cut: 'limit' }),
manifestPage([row(400)]),
],
})
t.after(restore)
const result = await bridge.readManifest()
assert.equal(result.rows.length, 3)
assert.equal(result.catalog, CATALOG)
assert.deepEqual(result.playerBodies, [400, 401, 402, 403])
assert.deepEqual(
calls.manifest.map((c) => c.cursor),
[null, 'b:34'],
)
})
test('a short page that did not end the catalogue is refused', async (t) => {
// `cut: 'limit'` with `more: false` is the shard saying it stopped for its own
// reason. Importing what arrived would silently drop every body after it, and
// the result is indistinguishable from a client with fewer creatures.
stub({ manifest: [manifestPage([row(12)], { more: false, cut: 'limit' })] })
t.after(restore)
await assert.rejects(() => bridge.readManifest(), /stopped sending assets/)
})
test('a cursor that does not advance is refused rather than looped on', async (t) => {
stub({
manifest: [
manifestPage([row(12)], { more: true, cursor: 'b:12', cut: 'budget' }),
manifestPage([row(13)], { more: true, cursor: 'b:12', cut: 'budget' }),
],
})
t.after(restore)
await assert.rejects(() => bridge.readManifest(), /without advancing its cursor/)
})
test('the client files changing mid-walk aborts the whole import', async (t) => {
// The catalogue id is derived from the client files themselves, so a change
// between two pages means half of what we hold describes files that no longer
// exist — and nothing later can tell which half.
stub({
manifest: [
manifestPage([row(12)], { more: true, cursor: 'b:12', cut: 'limit' }),
manifestPage([row(34)], { catalog: 'something-else' }),
],
})
t.after(restore)
await assert.rejects(() => bridge.readManifest(), /changed while the manifest was being read/)
})
// ── the fetch (§5) ────────────────────────────────────────────────────────
test('a fetch passes the catalogue id and decodes the PNG', async (t) => {
const calls = stub({
fetch: [fetchPage([{ key: 'body/12/a0', status: 'ok', sha256: 'aa', bytes: 4, width: 24, height: 63, body: 12, direction: 1, png }])],
})
t.after(restore)
const { assets } = await bridge.fetchAssets({ keys: ['body/12/a0'], catalog: CATALOG })
assert.equal(calls.fetch[0].catalog, CATALOG)
assert.equal(assets.get('body/12/a0').png.length, 4)
assert.equal(assets.get('body/12/a0').width, 24)
})
test('an absent asset is a counted row, not a failed fetch', async (t) => {
// The whole reason this is not an error: two thirds of the playable ghost and
// gargoyle bodies have no art on a stock client (§5.2), and an import that
// failed on them could never succeed.
stub({
fetch: [
fetchPage([
{ key: 'body/12/a0', status: 'ok', sha256: 'aa', bytes: 4, png },
{ key: 'body/666/a0', status: 'absent' },
{ key: 'body/400/a2/f3', status: 'unsupported' },
]),
],
})
t.after(restore)
const { assets, missing } = await bridge.fetchAssets({
keys: ['body/12/a0', 'body/666/a0', 'body/400/a2/f3'],
catalog: CATALOG,
})
assert.equal(assets.size, 1)
// Counted apart, because they mean different things: `absent` is a gap in the
// operator's client and `unsupported` is a bug on this side.
assert.equal(missing.absent, 1)
assert.equal(missing.unsupported, 1)
})
test('a busy shard is retried rather than failing the walk', async (t) => {
saved.fetchAssets = uoLinkClient.fetchAssets
t.after(restore)
let attempts = 0
uoLinkClient.fetchAssets = async () => {
attempts++
if (attempts < 3) return fail(425, { reason: 'busy' })
return fetchPage([{ key: 'body/12/a0', status: 'ok', sha256: 'aa', bytes: 4, png }])
}
const { assets } = await bridge.fetchAssets({ keys: ['body/12/a0'], catalog: CATALOG })
assert.equal(attempts, 3)
assert.equal(assets.size, 1)
})
test('a shard host with no libgdiplus is named, not reported as a dead shard', async (t) => {
saved.getAssetManifest = uoLinkClient.getAssetManifest
t.after(restore)
uoLinkClient.getAssetManifest = async () =>
fail(503, { reason: "this shard host cannot render images - Mono's System.Drawing needs libgdiplus" })
await assert.rejects(
() => bridge.readManifest(),
(err) => err.code === 'NO_IMAGING',
)
})
// ── the body pass (§8) ────────────────────────────────────────────────────
test('body resolution chunks to the shard cap and records every outcome', async (t) => {
const creatures = []
for (let i = 0; i < bridge.BODY_CHUNK + 5; i++) {
creatures.push({ slug: `c-${i}`, name: `Creature${i}` })
}
const reply = (types) =>
ok({
kind: 'assets.bodies.ok',
rows: types.map((type, i) => (i === 0 ? { type, status: 'unknown' } : { type, status: 'ok', body: 100 + i })),
more: false,
cut: 'end',
})
const calls = stub({ bodies: [] })
t.after(restore)
uoLinkClient.resolveBodies = async (types) => {
calls.bodies.push(types)
return reply(types)
}
const rows = await bridge.resolveBodies({ creatures })
// Two chunks, and neither over the cap: the shard REFUSES an over-long list
// rather than truncating it, so a chunk size above its cap does not degrade —
// every request fails.
assert.equal(calls.bodies.length, 2)
assert.ok(calls.bodies.every((chunk) => chunk.length <= bridge.BODY_CHUNK))
assert.equal(rows.length, creatures.length)
// The negative answers are kept. Without them the next pass asks again, and
// the pass costs a real constructor per name on the shard's Core thread.
assert.equal(rows.filter((r) => r.status === 'unknown').length, 2)
})
test('two slugs sharing a class name are asked once and both get the answer', async (t) => {
const calls = stub({
bodies: [
ok({ kind: 'assets.bodies.ok', rows: [{ type: 'GiantSpider', status: 'ok', body: 28 }], more: false, cut: 'end' }),
],
})
t.after(restore)
const rows = await bridge.resolveBodies({
creatures: [
{ slug: 'giant-spider', name: 'GiantSpider' },
{ slug: 'giantspider', name: 'GiantSpider' },
],
})
assert.deepEqual(calls.bodies[0], ['GiantSpider'])
assert.equal(rows.length, 2)
assert.ok(rows.every((r) => r.body === 28))
})