feat(assets): item pictures on the marketplace and the character sheet (Phase 5)

Both places this site already knew an item's (ItemID, hue) and could only print
it as text now show the picture, hued the way the client would draw it. The
shard does the hueing: whether a hue repaints every pixel or only the grey ones
is a flag in `tiledata.mul`, which a browser has no way to read.

**Ingest warms; the route only serves** (org lead, 2026-09-11). A page never
waits on the shard and never causes a fetch -- it renders what is stored and
leaves out what is not, which is the state every install was in before this
phase. Fetching happens behind that, on a timer, from the keys the site's own
rows name. The alternative, fetching on first request, was rejected on one
number: the shard's asset plane serves ONE request at a time, so a URL that
fetched would let any visitor walk 49,152 ids times 3,000 hues through that slot
and park an operator's own import behind it.

The wanted set is DERIVED (`SELECT DISTINCT item_id, hue`) rather than queued, so
it is self-healing: a restart loses nothing, and a key stops being wanted the
moment the vendor row naming it is deleted. The in-memory hint set on top is only
for the character sheet, which is fetched live from the shard and stored nowhere
-- nothing on disk would ever name those keys.

Staleness without a manifest (§7): every row records the shard's `catalog` id, a
hash of the files that decide its bytes. A client patch changes it and a restart
does not, so "is this out of date?" is a per-row question -- and pictures nobody
looks at any more are simply never re-fetched, which is why this is lazy rather
than a sweep. `shard_asset_meta` is deliberately NOT written here: it is the body
catalogue's singleton, and a warm pass touching it would tell the body import
that a client it never looked at is unchanged.

A key the shard has no art for writes no row at all. An empty row would make the
key held and it would never be asked again -- including after the operator
patches in the graphic that was missing.

`assets.sources` now reports which families an overlay serves, so an overlay
older than phase 5 is one reported state with a sentence naming the fix, instead
of a refusal per pass forever with no picture ever appearing.

688 server tests pass (14 new); client builds; the frozen manifest regenerates
with one added route, all documented, no core URL moved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4

server/test/shardItemArt.model.test.js

@@ -0,0 +1,414 @@
+const { test } = require('node:test')
+const assert = require('node:assert/strict')
+const fs = require('node:fs')
+const os = require('node:os')
+const path = require('node:path')
+
+const core = require('../core')
+const model = require('../model/shardAssets/shardItemArt.model')
+const db = require('../model/shardAssets/shardAssets.db')
+const bridge = require('../utils/assetBridge')
+const uoLinkConfig = require('../model/uoLinkConfig/uoLinkConfig.model')
+
+// The warm pass as a decision, with the shard and the database stubbed
+// (docs/link/v8.md §5, §11 — protocol 8, phase 5).
+//
+// Item art has no manifest, so almost everything the body import gets from a
+// hash diff this side has to get right by construction instead. Each test below
+// is a way that goes wrong quietly:
+//
+//   - Asking an overlay that cannot answer. A phase-4 plugin serves the creature
+//     catalogue and nothing else, and every static key it is sent is refused —
+//     once per pass, forever, in the log, with no picture ever appearing.
+//   - Re-fetching pictures the site already holds. There is no manifest to make
+//     that obvious, so the only thing standing between a working install and a
+//     pass that re-downloads its whole working set every five minutes is the
+//     per-row catalogue id.
+//   - NOT re-fetching after a client patch. The same field, read the other way.
+//   - Writing a row for a key the shard has no art for. It would make the key
+//     "held", and it would never be asked again — including after the operator
+//     patches in the graphic that was missing.
+//   - Spelling `static/3922/h0`. The shard refuses it outright (hue 0 means "not
+//     hued"), so a disagreement here is a picture that never arrives.
+
+const saved = {}
+
+function stub({
+  families = ['body', 'land', 'static'],
+  wanted = [],
+  fresh = new Set(),
+  files = new Map(),
+  fetched,
+  catalog = 'cat-current',
+  linked = true,
+} = {}) {
+  saved.sourceFingerprint = bridge.sourceFingerprint
+  saved.fetchAssets = bridge.fetchAssets
+  saved.freshKeys = db.freshKeys
+  saved.filesForKeys = db.filesForKeys
+  saved.saveAssets = db.saveAssets
+  saved.getSafe = uoLinkConfig.getSafe
+  saved.query = core.query
+
+  const seen = { asked: [], saved: null, freshAsked: null, calls: 0 }
+
+  uoLinkConfig.getSafe = async () =>
+    linked ? { enabled: true, baseUrl: 'http://127.0.0.1:8080' } : { enabled: false }
+
+  bridge.sourceFingerprint = async () => ({
+    files: { 'art.mul': { size: 1, mtime: 2, sha256: 'x' } },
+    extractorVersion: 2,
+    hashing: false,
+    complete: true,
+    imaging: { ok: true },
+    families,
+  })
+
+  bridge.fetchAssets = async ({ keys }) => {
+    seen.calls++
+    seen.asked.push(keys)
+
+    // The catalogue probe asks for exactly one key and throws the answer away.
+    if (keys.length === 1 && keys[0] === 'static/0' && !fetched?.assets?.has('static/0')) {
+      return { assets: new Map(), missing: { absent: 1, unsupported: 0 }, pages: 1, catalog }
+    }
+
+    return (
+      fetched ?? { assets: new Map(), missing: { absent: 0, unsupported: 0 }, pages: 1, catalog }
+    )
+  }
+
+  // The derived set: what `SELECT DISTINCT item_id, hue FROM shard_vendor_items`
+  // would return.
+  core.query = async () => wanted
+
+  db.freshKeys = async (keys, askedCatalog) => {
+    seen.freshAsked = { keys, catalog: askedCatalog }
+    return fresh
+  }
+  db.filesForKeys = async () => files
+  db.saveAssets = async (rows, meta) => {
+    seen.saved = { rows, meta }
+    return rows.length
+  }
+
+  return seen
+}
+
+function restore() {
+  if (saved.sourceFingerprint) bridge.sourceFingerprint = saved.sourceFingerprint
+  if (saved.fetchAssets) bridge.fetchAssets = saved.fetchAssets
+  if (saved.freshKeys) db.freshKeys = saved.freshKeys
+  if (saved.filesForKeys) db.filesForKeys = saved.filesForKeys
+  if (saved.saveAssets) db.saveAssets = saved.saveAssets
+  if (saved.getSafe) uoLinkConfig.getSafe = saved.getSafe
+  if (saved.query) core.query = saved.query
+}
+
+function useTempUploads(t) {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'uo-items-'))
+  const previous = core.uploads
+
+  Object.defineProperty(core, 'uploads', {
+    configurable: true,
+    get: () => ({ ...previous, UPLOAD_DIR: dir }),
+  })
+
+  t.after(() => {
+    Object.defineProperty(core, 'uploads', { configurable: true, get: () => previous })
+    fs.rmSync(dir, { recursive: true, force: true })
+  })
+
+  return dir
+}
+
+const picture = (sha) => ({
+  sha256: sha,
+  bytes: 294,
+  width: 22,
+  height: 26,
+  hue: null,
+  partialHue: null,
+  source: 'uop',
+  png: Buffer.from('not really a png'),
+})
+
+// ── keys ───────────────────────────────────────────────────────────────────
+
+test('hue 0 is the plain key, because the shard refuses /h0 for the same reason', () => {
+  // The wire's hue 0 means "this item is not hued". If this spelled `/h0` the
+  // shard would answer `unsupported` and the picture would never arrive; if the
+  // shard accepted it, the identical PNG would be stored twice under two names
+  // and diffed separately forever. The two sides agreeing is the whole point.
+  assert.equal(model.staticKey(3922, 0), 'static/3922')
+  assert.equal(model.staticKey(3922), 'static/3922')
+  assert.equal(model.staticKey(3922, null), 'static/3922')
+  assert.equal(model.staticKey(3922, 33), 'static/3922/h33')
+})
+
+test('a key is refused rather than fabricated for input that is not an item id', () => {
+  assert.equal(model.staticKey(-5), null)
+  assert.equal(model.staticKey('frog'), null)
+  assert.equal(model.staticKey(undefined), null)
+  assert.equal(model.landKey(0x4000), null)
+  assert.equal(model.landKey(3), 'land/3')
+})
+
+// ── the overlay gate ───────────────────────────────────────────────────────
+
+test('an overlay that serves only the creature catalogue is reported, not asked', async (t) => {
+  // A phase-3 or phase-4 plugin. Every static key sent to it comes back refused,
+  // so discovering this per request would mean a warn per pass forever and no
+  // picture ever. It is one check, once, with a sentence naming the fix.
+  const seen = stub({ families: ['body'], wanted: [{ item_id: 3922, hue: 0 }] })
+  t.after(restore)
+
+  const result = await model.warm()
+
+  assert.equal(result.status, 'unavailable')
+  assert.equal(result.code, 'UNSUPPORTED')
+  assert.match(result.reason, /does not serve item art/)
+  assert.equal(seen.calls, 0, 'nothing should have been asked of the shard')
+})
+
+test('no shard link is skipped, not failed', async (t) => {
+  stub({ linked: false })
+  t.after(restore)
+
+  assert.equal((await model.warm()).status, 'skipped')
+})
+
+test('a host that cannot render images is the named NO_IMAGING state', async (t) => {
+  stub()
+  t.after(restore)
+
+  bridge.sourceFingerprint = async () => ({
+    files: {},
+    extractorVersion: 2,
+    hashing: false,
+    complete: true,
+    imaging: { ok: false, reason: 'libgdiplus is not installed' },
+    families: ['body', 'static'],
+  })
+
+  const result = await model.warm()
+
+  assert.equal(result.status, 'unavailable')
+  assert.equal(result.code, 'NO_IMAGING')
+})
+
+// ── what gets asked for ────────────────────────────────────────────────────
+
+test('only the keys we do not already hold under the shard’s current catalogue are fetched', async (t) => {
+  useTempUploads(t)
+
+  const seen = stub({
+    wanted: [
+      { item_id: 3922, hue: 0 },
+      { item_id: 597, hue: 33 },
+      { item_id: 1, hue: 0 },
+    ],
+    // 3922 is held and current; the other two are not.
+    fresh: new Set(['static/3922']),
+    fetched: {
+      assets: new Map([
+        ['static/597/h33', picture('aaa')],
+        ['static/1', picture('bbb')],
+      ]),
+      missing: { absent: 0, unsupported: 0 },
+      pages: 1,
+      catalog: 'cat-current',
+    },
+  })
+  t.after(restore)
+
+  const result = await model.warm()
+
+  assert.equal(result.status, 'imported')
+
+  // The first call is the catalogue probe; the second is the real fetch.
+  const asked = seen.asked[seen.asked.length - 1]
+
+  assert.deepEqual(asked.sort(), ['static/1', 'static/597/h33'])
+  assert.equal(
+    seen.freshAsked.catalog,
+    'cat-current',
+    'staleness must be asked against the catalogue the shard answers under right now, ' +
+      'or a client patch never invalidates anything',
+  )
+})
+
+test('every stored row records the catalogue it was fetched under', async (t) => {
+  useTempUploads(t)
+
+  const seen = stub({
+    wanted: [{ item_id: 1, hue: 0 }],
+    fetched: {
+      assets: new Map([['static/1', picture('bbb')]]),
+      missing: { absent: 0, unsupported: 0 },
+      pages: 1,
+      catalog: 'cat-after-patch',
+    },
+  })
+  t.after(restore)
+
+  await model.warm()
+
+  // Without this field there is no way to answer "is this picture out of date?"
+  // for a family that has no manifest — which is the entire §7 story on this side.
+  assert.equal(seen.saved.rows.length, 1)
+  assert.equal(seen.saved.rows[0].catalog, 'cat-after-patch')
+  assert.equal(seen.saved.rows[0].family, 'static')
+})
+
+test('the body catalogue’s meta singleton is never written by a warm pass', async (t) => {
+  useTempUploads(t)
+
+  const seen = stub({
+    wanted: [{ item_id: 1, hue: 0 }],
+    fetched: {
+      assets: new Map([['static/1', picture('bbb')]]),
+      missing: { absent: 0, unsupported: 0 },
+      pages: 1,
+      catalog: 'cat-current',
+    },
+  })
+  t.after(restore)
+
+  await model.warm()
+
+  // `shard_asset_meta` is what an Update compares a BODY manifest against. A
+  // warm pass writing there would tell the body import that a client it never
+  // looked at is unchanged, and the creature catalogue would stop updating.
+  assert.equal(seen.saved.meta, null)
+})
+
+test('a key the shard has no art for produces no row, so it can be asked again', async (t) => {
+  useTempUploads(t)
+
+  const seen = stub({
+    wanted: [
+      { item_id: 1, hue: 0 },
+      { item_id: 60000, hue: 0 },
+    ],
+    fetched: {
+      assets: new Map([['static/1', picture('bbb')]]),
+      missing: { absent: 1, unsupported: 0 },
+      pages: 1,
+      catalog: 'cat-current',
+    },
+  })
+  t.after(restore)
+
+  const result = await model.warm()
+
+  assert.equal(result.absent, 1)
+  assert.deepEqual(
+    seen.saved.rows.map((r) => r.key),
+    ['static/1'],
+    'an empty row would make the key held, and it would never be asked again — ' +
+      'including after the operator patches in the graphic that was missing',
+  )
+})
+
+test('a pass is bounded, and says how much it left behind', async (t) => {
+  useTempUploads(t)
+
+  const wanted = []
+  for (let i = 1; i <= 10; i++) wanted.push({ item_id: i, hue: 0 })
+
+  const seen = stub({
+    wanted,
+    fetched: {
+      assets: new Map([['static/1', picture('bbb')]]),
+      missing: { absent: 0, unsupported: 0 },
+      pages: 1,
+      catalog: 'cat-current',
+    },
+  })
+  t.after(restore)
+
+  const result = await model.warm({ limit: 4 })
+
+  assert.equal(seen.asked[seen.asked.length - 1].length, 4)
+  assert.equal(result.asked, 4)
+  assert.equal(result.remaining, 6)
+})
+
+test('a picture whose bytes changed replaces its file instead of shadowing it', async (t) => {
+  const dir = useTempUploads(t)
+
+  const old = model.fileNameFor('static/1', 'old00000')
+  fs.mkdirSync(model.artDir(), { recursive: true })
+  fs.writeFileSync(path.join(model.artDir(), old), 'stale')
+
+  stub({
+    wanted: [{ item_id: 1, hue: 0 }],
+    files: new Map([['static/1', old]]),
+    fetched: {
+      assets: new Map([['static/1', picture('new00000')]]),
+      missing: { absent: 0, unsupported: 0 },
+      pages: 1,
+      catalog: 'cat-after-patch',
+    },
+  })
+  t.after(restore)
+
+  await model.warm()
+
+  const names = fs.readdirSync(path.join(dir, model.ART_SUBDIR))
+
+  // Content-addressed names mean a changed picture is a changed URL, so nothing
+  // keeps serving last client's sprite from a cache — and the superseded file is
+  // removed rather than left to accumulate one per client patch forever.
+  assert.deepEqual(names, [model.fileNameFor('static/1', 'new00000')])
+})
+
+// ── serving ────────────────────────────────────────────────────────────────
+
+test('decorate attaches a filename, never a URL, and null where there is none', async (t) => {
+  stub({ files: new Map([['static/3922', 'uo-static-3922-abcd1234.png']]) })
+  t.after(restore)
+
+  const rows = [
+    { itemId: 3922, hue: 0 },
+    { itemId: 597, hue: 33 },
+  ]
+
+  await model.decorate(rows)
+
+  // A filename, because the client is what knows where uploads are mounted —
+  // the same contract `shard_spawn_creatures.art` already uses.
+  assert.equal(rows[0].art, 'uo-static-3922-abcd1234.png')
+  assert.equal(rows[1].art, null)
+})
+
+test('decorate never throws a page away over a picture', async (t) => {
+  stub()
+  t.after(restore)
+
+  db.filesForKeys = async () => {
+    throw new Error('the database is on fire')
+  }
+
+  const rows = [{ itemId: 3922, hue: 0 }]
+
+  await model.decorate(rows)
+
+  assert.deepEqual(rows, [{ itemId: 3922, hue: 0 }], 'the row is returned unchanged, not lost')
+})
+
+test('what a page asked for is remembered, including what it could not show', async (t) => {
+  stub({ files: new Map() })
+  t.after(restore)
+
+  const before = model.noticedCount()
+
+  await model.decorate([{ itemId: 12345, hue: 7 }])
+
+  // The character sheet is fetched live from the shard and stored nowhere, so
+  // nothing on disk would ever name this key. Noticing it here is the only reason
+  // a warm pass can find it.
+  assert.ok(model.noticedCount() > before)
+  assert.ok((await model.wantedKeys()).includes('static/12345/h7'))
+})