feat(assets): item pictures on the marketplace and the character sheet (Phase 5)
All checks were successful
PR Checks / client-build (pull_request) Successful in 17s
PR Checks / frozen-manifest (pull_request) Successful in 41s
PR Checks / server-tests (pull_request) Successful in 8m23s

Both places this site already knew an item's (ItemID, hue) and could only print
it as text now show the picture, hued the way the client would draw it. The
shard does the hueing: whether a hue repaints every pixel or only the grey ones
is a flag in `tiledata.mul`, which a browser has no way to read.

**Ingest warms; the route only serves** (org lead, 2026-09-11). A page never
waits on the shard and never causes a fetch -- it renders what is stored and
leaves out what is not, which is the state every install was in before this
phase. Fetching happens behind that, on a timer, from the keys the site's own
rows name. The alternative, fetching on first request, was rejected on one
number: the shard's asset plane serves ONE request at a time, so a URL that
fetched would let any visitor walk 49,152 ids times 3,000 hues through that slot
and park an operator's own import behind it.

The wanted set is DERIVED (`SELECT DISTINCT item_id, hue`) rather than queued, so
it is self-healing: a restart loses nothing, and a key stops being wanted the
moment the vendor row naming it is deleted. The in-memory hint set on top is only
for the character sheet, which is fetched live from the shard and stored nowhere
-- nothing on disk would ever name those keys.

Staleness without a manifest (§7): every row records the shard's `catalog` id, a
hash of the files that decide its bytes. A client patch changes it and a restart
does not, so "is this out of date?" is a per-row question -- and pictures nobody
looks at any more are simply never re-fetched, which is why this is lazy rather
than a sweep. `shard_asset_meta` is deliberately NOT written here: it is the body
catalogue's singleton, and a warm pass touching it would tell the body import
that a client it never looked at is unchanged.

A key the shard has no art for writes no row at all. An empty row would make the
key held and it would never be asked again -- including after the operator
patches in the graphic that was missing.

`assets.sources` now reports which families an overlay serves, so an overlay
older than phase 5 is one reported state with a sentence naming the fix, instead
of a refusal per pass forever with no picture ever appearing.

688 server tests pass (14 new); client builds; the frozen manifest regenerates
with one added route, all documented, no core URL moved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
2026-09-11 06:13:08 -05:00
parent 6ece48f7d3
commit f335531538
18 changed files with 1712 additions and 10 deletions

View File

@@ -33,7 +33,7 @@ async function batched(conn, sql, rows) {
/** Every asset row we hold, as a Map of key → row. */
async function allAssets() {
const rows = await query(
'SELECT asset_key, family, sha256, bytes, width, height, body, direction, file FROM shard_assets',
'SELECT asset_key, family, sha256, bytes, width, height, body, direction, file, catalog FROM shard_assets',
)
const map = new Map()
@@ -49,6 +49,7 @@ async function allAssets() {
body: row.body === null ? null : Number(row.body),
direction: row.direction === null ? null : Number(row.direction),
file: row.file || null,
catalog: row.catalog || null,
})
}
@@ -82,16 +83,17 @@ async function saveAssets(rows, meta) {
r.body ?? null,
r.direction ?? null,
r.file ?? null,
r.catalog ?? meta?.catalog ?? null,
])
await batched(
conn,
'INSERT INTO shard_assets (asset_key, family, sha256, bytes, width, height, body, direction, file) ' +
'VALUES (?,?,?,?,?,?,?,?,?) ' +
'INSERT INTO shard_assets (asset_key, family, sha256, bytes, width, height, body, direction, file, catalog) ' +
'VALUES (?,?,?,?,?,?,?,?,?,?) ' +
'ON DUPLICATE KEY UPDATE family = VALUES(family), sha256 = VALUES(sha256), ' +
'bytes = VALUES(bytes), width = VALUES(width), height = VALUES(height), ' +
'body = VALUES(body), direction = VALUES(direction), file = VALUES(file), ' +
'imported_at = CURRENT_TIMESTAMP',
'catalog = VALUES(catalog), imported_at = CURRENT_TIMESTAMP',
values,
)
@@ -114,6 +116,74 @@ async function saveAssets(rows, meta) {
}
}
// ── the on-demand side (§11, phase 5) ──────────────────────────────────────
/**
* The pictures we hold for an explicit list of keys, as a Map of key → filename.
*
* This is the read on the hot path — every marketplace page and every character
* sheet runs it — so it is one statement over the primary key and it returns only
* what it is asked for. It deliberately does NOT check staleness: a page renders
* the picture it has, and deciding whether that picture is out of date is the warm
* pass's job, off the request.
*/
async function filesForKeys(keys) {
const list = [...new Set(keys.filter((k) => typeof k === 'string' && k !== ''))]
if (list.length === 0) return new Map()
const rows = await query(
`SELECT asset_key, file FROM shard_assets WHERE file IS NOT NULL AND asset_key IN (${list
.map(() => '?')
.join(',')})`,
list,
)
const map = new Map()
for (const row of rows) map.set(row.asset_key, row.file)
return map
}
/**
* Which of these keys we already hold under the shard's CURRENT catalogue.
*
* The warm pass subtracts this from what it wants, so everything it does not
* return gets fetched: a key we have never seen, and a key whose row was written
* against a catalogue the shard has since moved past (§7 — an operator patched
* their client). A row with no file is not held either, because the database and
* the uploads volume can disagree and a broken image is worse than a re-fetch.
*/
async function freshKeys(keys, catalog) {
const list = [...new Set(keys.filter((k) => typeof k === 'string' && k !== ''))]
if (list.length === 0) return new Set()
const rows = await query(
`SELECT asset_key FROM shard_assets WHERE file IS NOT NULL AND catalog <=> ? ` +
`AND asset_key IN (${list.map(() => '?').join(',')})`,
[catalog ?? null, ...list],
)
return new Set(rows.map((r) => r.asset_key))
}
/** Counts for the admin surface, split by family. */
async function countByFamily() {
const rows = await query(
'SELECT family, COUNT(*) AS total, SUM(file IS NOT NULL) AS stored FROM shard_assets GROUP BY family',
)
const out = {}
for (const row of rows) {
out[row.family] = { total: Number(row.total) || 0, stored: Number(row.stored) || 0 }
}
return out
}
async function getMeta() {
const rows = await query('SELECT payload, imported_at FROM shard_asset_meta WHERE id = 1')
if (rows.length === 0) return null
@@ -220,4 +290,7 @@ module.exports = {
allBodies,
countBodies,
artBySlug,
filesForKeys,
freshKeys,
countByFamily,
}