feat(assets): creature artwork from the shard's own client (Phase 3)
Until now the only way a creature got a picture on this site was for an
operator to open UOFiddler on a desktop, export sprites by hand, copy them to
the web host and write a spawnAtlas.art.json naming each one. Almost nobody
did, so shard_spawn_creatures.art was NULL on every install.
The shard has had those files the whole time. Admin -> Shard -> Import now
walks its asset manifest, fetches only the sprites whose hash changed, writes
them under uploads/atlas/, asks the shard for a body id per atlas creature
(§8: it CONSTRUCTS the creature and reads Body.BodyID, which is the only thing
that is right for a shard's own custom creatures) and points each creature at
its picture. On a stock client that is 787 portraits, about a megabyte.
**The one thing v8.md §12 got wrong, and it is not cosmetic.** It says
`shard_spawn_creatures.art` "starts being filled by the import". That table is
emptied and refilled by replaceAtlas on EVERY atlas refresh, and a refresh runs
on every boot -- so a filename stored there would be destroyed by an ordinary
re-parse of the ServUO tree, with the next Update finding the client files
unchanged, reporting "nothing to do", and never restoring it. Nothing would
report a fault; the pictures would just be gone.
So the assets and the body map live in their own tables outside that blast
radius, and applyAtlas re-derives `art` on the way past as
`{ ...derived, ...operatorMap }` -- which is also the one place "the operator's
own artwork wins" is enforced, on every rebuild rather than only at import.
Smaller decisions worth not rediscovering:
- The derivation joins on the catalogue KEY, not on the body id. The simpler
join is correct today and stops being correct the moment phase 6 adds
body/400/a2/f0, at which point one slug matches dozens of rows.
- Filenames are content-addressed. A stable name overwritten in place leaves
every browser and CDN serving the previous client's sprite, with the database
row perfectly correct.
- An unchanged key whose FILE is missing is fetched again. The row and the disk
can disagree (a wiped uploads volume, a restore from a dump), and a broken
image on a creature page is worse than one re-fetched sprite.
- A key the shard cannot render is not a failure. Two thirds of the playable
ghost and gargoyle bodies have no art on a stock client, and an import that
reported eight failures every time would teach an operator to ignore the panel.
- A key that VANISHED from the manifest needs review before anything changes:
an unmounted client volume and a deliberate downgrade look identical here.
23 new tests; 674 server and 42 client tests pass. The SQL was also run against
a real MariaDB, which is what proved the CONCAT join and the singleton CHECK.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
223
server/model/shardAssets/shardAssets.db.js
Normal file
223
server/model/shardAssets/shardAssets.db.js
Normal file
@@ -0,0 +1,223 @@
|
||||
const core = require('../../core')
|
||||
|
||||
const { query } = core
|
||||
|
||||
// Raw SQL for the Asset Bridge's three tables (docs/link/v8.md §6, §8, §12).
|
||||
//
|
||||
// Unlike `shard_clilocs` and the atlas tables, these are NOT import-owned in the
|
||||
// empty-and-refill sense, and the difference is the whole reason phase 3 put them
|
||||
// in their own tables rather than in columns on `shard_spawn_creatures`.
|
||||
//
|
||||
// An asset row is expensive to obtain — a decode on the shard, a PNG across the
|
||||
// wire, a file written under uploads/ — and it is valid until the operator
|
||||
// patches their client. An atlas refresh, by contrast, happens on every boot and
|
||||
// destroys everything it owns. Putting the two in one table would mean a routine
|
||||
// re-parse of the ServUO tree silently deleting every imported portrait, with the
|
||||
// next Update reporting "nothing changed" and never restoring them.
|
||||
//
|
||||
// So these are upserted per key, and the only thing that ever deletes from them
|
||||
// is an explicit removal of a key the shard no longer offers — which is staged
|
||||
// for review, never applied silently (§6).
|
||||
|
||||
const BATCH = 500
|
||||
|
||||
async function batched(conn, sql, rows) {
|
||||
for (let i = 0; i < rows.length; i += BATCH) {
|
||||
await conn.batch(sql, rows.slice(i, i + BATCH))
|
||||
}
|
||||
return rows.length
|
||||
}
|
||||
|
||||
// ── the manifest side ──────────────────────────────────────────────────────
|
||||
|
||||
/** Every asset row we hold, as a Map of key → row. */
|
||||
async function allAssets() {
|
||||
const rows = await query(
|
||||
'SELECT asset_key, family, sha256, bytes, width, height, body, direction, file FROM shard_assets',
|
||||
)
|
||||
|
||||
const map = new Map()
|
||||
|
||||
for (const row of rows) {
|
||||
map.set(row.asset_key, {
|
||||
key: row.asset_key,
|
||||
family: row.family,
|
||||
sha256: row.sha256,
|
||||
bytes: Number(row.bytes) || 0,
|
||||
width: Number(row.width) || 0,
|
||||
height: Number(row.height) || 0,
|
||||
body: row.body === null ? null : Number(row.body),
|
||||
direction: row.direction === null ? null : Number(row.direction),
|
||||
file: row.file || null,
|
||||
})
|
||||
}
|
||||
|
||||
return map
|
||||
}
|
||||
|
||||
/**
|
||||
* Write the assets an import produced, and record what the import was.
|
||||
*
|
||||
* One transaction for the rows and the meta together: the meta row is what an
|
||||
* Update compares against to decide there is nothing to do, so a meta written
|
||||
* without its rows would make the site believe it holds a catalogue it does not.
|
||||
*
|
||||
* `ON DUPLICATE KEY UPDATE` rather than delete-and-insert, because an unchanged
|
||||
* key must keep the file it already points at — re-writing the file for every
|
||||
* asset on every Update is exactly the cost the manifest diff exists to avoid.
|
||||
*/
|
||||
async function saveAssets(rows, meta) {
|
||||
const conn = await core.pool.getConnection()
|
||||
|
||||
try {
|
||||
await conn.beginTransaction()
|
||||
|
||||
const values = rows.map((r) => [
|
||||
r.key,
|
||||
r.family || 'body',
|
||||
r.sha256,
|
||||
r.bytes ?? 0,
|
||||
r.width ?? 0,
|
||||
r.height ?? 0,
|
||||
r.body ?? null,
|
||||
r.direction ?? null,
|
||||
r.file ?? null,
|
||||
])
|
||||
|
||||
await batched(
|
||||
conn,
|
||||
'INSERT INTO shard_assets (asset_key, family, sha256, bytes, width, height, body, direction, file) ' +
|
||||
'VALUES (?,?,?,?,?,?,?,?,?) ' +
|
||||
'ON DUPLICATE KEY UPDATE family = VALUES(family), sha256 = VALUES(sha256), ' +
|
||||
'bytes = VALUES(bytes), width = VALUES(width), height = VALUES(height), ' +
|
||||
'body = VALUES(body), direction = VALUES(direction), file = VALUES(file), ' +
|
||||
'imported_at = CURRENT_TIMESTAMP',
|
||||
values,
|
||||
)
|
||||
|
||||
if (meta) {
|
||||
await conn.query(
|
||||
'INSERT INTO shard_asset_meta (id, payload) VALUES (1, ?) ' +
|
||||
'ON DUPLICATE KEY UPDATE payload = VALUES(payload), imported_at = CURRENT_TIMESTAMP',
|
||||
[JSON.stringify(meta)],
|
||||
)
|
||||
}
|
||||
|
||||
await conn.commit()
|
||||
|
||||
return values.length
|
||||
} catch (err) {
|
||||
await conn.rollback().catch(() => {})
|
||||
throw err
|
||||
} finally {
|
||||
conn.release()
|
||||
}
|
||||
}
|
||||
|
||||
async function getMeta() {
|
||||
const rows = await query('SELECT payload, imported_at FROM shard_asset_meta WHERE id = 1')
|
||||
if (rows.length === 0) return null
|
||||
const payload = typeof rows[0].payload === 'string' ? JSON.parse(rows[0].payload) : rows[0].payload
|
||||
return { ...payload, importedAt: rows[0].imported_at }
|
||||
}
|
||||
|
||||
async function countAssets() {
|
||||
const rows = await query(
|
||||
'SELECT COUNT(*) AS n, SUM(file IS NOT NULL) AS stored FROM shard_assets',
|
||||
)
|
||||
return { total: Number(rows[0]?.n) || 0, stored: Number(rows[0]?.stored) || 0 }
|
||||
}
|
||||
|
||||
// ── the body resolution side (§8) ──────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Replace the whole slug → body map.
|
||||
*
|
||||
* This one IS a replace, and for the opposite reason to the assets above: it is
|
||||
* derived from the atlas's creature list, so a slug that has left the atlas has
|
||||
* no meaning any more and keeping its row would leave the map growing forever
|
||||
* across map changes. The pass that produces it is cheap to redo — a shard round
|
||||
* trip, no files — which is what makes replacing safe here and not there.
|
||||
*/
|
||||
async function replaceBodies(rows) {
|
||||
const conn = await core.pool.getConnection()
|
||||
|
||||
try {
|
||||
await conn.beginTransaction()
|
||||
await conn.query('DELETE FROM shard_creature_bodies')
|
||||
|
||||
const values = rows.map((r) => [r.slug, r.typeName, r.body ?? null, r.status || 'ok'])
|
||||
|
||||
await batched(
|
||||
conn,
|
||||
'INSERT INTO shard_creature_bodies (slug, type_name, body, status) VALUES (?,?,?,?)',
|
||||
values,
|
||||
)
|
||||
|
||||
await conn.commit()
|
||||
|
||||
return values.length
|
||||
} catch (err) {
|
||||
await conn.rollback().catch(() => {})
|
||||
throw err
|
||||
} finally {
|
||||
conn.release()
|
||||
}
|
||||
}
|
||||
|
||||
async function allBodies() {
|
||||
return query(
|
||||
'SELECT slug, type_name, body, status, resolved_at FROM shard_creature_bodies ORDER BY slug',
|
||||
)
|
||||
}
|
||||
|
||||
async function countBodies() {
|
||||
const rows = await query(
|
||||
"SELECT COUNT(*) AS n, SUM(status = 'ok') AS resolved FROM shard_creature_bodies",
|
||||
)
|
||||
return { total: Number(rows[0]?.n) || 0, resolved: Number(rows[0]?.resolved) || 0 }
|
||||
}
|
||||
|
||||
/**
|
||||
* The derivation `replaceAtlas` applies on the way past: slug → uploaded filename.
|
||||
*
|
||||
* One join rather than two reads, because it runs inside the atlas transaction —
|
||||
* the atlas rows are being inserted at that moment and every extra round trip is
|
||||
* time the site's creature list does not exist.
|
||||
*
|
||||
* Rows with no body, no asset or an asset whose bytes were never fetched are
|
||||
* simply absent from the result, which is what leaves `art` NULL. That is a
|
||||
* first-class state everywhere it is consumed and the expected one for two thirds
|
||||
* of the player bodies (§5.2).
|
||||
*
|
||||
* **The join is pinned to the catalogue key, not merely to the body id.** Today
|
||||
* one body has exactly one asset, so `a.body = b.body` alone would be correct —
|
||||
* and it would stop being correct the moment phase 6 adds `body/400/a2/f0`, at
|
||||
* which point one slug would match dozens of rows and whichever the engine
|
||||
* returned last would become the portrait. Naming the key here means that phase
|
||||
* adds rows without changing what a creature page shows.
|
||||
*/
|
||||
async function artBySlug() {
|
||||
const rows = await query(
|
||||
'SELECT b.slug, a.file FROM shard_creature_bodies b ' +
|
||||
"JOIN shard_assets a ON a.asset_key = CONCAT('body/', b.body, '/a0') " +
|
||||
"WHERE b.status = 'ok' AND b.body IS NOT NULL AND a.file IS NOT NULL",
|
||||
)
|
||||
|
||||
const map = {}
|
||||
|
||||
for (const row of rows) map[row.slug] = row.file
|
||||
|
||||
return map
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
allAssets,
|
||||
saveAssets,
|
||||
getMeta,
|
||||
countAssets,
|
||||
replaceBodies,
|
||||
allBodies,
|
||||
countBodies,
|
||||
artBySlug,
|
||||
}
|
||||
507
server/model/shardAssets/shardAssets.model.js
Normal file
507
server/model/shardAssets/shardAssets.model.js
Normal file
@@ -0,0 +1,507 @@
|
||||
const fs = require('fs')
|
||||
const path = require('path')
|
||||
|
||||
const db = require('./shardAssets.db')
|
||||
const atlasDb = require('../shardAtlas/shardAtlas.db')
|
||||
const core = require('../../core')
|
||||
const bridge = require('../../utils/assetBridge')
|
||||
const uoLinkConfig = require('../uoLinkConfig/uoLinkConfig.model')
|
||||
const log = require('../../core').logger('shardAssets')
|
||||
|
||||
// Client artwork, over the bridge (docs/link/v8.md — protocol 8, phase 3).
|
||||
//
|
||||
// What this replaces: until now the only way a creature got a picture on this
|
||||
// site was for an operator to open UOFiddler on a desktop, export sprites by
|
||||
// hand, copy them to the web host and write a `spawnAtlas.art.json` naming each
|
||||
// one. Almost nobody did, so `shard_spawn_creatures.art` was NULL on every
|
||||
// install and the atlas rendered as text.
|
||||
//
|
||||
// The shard has had those files the whole time — a ServUO server cannot boot
|
||||
// without a UO client — so as of protocol 8 it decodes them itself and hands the
|
||||
// pictures over the same request/reply path as every other shard read.
|
||||
//
|
||||
// ── Two passes, and they answer different questions ───────────────────
|
||||
//
|
||||
// **The catalogue** (§4.8, §11) is one thumbnail per creature body: the shard
|
||||
// walks bodies 0–2047, validates each index entry, decodes the ones that are real
|
||||
// and hands back `{ key, sha256 }` first and the PNG second. On a stock client
|
||||
// that is **787 sprites**, not the 1,144 the decoder claims — see below.
|
||||
//
|
||||
// **Body resolution** (§8) is the join. The atlas knows a creature by the class
|
||||
// name in `Spawns/*.xml`; the client knows it by a body id; nothing in the ServUO
|
||||
// tree declares the mapping. Only code inside ServUO can answer it, by
|
||||
// constructing the creature and reading `Body.BodyID`, and that is the whole
|
||||
// reason this could not be done off the shard.
|
||||
//
|
||||
// ── The 357, and why nothing here trusts a success ────────────────────
|
||||
//
|
||||
// 357 of the bodies ServUO's decoder returns a bitmap for **have no art**. Their
|
||||
// index entry reads `length 0`, the library's stream buffer still holds the
|
||||
// previous creature, and what comes back is whichever body was decoded before —
|
||||
// a real, plausible, correctly-sized picture of the wrong animal. The shard now
|
||||
// validates every index entry before it decodes, which is what cut the catalogue
|
||||
// from 1,144 to 787, and the count going down is the point.
|
||||
//
|
||||
// The consequence for this file is a rule: **a missing asset is a normal
|
||||
// outcome, never an error.** Two thirds of the player bodies have no art on a
|
||||
// stock client (§5.2), so an import that reported eight failures every time would
|
||||
// teach an operator to ignore the panel.
|
||||
//
|
||||
// ── Where the pictures go, and what still wins ────────────────────────
|
||||
//
|
||||
// Into `<uploads>/atlas/`, through the same door the operator's own artwork uses,
|
||||
// and `shard_spawn_creatures.art` is DERIVED from them rather than written by
|
||||
// them. **The operator's `spawnAtlas.art.json` still wins outright**: someone who
|
||||
// has drawn their own creature portraits must not have them replaced by a sprite
|
||||
// rip on the next Update.
|
||||
//
|
||||
// ── Why the resolution does not live on the atlas row ─────────────────
|
||||
//
|
||||
// `shard_spawn_creatures` is emptied and refilled on every atlas refresh. A body
|
||||
// id or a filename stored there would be destroyed by an ordinary re-parse of the
|
||||
// ServUO tree, and the next asset Update would find the client files unchanged,
|
||||
// report "nothing to do" and never restore it. So both live in their own tables
|
||||
// and the atlas import reads them on the way past.
|
||||
|
||||
/** Where imported sprites land, under core's upload directory. */
|
||||
const ART_SUBDIR = 'atlas'
|
||||
|
||||
// ── configuration ──────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Is there a shard to ask?
|
||||
*
|
||||
* Both halves matter, exactly as in `shardClilocs.model`: a `baseUrl` on a
|
||||
* disabled config is an install that was set up and then switched off, and
|
||||
* calling it would spend a 12 s timeout to learn what the row already says.
|
||||
*/
|
||||
async function shardLinked() {
|
||||
try {
|
||||
const config = await uoLinkConfig.getSafe()
|
||||
return Boolean(config?.enabled && config?.baseUrl)
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
function artDir() {
|
||||
return path.join(core.uploads.UPLOAD_DIR, ART_SUBDIR)
|
||||
}
|
||||
|
||||
/**
|
||||
* The operator's own art map, which wins over anything imported.
|
||||
*
|
||||
* Read through the atlas model rather than re-implemented, so there is one
|
||||
* definition of where that file lives and what an absent one means.
|
||||
*/
|
||||
function operatorArt() {
|
||||
// eslint-disable-next-line global-require
|
||||
return require('../shardAtlas/shardAtlas.model').loadArtMap()
|
||||
}
|
||||
|
||||
// ── writing a sprite ───────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* The filename one asset gets on disk.
|
||||
*
|
||||
* **Content-addressed on purpose.** A stable name per key (`uo-body-34.png`)
|
||||
* would be overwritten in place by an Update, and every browser and CDN that had
|
||||
* already cached it would keep serving last month's client's sprite — with
|
||||
* nothing anywhere to notice, because the database row would be correct. Putting
|
||||
* eight bytes of the hash in the name makes a changed sprite a changed URL.
|
||||
*
|
||||
* The old file is removed when a key's hash moves, so the directory tracks the
|
||||
* catalogue rather than accumulating one file per import forever.
|
||||
*/
|
||||
function fileNameFor(key, sha256) {
|
||||
const stem = key.replace(/[^a-zA-Z0-9]+/g, '-').replace(/^-+|-+$/g, '')
|
||||
return `uo-${stem}-${String(sha256).slice(0, 8)}.png`
|
||||
}
|
||||
|
||||
/**
|
||||
* Write one sprite and return its filename, or null if it could not be written.
|
||||
*
|
||||
* Never throws. A full disk or a read-only volume must degrade to "this creature
|
||||
* has no picture" — which the whole site already renders correctly, because it is
|
||||
* the state every install was in until this phase — rather than failing an import
|
||||
* that has already fetched hundreds of others.
|
||||
*/
|
||||
function writeSprite(key, sha256, png) {
|
||||
const name = fileNameFor(key, sha256)
|
||||
|
||||
try {
|
||||
fs.mkdirSync(artDir(), { recursive: true })
|
||||
fs.writeFileSync(path.join(artDir(), name), png)
|
||||
return name
|
||||
} catch (err) {
|
||||
log.warn('could not write an imported sprite', { key, error: err.message })
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
/** Best-effort removal of a sprite a key no longer points at. */
|
||||
function removeSprite(name) {
|
||||
if (!name) return
|
||||
|
||||
try {
|
||||
fs.unlinkSync(path.join(artDir(), name))
|
||||
} catch {
|
||||
// Already gone, or never written. Either way there is nothing to do, and an
|
||||
// import must not fail because a file it was tidying up was tidied already.
|
||||
}
|
||||
}
|
||||
|
||||
// ── the import ─────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Import (or update) the body catalogue and the slug → body map.
|
||||
*
|
||||
* Returns a result rather than throwing, so a controller can render it and an
|
||||
* operator can read it:
|
||||
*
|
||||
* `skipped` no shard configured — the file era had no equivalent here
|
||||
* `unavailable` the shard could not answer (down, plane off, no libgdiplus)
|
||||
* `unchanged` the client files match what was imported; nothing fetched
|
||||
* `imported` fetched and applied
|
||||
* `needsReview` a key we hold has vanished from the shard's manifest
|
||||
* `failed` something went wrong mid-import
|
||||
*
|
||||
* `force` re-imports even when the client files are unchanged (which is also how
|
||||
* an operator recovers from a deleted uploads directory — the database still
|
||||
* holds the hashes, but the files behind them are gone). `approve` accepts a
|
||||
* catalogue that no longer offers keys we hold.
|
||||
*/
|
||||
async function importAssets({ force = false, approve = false } = {}) {
|
||||
if (!(await shardLinked())) {
|
||||
return {
|
||||
status: 'skipped',
|
||||
reason: 'uo-link is not configured, so there is no shard to read client files from',
|
||||
}
|
||||
}
|
||||
|
||||
let sources
|
||||
|
||||
try {
|
||||
sources = await bridge.sourceFingerprint()
|
||||
} catch (err) {
|
||||
return failure(err, 'client file manifest')
|
||||
}
|
||||
|
||||
// §4.4: a Linux shard host without libgdiplus cannot render a sprite at all.
|
||||
// It is reported on the source gate precisely so an operator meets it while
|
||||
// setting the shard up rather than from an empty bestiary weeks later.
|
||||
if (sources.imaging && sources.imaging.ok === false) {
|
||||
return {
|
||||
status: 'unavailable',
|
||||
code: 'NO_IMAGING',
|
||||
reason: sources.imaging.reason || 'the shard host cannot render images',
|
||||
}
|
||||
}
|
||||
|
||||
const meta = await db.getMeta().catch(() => null)
|
||||
|
||||
if (!force && bridge.sameSources(sources, meta?.sources)) {
|
||||
const counts = await db.countAssets()
|
||||
const bodies = await db.countBodies()
|
||||
|
||||
return {
|
||||
status: 'unchanged',
|
||||
assets: counts.total,
|
||||
stored: counts.stored,
|
||||
bodies: bodies.resolved,
|
||||
hashing: sources.hashing,
|
||||
importedAt: meta?.importedAt ?? null,
|
||||
}
|
||||
}
|
||||
|
||||
let manifest
|
||||
|
||||
try {
|
||||
manifest = await bridge.readManifest({ family: bridge.FAMILY })
|
||||
} catch (err) {
|
||||
return failure(err, 'asset manifest')
|
||||
}
|
||||
|
||||
const held = await db.allAssets()
|
||||
const offered = new Set(manifest.rows.map((r) => r.key))
|
||||
|
||||
// A key we hold that the shard no longer offers. An unmounted client volume and
|
||||
// a deliberate downgrade look identical from here, and the wrong guess deletes
|
||||
// artwork, so it is staged rather than applied — the same rule, and the same
|
||||
// reasoning, as a vanished cliloc overlay or a disappearing atlas facet.
|
||||
const vanished = [...held.keys()].filter((key) => !offered.has(key))
|
||||
|
||||
if (vanished.length > 0 && !approve) {
|
||||
return {
|
||||
status: 'needsReview',
|
||||
reason:
|
||||
`${vanished.length} asset(s) this site holds are no longer offered by the shard; ` +
|
||||
'nothing was changed',
|
||||
vanished: vanished.slice(0, 50),
|
||||
vanishedCount: vanished.length,
|
||||
}
|
||||
}
|
||||
|
||||
// The diff, and the whole reason stage 2 carries hashes and not pixels. An
|
||||
// unchanged key is skipped ONLY if its file is actually still on disk: the row
|
||||
// and the file can disagree (a wiped uploads volume, a restore from a database
|
||||
// dump), and re-fetching a sprite is far cheaper than a creature page with a
|
||||
// broken image on it.
|
||||
const wanted = manifest.rows.filter((row) => {
|
||||
const existing = held.get(row.key)
|
||||
if (!existing || existing.sha256 !== row.sha256) return true
|
||||
if (!existing.file) return true
|
||||
return !fs.existsSync(path.join(artDir(), existing.file))
|
||||
})
|
||||
|
||||
let fetched = { assets: new Map(), missing: { absent: 0, unsupported: 0 } }
|
||||
|
||||
if (wanted.length > 0) {
|
||||
try {
|
||||
fetched = await bridge.fetchAssets({
|
||||
keys: wanted.map((r) => r.key),
|
||||
catalog: manifest.catalog,
|
||||
})
|
||||
} catch (err) {
|
||||
return failure(err, 'asset content')
|
||||
}
|
||||
}
|
||||
|
||||
const rows = []
|
||||
let written = 0
|
||||
|
||||
for (const row of manifest.rows) {
|
||||
const existing = held.get(row.key)
|
||||
const got = fetched.assets.get(row.key)
|
||||
|
||||
if (!got) {
|
||||
// Either it was unchanged and skipped, or the shard could not serve it. The
|
||||
// row is kept either way, with whatever file it already had — a key the
|
||||
// shard suddenly cannot render must not lose the picture we already hold.
|
||||
rows.push({ ...row, file: existing?.file ?? null })
|
||||
continue
|
||||
}
|
||||
|
||||
const name = writeSprite(row.key, got.sha256, got.png)
|
||||
|
||||
if (name) {
|
||||
written++
|
||||
if (existing?.file && existing.file !== name) removeSprite(existing.file)
|
||||
}
|
||||
|
||||
rows.push({
|
||||
...row,
|
||||
sha256: got.sha256 || row.sha256,
|
||||
bytes: got.bytes || row.bytes,
|
||||
width: got.width || row.width,
|
||||
height: got.height || row.height,
|
||||
body: got.body ?? row.body,
|
||||
direction: got.direction ?? row.direction,
|
||||
file: name ?? existing?.file ?? null,
|
||||
})
|
||||
}
|
||||
|
||||
const removed = []
|
||||
|
||||
if (vanished.length > 0) {
|
||||
for (const key of vanished) {
|
||||
removeSprite(held.get(key)?.file)
|
||||
removed.push(key)
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
await db.saveAssets(rows, {
|
||||
catalog: manifest.catalog,
|
||||
extractorVersion: manifest.extractorVersion,
|
||||
family: bridge.FAMILY,
|
||||
playerBodies: manifest.playerBodies,
|
||||
sources: { files: sources.files, extractorVersion: sources.extractorVersion },
|
||||
count: rows.length,
|
||||
})
|
||||
} catch (err) {
|
||||
return { status: 'failed', reason: err.message }
|
||||
}
|
||||
|
||||
const bodies = await resolveAtlasBodies()
|
||||
const art = await applyArt()
|
||||
|
||||
log.info('asset import applied', {
|
||||
assets: rows.length,
|
||||
fetched: fetched.assets.size,
|
||||
written,
|
||||
absent: fetched.missing.absent,
|
||||
bodies: bodies.resolved,
|
||||
art: art.applied,
|
||||
})
|
||||
|
||||
return {
|
||||
status: 'imported',
|
||||
catalog: manifest.catalog,
|
||||
extractorVersion: manifest.extractorVersion,
|
||||
assets: rows.length,
|
||||
fetched: fetched.assets.size,
|
||||
written,
|
||||
absent: fetched.missing.absent,
|
||||
unsupported: fetched.missing.unsupported,
|
||||
removed: removed.length,
|
||||
scanned: manifest.scanned,
|
||||
pages: manifest.pages,
|
||||
playerBodies: manifest.playerBodies,
|
||||
bodies,
|
||||
art,
|
||||
}
|
||||
}
|
||||
|
||||
function failure(err, what) {
|
||||
if (err instanceof bridge.AssetBridgeError) {
|
||||
return { status: 'unavailable', code: err.code, reason: err.message }
|
||||
}
|
||||
|
||||
log.warn(`asset import failed reading the ${what}`, { error: err.message })
|
||||
|
||||
return { status: 'failed', reason: err.message }
|
||||
}
|
||||
|
||||
// ── the body pass (§8) ─────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Ask the shard for a body id for every creature the atlas knows.
|
||||
*
|
||||
* `shard_spawn_creatures.name` is the ServUO class name — the atlas build picks
|
||||
* the winning spelling of the spawn TYPE token rather than inventing a display
|
||||
* name — so this needs no new column to ask its question.
|
||||
*
|
||||
* Never throws: a shard that goes down between the asset fetch and this pass
|
||||
* leaves the assets imported and the map as it was, which is a strictly better
|
||||
* state than failing the whole import back to nothing.
|
||||
*/
|
||||
async function resolveAtlasBodies() {
|
||||
let creatures = []
|
||||
|
||||
try {
|
||||
creatures = await atlasDb.allCreatureTypes()
|
||||
} catch (err) {
|
||||
return { resolved: 0, asked: 0, reason: err.message }
|
||||
}
|
||||
|
||||
if (creatures.length === 0) {
|
||||
return { resolved: 0, asked: 0, reason: 'the spawn atlas has no creatures loaded' }
|
||||
}
|
||||
|
||||
let rows
|
||||
|
||||
try {
|
||||
rows = await bridge.resolveBodies({ creatures })
|
||||
} catch (err) {
|
||||
return { resolved: 0, asked: creatures.length, reason: err.message }
|
||||
}
|
||||
|
||||
try {
|
||||
await db.replaceBodies(rows)
|
||||
} catch (err) {
|
||||
return { resolved: 0, asked: creatures.length, reason: err.message }
|
||||
}
|
||||
|
||||
const tally = { ok: 0, unknown: 0, notCreature: 0, failed: 0 }
|
||||
|
||||
for (const row of rows) {
|
||||
if (tally[row.status] === undefined) tally.failed++
|
||||
else tally[row.status]++
|
||||
}
|
||||
|
||||
return { asked: creatures.length, answered: rows.length, resolved: tally.ok, tally }
|
||||
}
|
||||
|
||||
// ── the derivation (§12) ───────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Point every atlas creature at its imported portrait.
|
||||
*
|
||||
* Two rules, and the second is the one worth stating:
|
||||
*
|
||||
* 1. The operator's `spawnAtlas.art.json` wins. Someone who drew their own
|
||||
* creature portraits must not have them replaced by a sprite rip.
|
||||
* 2. A slug with neither is set back to NULL rather than left alone. A creature
|
||||
* whose body stopped resolving — the operator removed a script package, say
|
||||
* — would otherwise keep pointing at a file that is about to be deleted, and
|
||||
* a broken image is worse than no image.
|
||||
*/
|
||||
async function applyArt() {
|
||||
const derived = await db.artBySlug()
|
||||
const operator = operatorArt()
|
||||
|
||||
const map = { ...derived, ...operator }
|
||||
|
||||
try {
|
||||
const applied = await atlasDb.setCreatureArt(map)
|
||||
return { applied, derived: Object.keys(derived).length, operator: Object.keys(operator).length }
|
||||
} catch (err) {
|
||||
log.warn('could not apply imported creature art', { error: err.message })
|
||||
return { applied: 0, derived: Object.keys(derived).length, error: err.message }
|
||||
}
|
||||
}
|
||||
|
||||
// ── status ─────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* What the admin panel renders: what is loaded, what the shard says, and whether
|
||||
* the two agree.
|
||||
*
|
||||
* Never throws and never fails a page: every branch that could — no shard, a
|
||||
* shard that is down, an asset plane the operator switched off — is a reported
|
||||
* state with a reason an operator can act on.
|
||||
*/
|
||||
async function getStatus() {
|
||||
const counts = await db.countAssets().catch(() => ({ total: 0, stored: 0 }))
|
||||
const bodies = await db.countBodies().catch(() => ({ total: 0, resolved: 0 }))
|
||||
const meta = await db.getMeta().catch(() => null)
|
||||
|
||||
const status = {
|
||||
loaded: {
|
||||
assets: counts.total,
|
||||
stored: counts.stored,
|
||||
creatures: bodies.total,
|
||||
resolved: bodies.resolved,
|
||||
catalog: meta?.catalog ?? null,
|
||||
extractorVersion: meta?.extractorVersion ?? null,
|
||||
importedAt: meta?.importedAt ?? null,
|
||||
},
|
||||
shard: null,
|
||||
drift: null,
|
||||
}
|
||||
|
||||
if (!(await shardLinked())) {
|
||||
status.reason = 'uo-link is not configured'
|
||||
return status
|
||||
}
|
||||
|
||||
try {
|
||||
const sources = await bridge.sourceFingerprint()
|
||||
|
||||
status.shard = {
|
||||
files: Object.keys(sources.files).length,
|
||||
extractorVersion: sources.extractorVersion,
|
||||
hashing: sources.hashing,
|
||||
complete: sources.complete,
|
||||
imaging: sources.imaging,
|
||||
}
|
||||
|
||||
status.drift = meta ? !bridge.sameSources(sources, meta.sources) : true
|
||||
} catch (err) {
|
||||
status.reason = err.message
|
||||
status.code = err instanceof bridge.AssetBridgeError ? err.code : 'UNAVAILABLE'
|
||||
}
|
||||
|
||||
return status
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
ART_SUBDIR,
|
||||
artDir,
|
||||
fileNameFor,
|
||||
importAssets,
|
||||
resolveAtlasBodies,
|
||||
applyArt,
|
||||
getStatus,
|
||||
}
|
||||
@@ -452,8 +452,64 @@ function listChampions({ facet = '' } = {}) {
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Every creature the atlas knows, as `{ slug, name }` (docs/link/v8.md §8).
|
||||
*
|
||||
* `name` is the ServUO CLASS NAME, not a display string invented here: the atlas
|
||||
* build picks the winning spelling of the spawn type token, so "GiantSpider" is
|
||||
* what the column holds and what `ScriptCompiler.FindTypeByName` will resolve.
|
||||
* That is the one property that lets the asset import ask its question without a
|
||||
* new column, and it is worth knowing before anyone "tidies" this into a
|
||||
* prettified label.
|
||||
*/
|
||||
async function allCreatureTypes() {
|
||||
return query('SELECT slug, name FROM shard_spawn_creatures ORDER BY slug')
|
||||
}
|
||||
|
||||
/**
|
||||
* Point creatures at their artwork, from a `{ slug: filename }` map.
|
||||
*
|
||||
* Everything NOT in the map is set back to NULL, which is deliberate: a creature
|
||||
* whose body stopped resolving must lose its portrait rather than keep pointing
|
||||
* at a file that is about to be deleted. A broken image is worse than no image,
|
||||
* and no image is the state the whole atlas UI was designed around.
|
||||
*
|
||||
* One transaction, and a single `CASE` update rather than a statement per slug —
|
||||
* at ~800 creatures the round trips are the cost, not the work.
|
||||
*/
|
||||
async function setCreatureArt(map) {
|
||||
const entries = Object.entries(map ?? {}).filter(
|
||||
([slug, file]) => typeof slug === 'string' && slug !== '' && typeof file === 'string' && file !== '',
|
||||
)
|
||||
|
||||
const conn = await core.pool.getConnection()
|
||||
|
||||
try {
|
||||
await conn.beginTransaction()
|
||||
await conn.query('UPDATE shard_spawn_creatures SET art = NULL WHERE art IS NOT NULL')
|
||||
|
||||
for (let i = 0; i < entries.length; i += BATCH) {
|
||||
await conn.batch(
|
||||
'UPDATE shard_spawn_creatures SET art = ? WHERE slug = ?',
|
||||
entries.slice(i, i + BATCH).map(([slug, file]) => [file, slug]),
|
||||
)
|
||||
}
|
||||
|
||||
await conn.commit()
|
||||
|
||||
return entries.length
|
||||
} catch (err) {
|
||||
await conn.rollback().catch(() => {})
|
||||
throw err
|
||||
} finally {
|
||||
conn.release()
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
replaceAtlas,
|
||||
allCreatureTypes,
|
||||
setCreatureArt,
|
||||
getMeta,
|
||||
getFacets,
|
||||
getPending,
|
||||
|
||||
@@ -107,8 +107,45 @@ function pointTypeRows(points) {
|
||||
return rows
|
||||
}
|
||||
|
||||
/**
|
||||
* The art each creature gets when the atlas is rebuilt.
|
||||
*
|
||||
* **`replaceAtlas` empties `shard_spawn_creatures` and refills it**, so anything
|
||||
* on that row is destroyed on every refresh — and a refresh happens on every
|
||||
* boot. Before protocol 8 that cost nothing: `art` came from a file on disk and
|
||||
* was simply re-read. As of phase 3 it can also come from an IMPORT, which is
|
||||
* expensive to obtain and whose gate (the shard's client-file hashes) would say
|
||||
* "unchanged" for weeks afterwards. So the imported values are re-derived here,
|
||||
* on the way past, rather than being restored by an import that has no reason to
|
||||
* run again.
|
||||
*
|
||||
* **The operator's map is spread last and therefore wins.** Someone who drew
|
||||
* their own creature portraits must not have them replaced by a sprite rip on the
|
||||
* next Update — the one property §12 states outright.
|
||||
*
|
||||
* Never throws: the asset tables are the newer half of this pair, and an atlas
|
||||
* refresh must not start failing because an asset query did. Losing the imported
|
||||
* art for one boot is recoverable by pressing Import; a boot that cannot rebuild
|
||||
* the atlas is not.
|
||||
*/
|
||||
async function artForAtlas() {
|
||||
const operator = loadArtMap()
|
||||
|
||||
try {
|
||||
// eslint-disable-next-line global-require
|
||||
const assetsDb = require('../shardAssets/shardAssets.db')
|
||||
const derived = await assetsDb.artBySlug()
|
||||
return { ...derived, ...operator }
|
||||
} catch (err) {
|
||||
log.warn('imported creature art could not be read; using the operator map alone', {
|
||||
error: err.message,
|
||||
})
|
||||
return operator
|
||||
}
|
||||
}
|
||||
|
||||
async function applyAtlas(atlas) {
|
||||
return db.replaceAtlas({ ...atlas, pointTypes: pointTypeRows(atlas.points) }, loadArtMap())
|
||||
return db.replaceAtlas({ ...atlas, pointTypes: pointTypeRows(atlas.points) }, await artForAtlas())
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
Reference in New Issue
Block a user