feat(assets): the website half of the Asset Bridge (Asset Bridge cutover, 3 of 5) #44
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: RunicGateway/Module-uo#44
Reference in New Issue
Block a user
No description provided.
Delete Branch "edge"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Step 3 of 5 of the Asset Bridge
edge → maincutover (docs/link/v8.md§16 row 9b). Mergeafter
website#202;link#44+servuo-plugins#36are the pair that has to land together.#43 is in —
mainis now fully contained inedge(0 commits ahead), so this no longerreverts the released
UniqueIdfix.What comes over
Seven phases, 46 files, +10,169 / −191, and everything the website does with a client file:
artcolumn filled and renderedAdmin → Client Files— the panel that operates all three importsNew tables
shard_assets,shard_creature_bodies,shard_asset_meta;uo_link_config.protocoldefaults to 8.
Two things an operator gets from this release that are not Asset Bridge features
shard_spawn_points.unique_idfinally reaches existing installs. Events phase 12b added itto the
CREATE TABLEand nowhere else, so it landed on fresh installs and no upgraded one —CREATE TABLE IF NOT EXISTSnever adds a column. Every spawn-atlas import on an upgraded installhas failed outright since v1.2.0 with
Unknown column 'unique_id': no bestiary refresh, nospawn map, no champion altars. The
ADD COLUMN IF NOT EXISTSline ships here (org lead's call at9a, weighing it against a hotfix to
main).SPAWN_ATLAS.mdused to require thewebsite — the one internet-facing component — to read the ServUO tree over a bind mount. That
was the single place "only the sidecar bridges the shard" was broken, and it is closed.
PARSER_VERSIONis 6, not 5Both branches bumped 4 → 5 for different derivations and main's 5 is released (v1.2.2, the
UniqueIdfix). Had the merge kept 5, every install that imported under v1.2.2 would already store5,
currentParserwould call a phase-7 build current, and phase 7's canonical read order would havereached nobody already running. Resolved in #43 with both meanings kept in the version log.
The pin stays put
ci/core-ref.jsonnames websitemain655fbf3f.website#202moves core'smainbut notMODULE_API_VERSION, and the pin asserts an API version rather than a tip — so the frozen-manifestjob keeps answering, and the Integration kit stays green through the window. Org lead's call,
2026-09-14.
Verification
npm test --prefix server— 723 pass, 0 failnpm run build --prefix clientthennpm test --prefix client— 45 pass, 0 failserver-tests,client-buildandfrozen-manifestall green3.18 s, 67,496 names in 1.42 s, 313 item pictures in 1.38 s, the atlas over the bridge
in ~2 s, an update with no drift in 0.99 s
AI disclosure
Authored with Claude Code (Claude Opus 5). Commits carry the
Co-Authored-Bytrailer.🤖 Generated with Claude Code
https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
The base cliloc table now comes over the bridge. `clilocBridge.js` walks `GET /cliloc` page by page and the model merges the `custom/` overlays over it — overlays stay on disk because ServUO has no server-side notion of a custom cliloc, so there is nothing on the shard to ask for. **The shard wins whenever uo-link is configured and enabled**, with no mode setting: there is no version of "which source?" an operator benefits from answering. A file on disk remains the source only where there is no shard link, plus a one-off explicit `path` — deprecated, not removed, and unchanged. **Boot no longer imports on the bridge.** The file path could hash 5 MB locally and skip in 14 ms; a shard round trip in the boot sequence would be spent answering "no" on every restart but the one after a client patch — and patching a client is an operator action, so importing became one. Admin → Shard → Import. Whatever table is loaded keeps serving until then. Three checks in the walk, each for a way a shard can hand back a table that looks complete: * only `cut: 'end'` finishes it — a short page can equally be a spent budget, and a truncated table renders some items named and some not, which is exactly what NO table looks like; * the cursor must advance, or the walk stops rather than spinning; * every page echoes the source's size and mtime, so a client patched mid-import is refused outright rather than stitched from two files. **The base is exempt from the vanished-source rule**, which is an upgrade detail rather than a preference: an install that used the file pipeline carries its base file's label in the stored fingerprint, and on the bridge that label is *supposed* to disappear. Counting it as vanished would demand an approval for a change the upgrade itself made. Overlays keep the rule in full. **The protocol pin moves 7 → 8** — the third declaration site, and the one nothing enforces. Phase 1 moved the sidecar and the overlay together because the installer refuses a mismatched bundle; this one has to be moved by hand, in the phase that first calls a protocol-8 route. The schema block above it is the record of what forgetting costs: two phases of every REST call answered 409. Verified against a live shard, sidecar and site: 12 pages, 67,496 rows imported in 1.68 s, the operator's three-row overlay overriding stock strings on top of it, and the next import correctly `unchanged`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4Until now the only way a creature got a picture on this site was for an operator to open UOFiddler on a desktop, export sprites by hand, copy them to the web host and write a spawnAtlas.art.json naming each one. Almost nobody did, so shard_spawn_creatures.art was NULL on every install. The shard has had those files the whole time. Admin -> Shard -> Import now walks its asset manifest, fetches only the sprites whose hash changed, writes them under uploads/atlas/, asks the shard for a body id per atlas creature (§8: it CONSTRUCTS the creature and reads Body.BodyID, which is the only thing that is right for a shard's own custom creatures) and points each creature at its picture. On a stock client that is 787 portraits, about a megabyte. **The one thing v8.md §12 got wrong, and it is not cosmetic.** It says `shard_spawn_creatures.art` "starts being filled by the import". That table is emptied and refilled by replaceAtlas on EVERY atlas refresh, and a refresh runs on every boot -- so a filename stored there would be destroyed by an ordinary re-parse of the ServUO tree, with the next Update finding the client files unchanged, reporting "nothing to do", and never restoring it. Nothing would report a fault; the pictures would just be gone. So the assets and the body map live in their own tables outside that blast radius, and applyAtlas re-derives `art` on the way past as `{ ...derived, ...operatorMap }` -- which is also the one place "the operator's own artwork wins" is enforced, on every rebuild rather than only at import. Smaller decisions worth not rediscovering: - The derivation joins on the catalogue KEY, not on the body id. The simpler join is correct today and stops being correct the moment phase 6 adds body/400/a2/f0, at which point one slug matches dozens of rows. - Filenames are content-addressed. A stable name overwritten in place leaves every browser and CDN serving the previous client's sprite, with the database row perfectly correct. - An unchanged key whose FILE is missing is fetched again. The row and the disk can disagree (a wiped uploads volume, a restore from a dump), and a broken image on a creature page is worse than one re-fetched sprite. - A key the shard cannot render is not a failure. Two thirds of the playable ghost and gargoyle bodies have no art on a stock client, and an import that reported eight failures every time would teach an operator to ignore the panel. - A key that VANISHED from the manifest needs review before anything changes: an unmounted client volume and a deliberate downgrade look identical here. 23 new tests; 674 server and 42 client tests pass. The SQL was also run against a real MariaDB, which is what proved the CONCAT join and the singleton CHECK. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4The shard's catalogue can now answer for 73 bodies it used to report absent — they have no art at action 0 and real art at a later one, and their key says which (`body/820/a23` is a horse). This side stores that action and stops assuming `a0` anywhere. The atlas join is the part that mattered. It read a.asset_key = CONCAT('body/', b.body, '/a0') which would have silently dropped exactly the creatures this phase adds. It now reads the row's own action, with COALESCE for rows written before the column existed — a NULL inside CONCAT makes the whole comparison NULL, which would have taken every portrait off the site on upgrade with the database perfectly correct and nothing in any log. It still matches at most one row per slug: a deeper key (`body/820/a23/f4`) does not equal the catalogue key. Verified against a real MariaDB with the live shard's own 1,095-row manifest: the ALTER applies to an installed-shape table and is idempotent, the horse joins to its a23 picture, a pre-phase-6 NULL-action row keeps its portrait, and a stored frame key does not become a second candidate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4`spawnAtlasSource.js` gains a second backend behind its existing interface (docs/link/v8.md 10). Where a shard is linked and enabled the tree arrives over the sidecar; where there is none, a local ServUO tree is read exactly as before. An explicit --servuo path is an instruction and overrules both. The parsers do not move. spawnAtlasParse.js is still pure, still fs-free and still CI-covered without a ServUO tree anywhere near it; `buildFromFiles` is now where the parse starts, and both readers feed it the same shape. treeBridge.js walks the manifest and then the chunks. Three of its checks are not decoration -- each is a way this ends in a tree that LOOKS imported, and XML is forgiving enough that a mis-assembled spawn file parses cleanly and simply has fewer spawns in it: - every chunk re-declares its address and carries the hash of its own uncompressed bytes, and chunks are placed by declared index rather than arrival order - the whole file is hashed after reassembly against its manifest row - the catalog must not move mid-walk, or the import is refused rather than stitched out of two trees Boot does not call the shard. The same answer 17.7 gave the cliloc table, and the same reasoning: a local tree hashes in ~120 ms and skips, while a round trip in the boot sequence would answer "no" on every restart that did not follow a map edit. Editing spawn files is an operator action, so importing is one -- Admin -> Spawn Atlas -> Import. What that costs is real and is said out loud in the panel, the CLI and the log: an install on the bridge has NO automatic refresh at all. Two things the live walk found that the unit tests could not: - PARSER_VERSION 4 -> 5. The parse is order-sensitive in one place -- the decoration index keeps the FIRST item id it sees for a type -- and the two readers agreed on a stock tree by coincidence, since the filesystem reader walks each directory with localeCompare while the shard sorts whole relative paths. buildFromFiles now sorts by label, ordinally, once, whatever order the files arrived in. Identical input, a different answer for a handful of types: exactly what the version number exists to push through the hash gate. The parity test asserted deepEqual, which ignores key order; it now asserts serialised equality too. - The source fingerprint is taken over RAW BYTES at both ends. Hashing decoded text hashes a UTF-8 re-encoding -- identical for valid UTF-8, different for a file that is not, because an undecodable byte becomes U+FFFD and never comes back. One Latin-1 character in a creature name would have made the drift gate report a change on every import, forever, with the tree untouched. A 200 from assets.sources also stopped meaning "the client files are on offer": a shard may now serve its configuration tree while declining to serve its UO client. Both client-file readers check `assetsEnabled` and say DISABLED, instead of reading an empty file list as "your client has no cliloc.enu" and sending an operator to their client install for a setting that lives on their shard. Measured end to end against a live shard and the real sidecar: 141 files, 11.9 MB, 158 chunks, 3 pages, 1.33 MB on the wire, 512 ms; every file byte-identical to disk; and the atlas built over the bridge identical to the one built off it -- 6,455 points, 800 creatures, 387 regions, 558 landmarks, 25 champions, 309 decoration types. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4Admin -> Client Files: one page over the three things that come out of the operator's UO client -- creature portraits, item and land pictures, and the cliloc table. One page rather than three because they are one job: same client install, same bridge, and all of them change at the same moment, when the operator patches that client. Boot never asks the shard for any of it, so these buttons are the only thing that imports. The cliloc pair had had no UI at all since phase 2. On a bridged install, where boot deliberately stopped calling the shard, that meant `curl` was the only way to load 67,496 names. Update and Re-import everything are section 6's two stages as two buttons rather than one button and a checkbox, because they cost wildly different things. A vanished key is reviewed in the page and not in a table -- an asset import only happens because someone pressed a button here, so the review is already in front of the person who caused it -- and it shows each key's PICTURE, since `body/820/a23` names nothing a human recognises. `shard_asset_meta` gained a `last` block (what the import did, who ran it) so the panel can answer "did last week's import do anything" without scrolling core's whole activity log. The live walk against a real shard imported 1,095 portraits in 3.5 s, warmed 313 item pictures in 0.6 s and reloaded 67,496 cliloc rows in 1.7 s -- and found two DELETIONS that predate this phase and that no test could see, because only a screen showing the numbers together makes them visible: * The body import diffed its manifest against every family's rows. Phase 5 put item and land art in the same table, and a body manifest never mentions them, so all 313 item pictures were staged for deletion with a sentence saying the shard had stopped offering them. * An approved vanish unlinked the sprite and kept the row. The catalogue went on counting a picture that was gone, the atlas could point a creature page at a missing file, and the next forced import offered the same key for review again -- reporting "nothing was changed" about a file it had deleted. Both fixed here, with the removals now inside `saveAssets`'s own transaction. The same whole-table read made the panel announce a 1,408-row creature catalogue on an install holding 1,095 portraits and 313 item pictures. Protocol stays 8 and EXTRACTOR_VERSION stays 3: nothing on the wire changed. Refs: docs/link/v8.md sections 12.2, 14, 16 (phase 8) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4Two defects the Phase 9 acceptance walk found on a real rig, one of them ours and one of them released (docs/link/v8.md §17.14). ## "The shard is not answering for client files" about a shard that was fine The status call exhausts its 425 backoff whenever something else holds the shard's single asset slot -- an import the operator started, or the item-art warm pass refilling itself after a client patch. Phase 8's panel rendered that with the same banner as a shard that is down or has the plane switched off, and left it standing, because the page only re-reads after an action. On the rig it was up for a quarter of an hour while the warm pass refilled 313 pictures and every direct call to the same route answered normally. BUSY now says what it is, and one automatic re-read four seconds later clears the ordinary case. One per mount, guarded by a ref: a page that retried forever would be holding the slot it is waiting for. DOWN, DISABLED and NO_IMAGING read exactly as they did. ## Every spawn-atlas import on an upgraded install has failed since v1.2.0 `shard_spawn_points.unique_id` (Events Phase 12b) was added to the CREATE TABLE and nowhere else. `CREATE TABLE IF NOT EXISTS` does not add a column to a table that already exists -- which is what the twenty-odd `ADD COLUMN IF NOT EXISTS` lines in this same file are for -- so it reached fresh installs and no existing one, and `replaceAtlas` inserts the column unconditionally: Unknown column 'unique_id' in 'INSERT INTO' No bestiary refresh, no spawn map, no champion altars, on every install whose tables predate 12b. A fresh install cannot reproduce it and neither can a test whose schema is this file applied to an empty database; it took a rig with old tables. Org lead, weighing that it is already released: it ships here on edge rather than as a hotfix to main. Verified by dropping the column, rebooting, watching the replay put it back, and importing 6,455 spawners over the bridge. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4mainintoedgebefore the Asset Bridge cutovermainintoedgebefore the Asset Bridge cutover (Phase 9b, 0 of 5)' (#43) from chore/cutover-sync-main into edge