RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(link): the catalogue is real, and UOFiddler's last job is gone (Phase 3)

Browse Source 
Phase 3 is built and walked on a live shard. What the walk measured, and the
two places the design of record needed correcting:

§8.1, new: the catalogue is 787 exactly as §4.8 predicted, and the whole scan
of bodies 1-2047 takes 734 ms cold -- so the wall-clock paging §11 designed
never fires on this client. Every §4.8/§5.2 prediction held when the bytes were
rendered and LOOKED at: 320, 607, 608 and 666 come back absent rather than as
another creature's picture, and the direction split is 783 at index 1 against 4
at index 0 -- four player bodies, not six.

44 of the 787 hashes are shared by two or three bodies, which is the exact
signature of the wrong-picture bug, so it was chased rather than assumed. It is
the client's own Body.def aliasing (83 {1}, 84 {1}, 106 {12, 59}), and the check
that settles it is at the source: Translate(ref body, ref hue) rewrites `body`
only when bit 31 is set, unlike the one-argument overload -- and ResolveAnimation
calls that same two-argument overload, so validator and decoder resolve the
identical record.

§12.1, new: **§12 is right about the outcome and wrong about the mechanism.**
`shard_spawn_creatures` is emptied and refilled by every atlas refresh, and a
refresh runs on every boot -- so an imported filename written to that row is
destroyed by an ordinary re-parse of the ServUO tree, and the next Update finds
the client files unchanged and never restores it. Three tables outside that
blast radius, and the atlas import re-derives `art` on the way past.

§14: **§16 listed phase 3 as servuo-plugins + module-uo and that was wrong.**
web.rs routes every command explicitly, so `link` is in the phase. Corrected in
both places.

UOFIDDLER.md is DELETED, two phases earlier than §9.1 predicted -- creature art
was the only thing still on it. SPAWN_ATLAS.md §Artwork is rewritten around the
import, keeping the operator's own map as the thing that wins; the module's
SCHEMA.md gains the three tables and API.md the two admin routes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
claude
2026-09-10 18:41:08 -05:00
parent 4c6b0c566a
commit 1a7481e9f4
7 changed files with 195 additions and 149 deletions
Download Patch File Download Diff File Expand all files Collapse all files

109
link/v8.md
View File

@@ -10,8 +10,8 @@ direction not five, ServUO's own decoders, the UOP reader for player bodies, the
requirement). The phase 0 spike ran on 2026-09-10 and its findings are §4.5-§4.7 -- §4's decision
stands, and the response to a malformed record is now measured rather than proposed. **Nothing in
§17 is open.**
**Supersedes the manual half of:** [`../website/UOFIDDLER.md`](../website/UOFIDDLER.md),
[`../website/CLILOCS.md`](../website/CLILOCS.md) §Converting,
**Supersedes the manual half of:** `../website/UOFIDDLER.md` (deleted in phase 3, once creature
art stopped needing it), [`../website/CLILOCS.md`](../website/CLILOCS.md) §Converting,
[`../website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md) §Artwork and §Configuring the tree.
Two features on this platform read data that only exists inside a UO client, and today both reach
@@ -695,6 +695,46 @@ Constructing arbitrary creature types has side effects: constructors pack items,
timers. The mitigations are per-type `try`/`catch`, immediate `Delete()`, small batches, and the
fact that the whole pass is admin-triggered rather than something that runs at boot.
### 8.1 What phase 3 measured
**Built and walked 2026-09-10** against a live ServUO with a real world (43,000 mobiles, 210,000
items) and the real sidecar.
**The catalogue is 787, exactly as §4.8 predicted**, and it arrives in one page: the whole scan of
bodies 1–2047 — index validation, 787 decodes, 787 PNG encodes and 787 SHA-256s — took **734 ms**
cold. That is well inside the 3 s scan budget, so the wall-clock paging §11 was designed for never
fired on this client. It stays, because the budget is what keeps a slower host or a larger family
inside the 10 s reply timeout rather than producing replies that are always thrown away.
Every prediction in §4.8 and §5.2 held when the bytes were actually rendered and looked at:
| Asked for | Answer | Why it matters |
|---|---|---|
| `body/320/a0` (`length 0`) | **absent** | The 357-class bug. The library would have returned the previously-decoded creature |
| `body/607/a0`, `body/608/a0` (elf ghosts) | **absent** | §4.8's two rows that moved; the library returns the elf female here |
| `body/666/a0` (gargoyle → `anim5`) | **absent** | The spider trap. Nothing swept, nothing found |
| `body/400/a0/…` deeper key | **unsupported** | Well-formed under §5, not served until phase 6 |
| direction distribution across 787 | **783 at index 1, 4 at index 0** | Four player bodies, not six — §4.8 again, from `Race.AllRaces` rather than a table |
**44 of the 787 hashes are shared by two or three bodies, and that is correct.** It is the exact
signature the wrong-picture bug produces, so it was chased rather than assumed: the client's own
`Body.def` says `83 {1}`, `84 {1}`, `138 {7}`, `139 {7}`, `106 {12, 59}`, and the sharing groups
match those lines rather than being runs of consecutive ids (which is what a reused stream buffer
produces). The distinguishing check is at the source: `Animations.Translate(ref body, ref hue)`
rewrites `body` **only when bit 31 of the table entry is set**, unlike the one-argument overload
which always does — and `BridgeAssetValidator.ResolveAnimation` calls that same two-argument
overload, matching `GetAnimation(…, preserveHue: false, …)`. Validator and decoder therefore
resolve the identical record, which is the property the whole §4.5 design rests on.
**The Core-thread pass costs about 190 ms per 100 types.** All 455 stock `Scripts/Mobiles/Normal`
classes were constructed and deleted in five chunks, producing every status the protocol defines
(`ok` 436, `unknown` 11, `notCreature` 4, `failed` 4). The world's mobile count went from 43,000 at
boot to 42,924 afterwards and its item count fell too — so `Delete()` is reclaiming the packed
inventory as well as the mobile, and nothing leaked.
The three refusals answer as designed end to end: an unknown `family` is **400**, a stale `catalog`
on a fetch is **422**, and 101 types in one `assets.bodies` is **400** naming the cap.
---
## 9. The cliloc decompressor is ours now
@@ -712,7 +752,8 @@ Protocol 8 writes rather than calls (§4) — and from that point:
- No operator runs `dotnet build` on a converter.
- No operator copies a 5 MB file to a server.
- `website/server/tools/cliloc-export/` is retired, and `UOFIDDLER.md` is deleted rather than
rewritten.
rewritten. (Phase 2 deleted its Part 1; **phase 3 deleted the page**, a phase earlier than that
section predicted, because creature art was the only thing left on it.)
### 9.1 What phase 2 built, and what the port cost
@@ -900,6 +941,46 @@ extracted on their own host, for their own shard. Nothing is committed, nothing
and nothing is redistributed. What changes is only that the extraction stopped requiring a GUI on a
desktop.
### 12.1 The one thing above that phase 3 had to build differently
"`shard_spawn_creatures.art` … starts being filled by the import" is right about the outcome and
wrong about the mechanism, and the difference is not cosmetic.
**That table is emptied and refilled by every atlas refresh** — `shardAtlas.db.js`'s `replaceAtlas`
DELETEs all six atlas tables inside one transaction — and a refresh runs on **every boot**. Before
protocol 8 that cost nothing, because `art` came from a file on disk and was simply re-read each
time. An imported sprite is the opposite: expensive to obtain, and gated on client-file hashes that
would say "unchanged" for weeks afterwards. Writing it onto the creature row would mean an ordinary
re-parse of the ServUO tree silently deleting every portrait, with the next Update reporting
*nothing to do* and never restoring them. Nothing would report a fault; the pictures would just be
gone.
So phase 3 built three tables outside that blast radius, and the atlas import reads them on the way
past (org lead, 2026-09-10):
| Table | Holds | Lifetime |
|---|---|---|
| `shard_assets` | `asset_key`, `sha256`, `bytes`, `width`, `height`, `body`, `direction`, `file` | Upserted per key; only an approved vanish deletes |
| `shard_creature_bodies` | `slug`, `type_name`, `body`, `status` | Replaced whole — it is derived from the atlas's creature list, so a slug that has left has no meaning |
| `shard_asset_meta` | The singleton an Update compares against | Replaced |
`replaceAtlas` now takes `{ ...derived, ...operatorMap }`, which is where "the operator's map wins"
is actually enforced — one spread, in one place, applied on every rebuild rather than only at
import time.
**Two details worth not rediscovering.** The derivation joins on the catalogue key
(`a.asset_key = CONCAT('body/', b.body, '/a0')`), not on `a.body = b.body`: today one body has
exactly one asset and the simpler join is correct, and it stops being correct the moment phase 6
adds `body/400/a2/f0`, at which point one slug matches dozens of rows and whichever the engine
returned last becomes the portrait. And the stored filename is **content-addressed**
(`uo-body-34-a0-<sha8>.png`), because a stable name overwritten in place leaves every browser and
CDN serving last month's client's sprite from cache with the database row perfectly correct.
`shard_creature_bodies` also answers §8 without a schema change on the atlas side:
`shard_spawn_creatures.name` already holds the ServUO **class name** — the atlas build picks the
winning spelling of the spawn type token rather than inventing a display label — so the import has
something `ScriptCompiler.FindTypeByName` will resolve without storing it twice.
---
## 13. Visibility
@@ -921,9 +1002,9 @@ new pipe.
| Command | Reply | Purpose | Built |
|---|---|---|---|
| `assets.sources` | `assets.sources.ok` | Stage 1: client file manifest + `EXTRACTOR_VERSION` | **phase 1** |
| `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes}]`, paged | phase 3 |
| `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged | phase 3 |
| `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) | phase 3 |
| `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes, width, height, body, direction}]`, paged | **phase 3** |
| `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged; base64 PNG per row | **phase 3** |
| `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) | **phase 3** |
| `cliloc.table` | `cliloc.table.ok` | The decompressed table, paged (`?lang=`, `?cursor=`) | **phase 2** |
| `tree.manifest` / `tree.fetch` | `.ok` | §10, the ServUO tree files | phase 7 |
@@ -938,9 +1019,21 @@ ordinary answer mid-import rather than a rare one) and reads the refusal's `code
**403** disabled, **404** no such file, **422** a file the shard cannot decode, **503** a shard that
cannot do this now (§9.1).
Phase 3 added the other three: `GET /assets/manifest?family=&cursor=`, `POST /assets/fetch` and
`POST /assets/bodies`. **The two POSTs are reads**, and the method is the request body rather than a
side effect — a few hundred asset keys do not belong in a query string. They are the only reads on
this link that take one. `422` gains a second meaning on this plane alongside "cannot decode": the
mid-import guard, a `catalog` that no longer describes the files on disk.
**§16 listed phase 3 as `servuo-plugins, module-uo` and that was wrong** — `web.rs` routes every
command explicitly and has no generic `/assets/*` forwarder, so `link` is in the phase too. The
table now says so.
**Website admin** (`Admin → Shard`, admin-only): status, **Import**, **Update**, approve/reject for
a vanished key, and the existing path settings. Every action to the admin activity log, as
`shard.assets.*`.
`shard.assets.*`. Phase 3 shipped the two that make it reachable — `GET /admin/shard/assets` and
`POST /admin/shard/assets/import`, mirroring the cliloc pair — so the phase could be accepted on a
real rig; the full panel is phase 8 (org lead, 2026-09-10).
---
@@ -970,7 +1063,7 @@ disagree, so a split bump means the next bundle silently fails to compose.
| 0 | **DONE 2026-09-10.** Spike: the vendored decoders driven from inside a running ServUO, over a client broken 21 ways. §4 stands; the finding was 22,102 wrong pictures on a *stock* client, and the validator that answers them (§4.5-§4.7) | servuo-plugins |
| 1 | **DONE 2026-09-10.** The transport: `assets.sources`, the single-slot gate (§3.2), the paging envelope (§3.4), the sidecar line cap (§3.3), `EXTRACTOR_VERSION`, `NO_IMAGING`, protocol 7→8. Plus §4.5's validator promoted into the overlay and extended to animations — which found **357 wrong pictures in the body catalogue on a stock client** and cut it from 1,144 to 787 (§4.8) | servuo-plugins, link |
| 2 | **DONE 2026-09-10.** Clilocs end to end (§9.1, §9.2): the Mythic decompressor ported into the overlay, `cliloc.table` + `GET /cliloc`, the paging walk and the source switch on the website, module-uo's protocol pin 7→8. `cliloc-export/` deleted and `UOFIDDLER.md` §Part 1 with it. **67,496 rows, 290 ms, ~11 pages** — the same count UOFiddler's own DLL produced from this client | all |
| 3 | Body resolution (§8) + the **787**-body catalogue (§4.8); `shard_spawn_creatures.art` filled | servuo-plugins, module-uo |
| 3 | **DONE 2026-09-10.** Body resolution (§8) + the **787**-body catalogue (§4.8), `assets.manifest` / `assets.fetch` / `assets.bodies` and their REST mirrors, `shard_spawn_creatures.art` filled and rendered (§8.1, §12.1). **787 rows in one 734 ms page; 455 types resolved at ~190 ms per 100 on the Core thread; zero mobiles leaked.** `UOFIDDLER.md` deleted, two phases early | servuo-plugins, **link**, module-uo |
| 4 | **The UOP animation decoder (§4.3)** — the **eight** player bodies the legacy path cannot reach (§4.8 moved the elf ghosts across), the never-sweep rule already enforced in code by phase 1's `ResolveAnimation` | servuo-plugins |
| 5 | Item statics and land on demand, hued keys, the cache | servuo-plugins, module-uo |
| 6 | Deep animation keys (`body/<id>/a<n>/f<n>`) for the future project, plus the bulk-fill switch | servuo-plugins, module-uo |