docs(link): two of the eight player bodies existed, and 233 nobody asked about (Phase 4) #239

Merged
whitlocktech merged 1 commits from docs/asset-bridge-p4 into main 2026-09-11 10:09:41 +00:00
3 changed files with 232 additions and 61 deletions

View File

@@ -4,12 +4,14 @@
branches (`feat/asset-bridge-p<n>`), and conversation. The protocol number is **8** and this file is branches (`feat/asset-bridge-p<n>`), and conversation. The protocol number is **8** and this file is
`docs/link/v8.md`. `docs/link/v8.md`.
**Status:** Design of record, **phase 0 complete**. Approved in principle 2026-09-09 **Status:** Design of record, **phases 0-4 complete** (§16). Approved in principle 2026-09-09
(architecture, asset scope, built-in cliloc decoder, atlas cleanup); refined 2026-09-10 (one (architecture, asset scope, built-in cliloc decoder, atlas cleanup); refined 2026-09-10 (one
direction not five, ServUO's own decoders, the UOP reader for player bodies, the libgdiplus direction not five, ServUO's own decoders, the UOP reader, the libgdiplus requirement). The phase 0
requirement). The phase 0 spike ran on 2026-09-10 and its findings are §4.5-§4.7 -- §4's decision spike ran on 2026-09-10 and its findings are §4.5-§4.7 -- §4's decision stands, and the response to
stands, and the response to a malformed record is now measured rather than proposed. **Nothing in a malformed record is now measured rather than proposed. Phase 4 built the one decoder §4.3
§17 is open.** reserved, and §4.9 is what it measured: **two** of the eight player bodies it was scoped to exist
at all, and the same fallback found **233** other bodies, taking the catalogue to **1,022**.
**Nothing in §17 is open.**
**Supersedes the manual half of:** `../website/UOFIDDLER.md` (deleted in phase 3, once creature **Supersedes the manual half of:** `../website/UOFIDDLER.md` (deleted in phase 3, once creature
art stopped needing it), [`../website/CLILOCS.md`](../website/CLILOCS.md) §Converting, art stopped needing it), [`../website/CLILOCS.md`](../website/CLILOCS.md) §Converting,
[`../website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md) §Artwork and §Configuring the tree. [`../website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md) §Artwork and §Configuring the tree.
@@ -263,12 +265,15 @@ Three costs come with it, all known and none of them blocking:
The residual risk that remains is a patched or custom client tripping an out-of-bounds read on a The residual risk that remains is a patched or custom client tripping an out-of-bounds read on a
path we *do* call. §16's phase 0 is where that gets exercised rather than assumed. path we *do* call. §16's phase 0 is where that gets exercised rather than assumed.
### 4.3 One decoder we do write: UOP animation, for the player bodies ### 4.3 One decoder we do write: UOP animation
The six missing bodies are the player character, and the scope is player models, so they get a **Built in phase 4. What it reaches is not what this section predicted, and §4.9 is the
decoder rather than a caveat. It is deliberately the **narrowest possible** addition: a reader for measurement.** The section stands as the argument; the numbers below are the corrected ones.
`AnimationFrame*.uop`, used *only* for bodies the legacy path cannot resolve. Everything the
vendored code can already decode keeps going through the vendored code. The bodies the legacy path cannot resolve get a decoder rather than a caveat. It is deliberately
the **narrowest possible** addition: a reader for `AnimationFrame*.uop`, used *only* where the
vendored code has nothing. Everything it can already decode keeps going through it — which after
phase 4 is still 787 of the catalogue's 1,022 bodies.
This client ships `AnimationFrame1/2/3/4/6.uop` (107, 118, 253, 115 and 24 MB) plus This client ships `AnimationFrame1/2/3/4/6.uop` (107, 118, 253, 115 and 24 MB) plus
`AnimationSequence.uop`. ServUO's `FileIndex` already contains a UOP reader — but `Animations` `AnimationSequence.uop`. ServUO's `FileIndex` already contains a UOP reader — but `Animations`
@@ -277,12 +282,18 @@ mul record, so wiring the existing `FileIndex` in is not sufficient. Two license
references exist to work from: ClassicUO's animation loader (GPL-3, and we are GPL-3-or-later) and references exist to work from: ClassicUO's animation loader (GPL-3, and we are GPL-3-or-later) and
UOFiddler 4.22 (Beerware, already established in §9). UOFiddler 4.22 (Beerware, already established in §9).
**What it was for, and what it actually delivers.** The phase was scoped to the eight player
bodies §4.8 left undecodable. **Two of the eight are in the client at all** — gargoyles 666 and
667, in `AnimationFrame3.uop`. The six ghost bodies are in no package, and §4.9 shows how that was
established rather than assumed. What the same fallback *does* deliver is **233 further bodies**
the catalogue had nothing for, so the phase's real effect is 787 → **1,022**.
**The trap this must not fall into, and it is why the phase exists at all.** `Bodyconv.def` maps **The trap this must not fall into, and it is why the phase exists at all.** `Bodyconv.def` maps
gargoyle 666 to `anim5`, and `BodyConverter.Convert` faithfully returns fileType 5 — where the gargoyle 666 to `anim5`, and `BodyConverter.Convert` faithfully returns fileType 5 — where this
client has nothing. Asking the *other* anim files for index 666 does not fail. It returns 175 client's index runs out before the entry. Asking the *other* anim files for index 666 does not
decodable action/direction combinations of **a giant spider**, because something unrelated occupies fail. It returns 175 decodable action/direction combinations of **a giant spider**, because
that index in `anim2.mul`; fileTypes 3 and 4 return misaligned colour fragments. Rendered and something unrelated occupies that index in `anim2.mul`; fileTypes 3 and 4 return misaligned colour
confirmed. fragments. Rendered and confirmed.
So the extractor takes `BodyConverter.Convert`'s answer and, if that yields nothing, reports So the extractor takes `BodyConverter.Convert`'s answer and, if that yields nothing, reports
nothing. **It must never sweep file types looking for a hit** — that does not find missing art, it nothing. **It must never sweep file types looking for a hit** — that does not find missing art, it
@@ -290,6 +301,14 @@ silently puts a spider on the gargoyle page, and no error is raised anywhere. A
correct behaviour; a plausible wrong picture is the failure this protocol most needs to avoid, correct behaviour; a plausible wrong picture is the failure this protocol most needs to avoid,
because nothing downstream can detect it. because nothing downstream can detect it.
**Searching all five UOP packages for one body is not that, and the difference is structural.** A
legacy index is addressed **by position**: nothing in the file says which body a record belongs to,
so a wrong lookup returns a confident picture of something else. A UOP entry is addressed by the
**hash of a name that contains the body id** — `build/animationlegacyframe/000666/00.bin` — and the
payload then declares that id again in its own header, which the reader checks against the id it
asked for. A hit is proof of identity rather than a coincidence of position. Measured: no hash
appears in two packages.
### 4.4 Requirement: a Linux shard host needs libgdiplus ### 4.4 Requirement: a Linux shard host needs libgdiplus
**Stated prerequisite, not a soft recommendation.** ServUO targets `net48`, so on Linux it runs under **Stated prerequisite, not a soft recommendation.** ServUO targets `net48`, so on Linux it runs under
@@ -314,7 +333,9 @@ Upstream is <https://github.com/mono/libgdiplus>, with the Mono project's own pa
read-only.** Distributions still package and patch it, so `apt-get install libgdiplus` is a normal, read-only.** Distributions still package and patch it, so `apt-get install libgdiplus` is a normal,
supported thing to do today — but upstream is not maintained. It is the strongest long-term argument supported thing to do today — but upstream is not maintained. It is the strongest long-term argument
for eventually moving extraction off `System.Drawing`, and phase 4's UOP reader (§4.3) is written for eventually moving extraction off `System.Drawing`, and phase 4's UOP reader (§4.3) is written
without it precisely so that door stays open. It does not change the decision now. without it precisely so that door stays open — decode and encode both (§4.9). It does not change
the decision now, and §17.9 is the decision not to walk through that door yet: a host with no
libgdiplus still gets the flat refusal below rather than the ~238 bodies that no longer need it.
**How its absence must present.** Never a stack trace and never a 500. Missing libgdiplus is a **How its absence must present.** Never a stack trace and never a 500. Missing libgdiplus is a
named, actionable outcome in the same family as the cliloc reader's `COMPRESSED`: named, actionable outcome in the same family as the cliloc reader's `COMPRESSED`:
@@ -475,7 +496,7 @@ decoding do not have art:
| Elf male/female (605, 606) | real | 24×64, 24×63 | **art** | | Elf male/female (605, 606) | real | 24×64, 24×63 | **art** |
| **Elf ghosts (607, 608)** | `lookup 27221378, length 0` | 24×63 | **the previous body's picture** | | **Elf ghosts (607, 608)** | `lookup 27221378, length 0` | 24×63 | **the previous body's picture** |
| Human ghosts (402, 403) | `lookup -1` | nothing | honestly absent | | Human ghosts (402, 403) | `lookup -1` | nothing | honestly absent |
| Gargoyle (666, 667, 694, 695) | `lookup -1` | nothing | honestly absent | | Gargoyle (666, 667, 694, 695) | `lookup -1` / past the index | nothing | honestly absent |
Confirmed the same way: body 607 decoded after the dragon is the dragon, after the wolf is the Confirmed the same way: body 607 decoded after the dragon is the dragon, after the wolf is the
wolf, after the human male is the human male. Its 24×63 was the elf *female*'s dimensions, because wolf, after the human male is the human male. Its 24×63 was the elf *female*'s dimensions, because
@@ -485,6 +506,8 @@ Two consequences, both of which change work elsewhere in this document:
- **§4.3's UOP decoder covers eight player bodies, not six.** The elf ghosts join the human ghosts - **§4.3's UOP decoder covers eight player bodies, not six.** The elf ghosts join the human ghosts
and the four gargoyle bodies. Phase 4's scope grows by two ids and its argument does not change. and the four gargoyle bodies. Phase 4's scope grows by two ids and its argument does not change.
— *Phase 4 then found that six of those eight are in no client file at all, and recovered the
other two (§4.9). The reasoning above is why they were in scope; the answer is there.*
- **The `lookup -1` / `length 0` distinction is the whole difference between an honest absence and - **The `lookup -1` / `length 0` distinction is the whole difference between an honest absence and
a wrong picture**, and it is not visible from outside the index. `artidx.mul` and the legacy a wrong picture**, and it is not visible from outside the index. `artidx.mul` and the legacy
`anim*.idx` write `-1` for a record that is not there; an empty UOP slot, and evidently a `anim*.idx` write `-1` for a record that is not there; an empty UOP slot, and evidently a
@@ -505,6 +528,115 @@ file lookup is addressed by index, a success count is evidence of nothing.
--- ---
### 4.9 What phase 4 measured: two of eight, and 233 nobody was looking for
§4.3 was scoped to eight player-character bodies. Before writing the reader, phase 4 opened the
five packages and asked which of the eight were in them.
**Two.** Gargoyles 666 and 667, in `AnimationFrame3.uop`. The other six are in no package at all.
| Body | Legacy index | In a UOP package? |
|---|---|---|
| Gargoyle 666, 667 | index position runs **past the end of `anim5.idx`** | **yes — `AnimationFrame3.uop`** |
| Human ghosts 402, 403 | `lookup -1` — honest absence | no |
| Elf ghosts 607, 608 | `length 0` — the §4.8 shape | no |
| Gargoyle ghosts 694, 695 | `lookup -1` | no |
**That absence is established, not merely unfound**, which matters because "I looked and did not
see it" is exactly how a name scheme that is subtly wrong presents. The five packages hold
**10,724 entries** between them, and hashing `build/animationlegacyframe/%06d/%02d.bin` over bodies
04095 and actions 099 claims **all 10,724** — every entry accounted for, none left over for
another naming to hide in. No client ships ghost art. §5.2 is where that lands, and §17.9 is the
decision it produced.
#### The 233 the phase was not asked for
The rule "legacy first, UOP when it has nothing" is the same code whether it is applied to six
bodies or to all of them, and applied to all of them it finds **244 bodies with a group 0** in the
packages, of which only **8** also have legacy art. So the catalogue's working set goes from 787
to **1,022**, measured on the live rig:
| | Rows | Of which |
|---|---|---|
| `legacy` — vendored `Animations` over `anim*.mul` | **787** | 366 Equipment, 240 Monster, 95 Animal, 65 unlisted, 17 Human, 4 Sea |
| `uop` — §4.3's reader | **235** | 97 Equipment, 57 Monster, 50 unlisted, 26 Animal, 3 Human, 2 Sea |
The mix is worth a sentence because "add every body" sounds like it changes what a catalogue *is*:
it does not. The legacy 787 was **already** 366 equipment bodies — weapons and tools drawn alone
for the client to composite — so the UOP's 97 more change the proportion by a point. What a
bestiary sees is 83 creatures it could not show before, plus the gargoyles.
#### The format, as read rather than as assumed
A group file is one action of one body, every direction concatenated, zlib-compressed inside the
package (`flag == 1`, `78 9c`). The payload is not a repackaged mul record:
```
'AMOU' version decompressedSize bodyId … frameCount frameTableOffset
frame table: frameCount × 16 bytes (group, frameId, two unknowns, pixel offset)
each frame: its OWN 512-byte ARGB1555 palette, then centreX/centreY/width/height,
then the same run-length rows the legacy Frame decoder walks
```
Two consequences the reader is built on. The per-frame palette means **no `Ultima.Frame` and no
`Bitmap`**: the runs are written into a `ushort[]` of our own, which is what §4.4's promise about
this reader and `System.Drawing` was always about — and it leaves the encode as the only GDI+ step,
which phase 4 replaced with a ~200-line PNG writer (zlib around net48's raw-deflate-only
`DeflateStream`, CRC32, one IDAT, filter 0).
And **direction is a slice of the frame table**, not an index into five records: direction *d*
starts at `d * (frameCount / 5)`. On **nine** of the 244 bodies the frame count is not a multiple
of five (41, 42, 46…), where integer division lands slightly early in the run. That is what the
reference implementations do and it is the right trade here, because the failure §4.5 and §4.8 are
written against is a picture of the **wrong creature**, and this cannot produce one — the worst
case is the right creature at a marginally different angle, on nine bodies, against losing nine
bodies outright.
#### Validate-as-we-go, and the measurement that says it is in the right place
§4.5's rule is "validate before calling", because `Ultima`'s decoders take their bounds from the
file they read. Here there is no library to validate ahead of: this code **is** the decode. So the
same discipline appears as a bound on every read — the block chain against the file length, a
record against the file, the inflated length against the declared one, the frame table against the
payload, and every run header against **both** the record's remaining bytes and the bitmap it
writes into, which is the bound `Frame` itself does not have.
Measured exactly the way §4.5 was measured, because the second number is what makes a checker
defensible: across every UOP body on a stock client it refused **nothing** that carries art. The
one body it refuses, 286, declares a 0×0 frame — which the vendored decoder also treats as no art
rather than as damage, so it reports absent silently rather than logging a defect on every scan.
#### The live rig
Real sidecar, real ServUO, this machine's client:
| | |
|---|---|
| Catalogue | **1,022 rows in ONE page, 1,409 ms cold** |
| Player bodies | **six, and all six have art** — 400/401/605/606 `legacy`, 666/667 `uop` — every one at direction 0 |
| Direction split | 6 at direction 0, 1,016 at direction 1 |
| The six ghost bodies | `absent` |
| Duplicate-sha256 scan | 45 groups, of which **one** is new |
That last row is phase 3's cheap detector for the §4.8 bug, and it earned its keep again. The new
group is bodies 1531 and 1532 — two **distinct** records, 48,281 bytes each, whose first frames are
identical pixels. What separates that from a reused-buffer defect is not the hash: it is that each
payload declares its own body id, and the reader checked. A duplicate you can explain at the source
is data; one you cannot is the bug.
Manifest and fetch rows now carry `source` (`legacy` / `uop`) for exactly this reason — an operator
looking at a wrong picture can say which half of the extractor to doubt, and an acceptance walk can
prove the fallback fired rather than infer it from a count.
**`EXTRACTOR_VERSION` is 2.** Every client file is byte-identical and the answer is different,
which is the whole of what §7's number exists to say. The UOP packages join `assets.sources` and
the catalogue id with it, so patching one is drift rather than a silent no-op — and resolving them
needed its own lookup, because `Ultima.Files`' table of known client files predates UOP animations
and answers null for every `AnimationFrame*.uop` however present they are. The replacement matches
**case-insensitively by enumeration**, which is a Linux-host concern rather than a tidiness one.
---
## 5. Addressing: one key for every asset ## 5. Addressing: one key for every asset
Every asset the bridge can serve is named by a single string key, and the key is the cache key, Every asset the bridge can serve is named by a single string key, and the key is the cache key,
@@ -570,14 +702,14 @@ later is a setting and a re-import, not a protocol change.
### 5.2 "Player character body" is asked of the shard, never hardcoded ### 5.2 "Player character body" is asked of the shard, never hardcoded
`Server.Race.AllRaces` gives every registered race, and each carries `MaleBody`, `FemaleBody`, `Server.Race.AllRaces` gives every registered race, and each carries `MaleBody`, `FemaleBody`,
`MaleGhostBody` and `FemaleGhostBody`. The plugin enumerates those four ids per race and that set — `MaleGhostBody` and `FemaleGhostBody`. The plugin enumerates the **living pair** per race, and that
nothing else — takes index 0. On stock ServUO 57.4 that is twelve ids: set — nothing else — takes index 0. On stock ServUO 57.4 that is six ids:
| Race | Male | Female | Male ghost | Female ghost | | Race | Male | Female | Male ghost | Female ghost |
|---|---|---|---|---| |---|---|---|---|---|
| Human | 400 | 401 | 402 | 403 | | Human | **400** | **401** | 402 | 403 |
| Elf | 605 | 606 | 607 | 608 | | Elf | **605** | **606** | 607 | 608 |
| Gargoyle | 666 | 667 | **695** | **694** | | Gargoyle | **666** | **667** | 695 | 694 |
This is the §8 argument again in miniature: only code inside ServUO can answer it, and asking is This is the §8 argument again in miniature: only code inside ServUO can answer it, and asking is
the only thing that works on a shard with a custom race. Two details make the case that a the only thing that works on a shard with a custom race. Two details make the case that a
@@ -585,27 +717,32 @@ hardcoded list would have been wrong — `RaceDefinitions.cs` passes the gargoyl
the **opposite order** to the other two races (695 male, 694 female), and a shard that calls the **opposite order** to the other two races (695 male, 694 female), and a shard that calls
`RegisterRace` adds ids no table of ours would contain. `RegisterRace` adds ids no table of ours would contain.
**Two thirds of that set has no art in ServUO's vendored library.** Measured — and re-measured in **The ghost columns are greyed above because no client has art for any of them, anywhere.** That is
phase 1, which moved two rows across (§4.8): phase 4's measurement and it is what removed them from the set (§17.9). The evidence is in §4.9:
402/403 and 694/695 read `lookup -1` in the legacy index, 607/608 read `length 0` — the §4.8 shape,
where the library hands back the previously-decoded body's picture — and **none of the six appears
in any `AnimationFrame*.uop`**, established by claiming all 10,724 entries of the five packages
with a single name scheme rather than by failing to find them.
| Has art | Does not | So the set is asked of the shard exactly as before; only the question changed, from four ids per
race to two. A shard whose client *does* ship ghost art loses nothing: the body is catalogued like
any other, at direction 1 rather than 0.
**And with phase 4 in, all six have art.** The gargoyles were the last gap, and they were never a
missing-art problem: `Bodyconv.def` sends 666 and 667 to `anim5`, at an index past the end of
`anim5.idx`, while `AnimationFrame3.uop` has held both all along.
| Has art | Via |
|---|---| |---|---|
| Human male/female (400, 401) | **Human ghosts (402, 403)** — `lookup -1` | | Human male/female (400, 401) | vendored `Animations`, `anim.mul` |
| Elf male/female (605, 606) | **Elf ghosts (607, 608)** — `length 0`, and the library returns the previously-decoded body's bitmap | | Elf male/female (605, 606) | vendored `Animations`, `anim5.mul` |
| | **Every gargoyle body (666, 667, 694, 695)** — `lookup -1` | | **Gargoyle male/female (666, 667)** | **§4.3's UOP reader, `AnimationFrame3.uop`** |
**Four of twelve, not six.** The elf ghosts were counted as decoding for as long as the library's The catalogue must still **not** treat a missing player body as an error. It is now an unlikely
own answer was the measurement: their index entry has no record, and what came back was whatever answer rather than the expected one, but a client that ships fewer bodies than this one is an
the catalogue walk had decoded the moment before — the elf female, at her exact dimensions. All ordinary thing to meet, and a status screen that flags failures on every import teaches an operator
eight are UOP-only and **§4.3 adds a decoder for them** — verified genuinely absent from the legacy to ignore it. `shard_spawn_creatures.art` staying NULL remains a first-class state everywhere it is
files rather than merely mis-addressed, by rendering what the other anim files hold at those consumed, which it already is.
indices and finding spiders.
Until that phase lands, and on any client that lacks them entirely, the catalogue must **not** treat
a missing player body as an error. It is the expected answer for two thirds of the set, and a
status screen that flags eight failures on every import teaches an operator to ignore it.
`shard_spawn_creatures.art` staying NULL remains a first-class state everywhere it is consumed,
which it already is.
--- ---
@@ -714,7 +851,7 @@ Every prediction in §4.8 and §5.2 held when the bytes were actually rendered a
| `body/607/a0`, `body/608/a0` (elf ghosts) | **absent** | §4.8's two rows that moved; the library returns the elf female here | | `body/607/a0`, `body/608/a0` (elf ghosts) | **absent** | §4.8's two rows that moved; the library returns the elf female here |
| `body/666/a0` (gargoyle → `anim5`) | **absent** | The spider trap. Nothing swept, nothing found | | `body/666/a0` (gargoyle → `anim5`) | **absent** | The spider trap. Nothing swept, nothing found |
| `body/400/a0/…` deeper key | **unsupported** | Well-formed under §5, not served until phase 6 | | `body/400/a0/…` deeper key | **unsupported** | Well-formed under §5, not served until phase 6 |
| direction distribution across 787 | **783 at index 1, 4 at index 0** | Four player bodies, not six — §4.8 again, from `Race.AllRaces` rather than a table | | direction distribution across 787 | **783 at index 1, 4 at index 0** | Four player bodies, not six — §4.8 again, from `Race.AllRaces` rather than a table. *Phase 4 made it six of six and 1,016/6 across 1,022 rows: §4.9* |
**44 of the 787 hashes are shared by two or three bodies, and that is correct.** It is the exact **44 of the 787 hashes are shared by two or three bodies, and that is correct.** It is the exact
signature the wrong-picture bug produces, so it was chased rather than assumed: the client's own signature the wrong-picture bug produces, so it was chased rather than assumed: the client's own
@@ -882,22 +1019,27 @@ project. §1.1's measurements make the sizing question concrete:
|---|---|---| |---|---|---|
| Item statics | **~49,150** | No — on demand, cached, keyed by `itemId` (+ hue) | | Item statics | **~49,150** | No — on demand, cached, keyed by `itemId` (+ hue) |
| Land tiles | **16,384** | No — on demand | | Land tiles | **16,384** | No — on demand |
| Creature/player bodies, first frame | **787** (§4.8*not* the 1,144 the library reports) | **Yes** — this is the catalogue | | Creature/player bodies, first frame | **1,022** — 787 legacy (§4.8, *not* the 1,144 the library reports) + 235 UOP (§4.9) | **Yes** — this is the catalogue |
| One body, every action, one direction | **210 frames** (body 400); 96210 measured across six bodies | No — on demand, per body | | One body, every action, one direction | **210 frames** (body 400); 96210 measured across six bodies | No — on demand, per body |
| All bodies, every action, one direction | **~119,000 frames**, ~117 MB | No — but no longer unthinkable | | All bodies, every action, one direction | **~119,000 frames**, ~117 MB | No — but no longer unthinkable |
| The same at five directions | ~865,000 frames | Not built (§5.1) | | The same at five directions | ~865,000 frames | Not built (§5.1) |
| Cliloc table | 123,490 entries → 67,496 rows | **Yes** — whole-table replace | | Cliloc table | 123,490 entries → 67,496 rows | **Yes** — whole-table replace |
| ServUO tree files (§10) | ~21 files, ~10.6 MB | **Yes** | | ServUO tree files (§10) | ~21 files, ~10.6 MB | **Yes** |
**The working set is one thumbnail per body, plus the atlas's own creatures.** 787 sprites at **The working set is one thumbnail per body, plus the atlas's own creatures.** 1,022 sprites at
roughly a kilobyte each is under 1 MB — trivial to import, trivial to re-hash, and it is the set roughly a kilobyte each is about a megabyte — trivial to import, trivial to re-hash, and it is the
that makes a bestiary, a marketplace listing and a character sheet render. set that makes a bestiary, a marketplace listing and a character sheet render.
Phase 1 took that count down from 1,144, and the 357 it removed are the important part: those are Phase 1 took that count down from 1,144, and the 357 it removed are the important part: those are
ids with **no art** that the library returns a picture for (§4.8). Importing them would have ids with **no art** that the library returns a picture for (§4.8). Importing them would have
written 357 duplicate creature portraits into the site, each one showing whichever body the walk written 357 duplicate creature portraits into the site, each one showing whichever body the walk
happened to decode before it. The count went down; what the catalogue is worth went up. happened to decode before it. The count went down; what the catalogue is worth went up.
Phase 4 then took it **up** to 1,022, and the direction of travel is the same argument rather than
its opposite: the 235 it added are bodies with real art in a file the vendored decoder does not
open (§4.9), each one validated at every bound and identified by a name that carries its body id.
Both numbers moved because something was measured rather than reported.
Everything deeper is the *same protocol at a deeper key* (§5), fetched on demand and cached. That Everything deeper is the *same protocol at a deeper key* (§5), fetched on demand and cached. That
is what serves the future project without exporting 3.5 GB of someone else's copyrighted client is what serves the future project without exporting 3.5 GB of someone else's copyrighted client
into a database: a viewer that wants body 400's full walk cycle asks for `body/400/a2/f0..f9` and into a database: a viewer that wants body 400's full walk cycle asks for `body/400/a2/f0..f9` and
@@ -1002,8 +1144,8 @@ new pipe.
| Command | Reply | Purpose | Built | | Command | Reply | Purpose | Built |
|---|---|---|---| |---|---|---|---|
| `assets.sources` | `assets.sources.ok` | Stage 1: client file manifest + `EXTRACTOR_VERSION` | **phase 1** | | `assets.sources` | `assets.sources.ok` | Stage 1: client file manifest + `EXTRACTOR_VERSION` | **phase 1** |
| `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes, width, height, body, direction}]`, paged | **phase 3** | | `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes, width, height, body, direction, source}]`, paged | **phase 3** (`source` phase 4) |
| `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged; base64 PNG per row | **phase 3** | | `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged; base64 PNG per row | **phase 3** (`source` phase 4) |
| `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) | **phase 3** | | `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) | **phase 3** |
| `cliloc.table` | `cliloc.table.ok` | The decompressed table, paged (`?lang=`, `?cursor=`) | **phase 2** | | `cliloc.table` | `cliloc.table.ok` | The decompressed table, paged (`?lang=`, `?cursor=`) | **phase 2** |
| `tree.manifest` / `tree.fetch` | `.ok` | §10, the ServUO tree files | phase 7 | | `tree.manifest` / `tree.fetch` | `.ok` | §10, the ServUO tree files | phase 7 |
@@ -1025,6 +1167,12 @@ side effect — a few hundred asset keys do not belong in a query string. They a
this link that take one. `422` gains a second meaning on this plane alongside "cannot decode": the this link that take one. `422` gains a second meaning on this plane alongside "cannot decode": the
mid-import guard, a `catalog` that no longer describes the files on disk. mid-import guard, a `catalog` that no longer describes the files on disk.
**Phase 4 added one field and no command.** `source` on a manifest or fetch row is `legacy` or
`uop` — which reader produced the bytes (§4.9). It is additive, so **the protocol stays 8**: a
consumer that does not read it is unaffected, and one that does can say which half of the extractor
to doubt when a picture is wrong. `EXTRACTOR_VERSION` went to **2** in the same phase, which is the
change every consumer *does* see, exactly as §7 intends.
**§16 listed phase 3 as `servuo-plugins, module-uo` and that was wrong** — `web.rs` routes every **§16 listed phase 3 as `servuo-plugins, module-uo` and that was wrong** — `web.rs` routes every
command explicitly and has no generic `/assets/*` forwarder, so `link` is in the phase too. The command explicitly and has no generic `/assets/*` forwarder, so `link` is in the phase too. The
table now says so. table now says so.
@@ -1064,7 +1212,7 @@ disagree, so a split bump means the next bundle silently fails to compose.
| 1 | **DONE 2026-09-10.** The transport: `assets.sources`, the single-slot gate (§3.2), the paging envelope (§3.4), the sidecar line cap (§3.3), `EXTRACTOR_VERSION`, `NO_IMAGING`, protocol 7→8. Plus §4.5's validator promoted into the overlay and extended to animations — which found **357 wrong pictures in the body catalogue on a stock client** and cut it from 1,144 to 787 (§4.8) | servuo-plugins, link | | 1 | **DONE 2026-09-10.** The transport: `assets.sources`, the single-slot gate (§3.2), the paging envelope (§3.4), the sidecar line cap (§3.3), `EXTRACTOR_VERSION`, `NO_IMAGING`, protocol 7→8. Plus §4.5's validator promoted into the overlay and extended to animations — which found **357 wrong pictures in the body catalogue on a stock client** and cut it from 1,144 to 787 (§4.8) | servuo-plugins, link |
| 2 | **DONE 2026-09-10.** Clilocs end to end (§9.1, §9.2): the Mythic decompressor ported into the overlay, `cliloc.table` + `GET /cliloc`, the paging walk and the source switch on the website, module-uo's protocol pin 7→8. `cliloc-export/` deleted and `UOFIDDLER.md` §Part 1 with it. **67,496 rows, 290 ms, ~11 pages** — the same count UOFiddler's own DLL produced from this client | all | | 2 | **DONE 2026-09-10.** Clilocs end to end (§9.1, §9.2): the Mythic decompressor ported into the overlay, `cliloc.table` + `GET /cliloc`, the paging walk and the source switch on the website, module-uo's protocol pin 7→8. `cliloc-export/` deleted and `UOFIDDLER.md` §Part 1 with it. **67,496 rows, 290 ms, ~11 pages** — the same count UOFiddler's own DLL produced from this client | all |
| 3 | **DONE 2026-09-10.** Body resolution (§8) + the **787**-body catalogue (§4.8), `assets.manifest` / `assets.fetch` / `assets.bodies` and their REST mirrors, `shard_spawn_creatures.art` filled and rendered (§8.1, §12.1). **787 rows in one 734 ms page; 455 types resolved at ~190 ms per 100 on the Core thread; zero mobiles leaked.** `UOFIDDLER.md` deleted, two phases early | servuo-plugins, **link**, module-uo | | 3 | **DONE 2026-09-10.** Body resolution (§8) + the **787**-body catalogue (§4.8), `assets.manifest` / `assets.fetch` / `assets.bodies` and their REST mirrors, `shard_spawn_creatures.art` filled and rendered (§8.1, §12.1). **787 rows in one 734 ms page; 455 types resolved at ~190 ms per 100 on the Core thread; zero mobiles leaked.** `UOFIDDLER.md` deleted, two phases early | servuo-plugins, **link**, module-uo |
| 4 | **The UOP animation decoder (§4.3)** — the **eight** player bodies the legacy path cannot reach (§4.8 moved the elf ghosts across), the never-sweep rule already enforced in code by phase 1's `ResolveAnimation` | servuo-plugins | | 4 | **DONE 2026-09-11.** The UOP animation decoder (§4.3, §4.9): `BridgeUop` + a PNG encoder that never touches `System.Drawing`, wired in beneath the legacy reader. **Two of the eight player bodies turned out to exist** (gargoyles 666/667); the other six are in no client file, and ghost ids left the player-body set (§5.2, §17.9). The same fallback added **233 other bodies**: the catalogue is **1,022 rows, 1,409 ms cold**, and all six player bodies have art for the first time. `EXTRACTOR_VERSION` 1 → 2 | servuo-plugins |
| 5 | Item statics and land on demand, hued keys, the cache | servuo-plugins, module-uo | | 5 | Item statics and land on demand, hued keys, the cache | servuo-plugins, module-uo |
| 6 | Deep animation keys (`body/<id>/a<n>/f<n>`) for the future project, plus the bulk-fill switch | servuo-plugins, module-uo | | 6 | Deep animation keys (`body/<id>/a<n>/f<n>`) for the future project, plus the bulk-fill switch | servuo-plugins, module-uo |
| 7 | The atlas over the sidecar (§10); shared-filesystem requirement retired | module-uo | | 7 | The atlas over the sidecar (§10); shared-filesystem requirement retired | module-uo |
@@ -1077,11 +1225,12 @@ a client that has been patched — *before* building eight phases on top of it.
§1.1 were run from PowerShell against a stock client; neither of those is the environment this will §1.1 were run from PowerShell against a stock client; neither of those is the environment this will
actually run in. actually run in.
Phase 4 sits **after** the catalogue rather than inside it on purpose. The catalogue is useful with Phase 4 sat **after** the catalogue rather than inside it on purpose. The catalogue was useful with
783 of its 787 bodies, the UOP reader is the one piece of genuinely new format work in this 783 of its 787 bodies, the UOP reader is the one piece of genuinely new format work in this
protocol, and putting it on the critical path would hold up every website-side phase behind it. Its protocol, and putting it on the critical path would have held up every website-side phase behind
acceptance test is not "it decodes" — it is that a gargoyle looks like a gargoyle, checked by eye, it. Its acceptance test was not "it decodes" — it was that a gargoyle looks like a gargoyle,
because §4.3's whole point is that this failure mode produces confident, wrong pictures. checked by eye, because §4.3's whole point is that this failure mode produces confident, wrong
pictures. It does, and it was.
--- ---
@@ -1091,8 +1240,8 @@ Every item here is settled. Each is recorded because it changes numbers or oblig
in the document. in the document.
1. **§4: settled 2026-09-10 — call ServUO's vendored `Ultima`,** with one exception added the same 1. **§4: settled 2026-09-10 — call ServUO's vendored `Ultima`,** with one exception added the same
day: **§4.3's UOP animation decoder**, scoped to the player bodies the legacy path cannot reach day: **§4.3's UOP animation decoder** (built in phase 4; §17.9 widened its scope from the
and to nothing else (phase 4). The crash is confined to the `hasExtra: true` branch that only player bodies to every body the legacy path cannot reach). The crash is confined to the `hasExtra: true` branch that only
`Gumps` reaches, and nothing here calls `Gumps`. **Phase 0 confirmed both halves in the real `Gumps` reaches, and nothing here calls `Gumps`. **Phase 0 confirmed both halves in the real
host process** — nothing faulted on a path we call, and one `GetGump(2)` killed the shard host process** — nothing faulted on a path we call, and one `GetGump(2)` killed the shard
outright (§4.7). outright (§4.7).
@@ -1148,3 +1297,24 @@ in the document.
found: the animation path has §4.5's shared-buffer defect too, and 357 ids with no art were found: the animation path has §4.5's shared-buffer defect too, and 357 ids with no art were
returning the previously-decoded body's bitmap. The elf ghosts moved from "decodes" to "no art" returning the previously-decoded body's bitmap. The elf ghosts moved from "decodes" to "no art"
in §5.2 for the same reason, taking phase 4's set from six player bodies to eight. in §5.2 for the same reason, taking phase 4's set from six player bodies to eight.
9. **§4.3/§5.2: phase 4's four, settled 2026-09-11.** Put to the org lead after the packages were
opened and before the reader was written, because the first measurement changed what the phase
was worth:
- **The UOP fallback applies to every body, not only to player bodies.** It is the same rule
either way — "legacy first, UOP when it has nothing" — and restricting it would have needed an
extra filter to *refuse* art the client has. 787 → 1,022 rather than 787 → 789.
- **Ghost ids leave the player-body set** (org lead, over keeping them with a new status or
leaving them as `absent`). No client has art for any of the six, so listing them advertised
keys that cannot exist. The shard is still asked rather than told — the question is now the
living pair per race.
- **The UOP path gets its own PNG encoder** rather than `Bitmap.Save`, which is what §4.4's
sentence about this reader and `System.Drawing` was always promising. Phase 3's `ToPng` is
untouched: its input comes from a decoder that needs GDI+ to produce a pixel, so encoding it
without GDI+ buys nothing.
- **A host without libgdiplus keeps the flat `NO_IMAGING` refusal** (org lead, over serving the
~238 bodies that no longer need it). One answer beats a quarter-full catalogue nobody can tell
from a complete one. The door §4.4 wanted open stays open; phase 4 simply does not walk
through it.

View File

@@ -82,7 +82,7 @@ account-linking routes and the sidecar config under `/admin/uo-link` are in the
| GET | `/shard/clilocs` | cliloc-table status (`adminOnly`): every source found now (base first, then `custom/` overlays in merge order), what each contributed at the last import, readability, drift across the set, the entry count, and `missingSources`. `configured:false` is a supported state — item names then render as ids. No public counterpart: the table is never served *as* a table. | | GET | `/shard/clilocs` | cliloc-table status (`adminOnly`): every source found now (base first, then `custom/` overlays in merge order), what each contributed at the last import, readability, drift across the set, the entry count, and `missingSources`. `configured:false` is a supported state — item names then render as ids. No public counterpart: the table is never served *as* a table. |
| POST | `/shard/clilocs/import` | reload after a client patch or an overlay edit; `{force}` ignores the hash gate, `{approve}` accepts a **vanished** source (refused by default — see the table notes above). **A missing path — or the likely mistake of pointing at the client's own COMPRESSED `Cliloc.enu` — answers 200 with `status:"unavailable"` and a `code`, not 500.** `COMPRESSED` is called out by name: a 500 would say only "something broke", and the operator needs to be told which file to convert. | | POST | `/shard/clilocs/import` | reload after a client patch or an overlay edit; `{force}` ignores the hash gate, `{approve}` accepts a **vanished** source (refused by default — see the table notes above). **A missing path — or the likely mistake of pointing at the client's own COMPRESSED `Cliloc.enu` — answers 200 with `status:"unavailable"` and a `code`, not 500.** `COMPRESSED` is called out by name: a 500 would say only "something broke", and the operator needs to be told which file to convert. |
| GET | `/shard/assets` | client-asset import status (`adminOnly`, Protocol 8): the imported body catalogue, how many sprites are on disk, how many atlas creatures resolved to a body id, and the shard's own client files beside them. `drift:true` means the client was patched. `shard.imaging.ok:false` is the named `NO_IMAGING` state — a Linux shard host with no `libgdiplus` cannot decode a sprite at all, and the reason names the package. No public counterpart: the pictures are served as ordinary files under `/uploads`. | | GET | `/shard/assets` | client-asset import status (`adminOnly`, Protocol 8): the imported body catalogue, how many sprites are on disk, how many atlas creatures resolved to a body id, and the shard's own client files beside them. `drift:true` means the client was patched. `shard.imaging.ok:false` is the named `NO_IMAGING` state — a Linux shard host with no `libgdiplus` cannot decode a sprite at all, and the reason names the package. No public counterpart: the pictures are served as ordinary files under `/uploads`. |
| POST | `/shard/assets/import` | import creature artwork from the shard's UO client; `{force}` ignores the hash gate, `{approve}` accepts a catalogue that no longer offers assets this site holds (refused by default — an unmounted client volume and a deliberate downgrade are indistinguishable, and the wrong guess deletes artwork). **This is the ONLY thing that imports** — boot deliberately never calls the shard. An operator's `spawnAtlas.art.json` always wins over an imported sprite. A body this client has no art for is **not** a failure: two thirds of the playable ghost and gargoyle bodies are in that state on a stock client. | | POST | `/shard/assets/import` | import creature artwork from the shard's UO client; `{force}` ignores the hash gate, `{approve}` accepts a catalogue that no longer offers assets this site holds (refused by default — an unmounted client volume and a deliberate downgrade are indistinguishable, and the wrong guess deletes artwork). **This is the ONLY thing that imports** — boot deliberately never calls the shard. An operator's `spawnAtlas.art.json` always wins over an imported sprite. A body this client has no art for is **not** a failure: about half the addressable body range is in that state on a stock client. |
| PUT | `/shard/clilocs/path` | point the site at a different cliloc base file or directory (persisted as `cliloc_client_path`, which wins over `UO_CLIENT_PATH`). Overlays are read from `custom/` beside it either way. Blank clears it. Deliberately **does not import**, same reasoning as the atlas path. | | PUT | `/shard/clilocs/path` | point the site at a different cliloc base file or directory (persisted as `cliloc_client_path`, which wins over `UO_CLIENT_PATH`). Overlays are read from `custom/` beside it either way. Blank clears it. Deliberately **does not import**, same reasoning as the atlas path. |
## 4. Shard visibility — the audience boundary (Protocol 3.0) ## 4. Shard visibility — the audience boundary (Protocol 3.0)

View File

@@ -233,15 +233,16 @@ custom creatures) and points each `shard_spawn_creatures.art` at its picture.
Boot never calls the shard for this: the files change when an operator patches Boot never calls the shard for this: the files change when an operator patches
their client, which is an event they know about and the site does not. their client, which is an event they know about and the site does not.
On this machine's stock client that is **787 creature portraits**, about a On this machine's stock client that is **1,022 creature portraits**, about a
megabyte in total. megabyte in total — 787 out of the legacy `anim*.mul` files and 235 more out of
`AnimationFrame*.uop`, which ServUO's own decoder never opens
([`../link/v8.md`](../link/v8.md) §4.9).
**NULL stays a first-class state, and always will be.** An install with no shard **NULL stays a first-class state, and always will be.** An install with no shard
link has never imported one; a Linux shard host without `libgdiplus` cannot link has never imported one; a Linux shard host without `libgdiplus` cannot
render a sprite at all (a named `NO_IMAGING` status, not an error); and even render a sprite at all (a named `NO_IMAGING` status, not an error); and about
after a complete import, two thirds of the playable ghost and gargoyle bodies half the addressable body range has no art in any client file. Pages render
have no art in the client. Pages render without images, which is normal and without images, which is normal and supported, not degraded.
supported, not degraded.
### The operator's own artwork still wins ### The operator's own artwork still wins