docs(link): the Asset Bridge (Protocol 8) — client assets without UOFiddler #234
124
link/v8.md
124
link/v8.md
@@ -139,46 +139,57 @@ would exceed the cap. Base64 costs 33%; the budget must be stated in encoded byt
|
||||
|
||||
---
|
||||
|
||||
## 4. The decoders: whose code, and the crash that decides it
|
||||
## 4. The decoders are ServUO's own — decided, and the crash is narrower than it looked
|
||||
|
||||
§1.1 found that `Gumps.GetGump(2)` does not fail — it **corrupts the process**. An
|
||||
`AccessViolationException` from `unsafe` pointer code is a corrupted-state exception; on .NET
|
||||
Framework 4.8 it is *not catchable* by an ordinary `try/catch`. In-process, on a live shard, that
|
||||
is a shard crash with players on it.
|
||||
**We call ServUO's vendored `Ultima` (decided 2026-09-10).** No decoders are reimplemented.
|
||||
`overlay/Scripts/Scripts.csproj:39` already references the project, so the art half of this protocol
|
||||
costs plumbing rather than pixel code, and only §9's cliloc decompressor is written from scratch.
|
||||
|
||||
Art and animation probed clean across ~66,000 and ~2,000 ids respectively. That is reassuring and
|
||||
it is not a guarantee: the inputs we have not tested are exactly the ones that matter — a shard's
|
||||
own **patched or custom** client files, which is the population this feature exists to serve.
|
||||
The reason that is safe, rather than merely cheap, is a distinction §1.1 did not draw at first.
|
||||
|
||||
Two ways to hold this safely:
|
||||
### 4.1 The crash lives on one code path, and nothing we call uses it
|
||||
|
||||
**A1 — call ServUO's vendored `Ultima`, isolate the fault.** Fastest to build, and proven for the
|
||||
kinds we need. Requires running the decode where a fault costs one batch rather than the shard:
|
||||
a short-lived child process. That means a binary to deploy, which the overlay (deployed as
|
||||
*source*, compiled by ServUO at boot) has no mechanism for.
|
||||
`Gumps.GetGump(2)` does not fail — it **corrupts the process**: `AccessViolationException`, exit
|
||||
`0xC0000005`. That is a corrupted-state exception, uncatchable by an ordinary `try/catch` on .NET
|
||||
Framework 4.8, so in-process on a live shard it is a crash with players on it. That much is
|
||||
alarming, and on its own it looked like an argument against using this library at all.
|
||||
|
||||
**A2 — own bounds-checked decoders in the overlay.** Roughly 600–900 lines of ordinary safe C#:
|
||||
`FileIndex` (~150), the RLE frame decoder (~60, and it is the same loop, writing to a `ushort[]`
|
||||
instead of through a `LockBits` pointer), the static-art decoder, `Body.def`/`Bodyconv.def`, hue
|
||||
application, and a minimal PNG writer over `System.IO.Compression.DeflateStream` (~80).
|
||||
It is not, because of how the three decoders construct their `FileIndex`:
|
||||
|
||||
**A2 is recommended**, for four reasons that compound:
|
||||
| Decoder | UOP file | `hasExtra` | Probed |
|
||||
|---|---|---|---|
|
||||
| `Art` | `artLegacyMUL.uop` | **false** | 49,150 statics + 16,384 land tiles, **0 faults** |
|
||||
| `Animations` | *none — legacy `anim*.mul` only* | — | 1,144 bodies, **0 faults** |
|
||||
| `Gumps` | `gumpartLegacyMUL.uop` | **true** | **faults on the second id** |
|
||||
|
||||
1. **It removes the crash class**, rather than containing it.
|
||||
2. **It removes `System.Drawing` entirely.** ServUO's `Frame` decoder writes ARGB1555 straight
|
||||
into a `Bitmap` via `LockBits`, so System.Drawing is in the *decode*, not just the encode — a
|
||||
Linux/Mono shard needs libgdiplus even to read a sprite. Writing our own pixels and our own PNG
|
||||
makes the feature portable by construction.
|
||||
3. **It removes the UOP gap, which lands squarely on the player bodies.** ServUO's vendored
|
||||
`Animations` reads legacy `anim*.mul` only, never `AnimationFrame*.uop`. This client has a full
|
||||
195 MB `anim.mul` so most creatures resolve — but **six of the twelve stock player-character
|
||||
bodies return nothing**, including every gargoyle and both human ghosts (§5.2). A missing
|
||||
playable race in an asset store built for "player models and everything" is not a caveat, it is
|
||||
a defect, and it is in the highest-attention part of the scope.
|
||||
4. **We are already in this business.** §9 writes a cliloc decompressor from scratch regardless.
|
||||
`FileIndex.cs`'s own comment says the extra-field handling exists *for* `gumpartlegacy.uop` — it is
|
||||
the one UOP layout carrying an extra field, and `hasExtra: true` is the branch written to cope with
|
||||
it. **Gumps is the only caller that sets it.** So the fault is not a general fragility in this
|
||||
library's `unsafe` code; it is a bug on a branch that exactly one decoder reaches, and that decoder
|
||||
is already out of scope (§11).
|
||||
|
||||
A2 also ends the dependency on whatever version of `Ultima` a given ServUO happens to vendor,
|
||||
which is the kind of thing that silently changes under a shard upgrade.
|
||||
The rule this turns into is a safety rule, not a preference: **nothing in this protocol calls
|
||||
`Ultima.Gumps`.** Adding gump art later means fixing or replacing that path first, deliberately,
|
||||
not discovering it in production.
|
||||
|
||||
### 4.2 What the decision accepts
|
||||
|
||||
Three costs come with it, all known and none of them blocking:
|
||||
|
||||
1. **Six of the twelve stock player-character bodies have no art** — both human ghosts and every
|
||||
gargoyle body (§5.2). `Animations` never reads `AnimationFrame*.uop`, and UOP animation is a
|
||||
different container with its own `AnimationSequence.uop`, not a mirror of the mul layout, so
|
||||
this is not reachable by calling the existing code differently. Those bodies degrade to no
|
||||
image, which is the same state every creature is in today.
|
||||
2. **`System.Drawing` is a hard dependency, in the decode and not just the encode.** `Frame`
|
||||
writes ARGB1555 straight through a `LockBits` pointer, so a Linux/Mono shard needs libgdiplus
|
||||
to read a sprite at all. See §17.
|
||||
3. **We inherit whatever `Ultima` a given ServUO vendors**, which can change under a shard upgrade.
|
||||
`EXTRACTOR_VERSION` (§7) is the mitigation: it already counts as drift, so a shard whose library
|
||||
changed re-derives on the next import.
|
||||
|
||||
The residual risk that remains is a patched or custom client tripping an out-of-bounds read on a
|
||||
path we *do* call. §16's phase 0 is where that gets exercised rather than assumed.
|
||||
|
||||
---
|
||||
|
||||
@@ -270,10 +281,14 @@ the **opposite order** to the other two races (695 male, 694 female), and a shar
|
||||
| Elf male/female (605, 606) | **Every gargoyle body (666, 667, 694, 695)** |
|
||||
| Elf ghosts (607, 608) | |
|
||||
|
||||
Six of twelve, including a whole playable race. So the one part of the asset scope with the most
|
||||
attention on it — the player character, head-on — is precisely the part the vendored `Animations`
|
||||
serves worst, and §4's recommendation to own the decoders is what fixes it. This is the single
|
||||
strongest piece of evidence for that choice.
|
||||
Six of twelve, including a whole playable race — and §4 accepts that rather than reimplementing
|
||||
`Animations` to reach `AnimationFrame*.uop`. Those bodies render without an image, exactly as every
|
||||
creature does today.
|
||||
|
||||
Two things follow for the build. The catalogue must **not** treat a missing player body as an
|
||||
error — it is the expected answer for half the set, and a status screen that flags six failures on
|
||||
every import will teach an operator to ignore it. And `shard_spawn_creatures.art` staying NULL has
|
||||
to remain a first-class state everywhere it is consumed, which it already is.
|
||||
|
||||
---
|
||||
|
||||
@@ -356,8 +371,8 @@ Every modern client ships `Cliloc.*` in the Mythic compressed container — this
|
||||
the shard's own `VendorSearch.GetItemName` is already inert.
|
||||
|
||||
**UOFiddler is released under the Beerware licence**, so porting its decompressor into our
|
||||
GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C#, alongside §4's decoders,
|
||||
and from that point:
|
||||
GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C# — the **only** decoder
|
||||
Protocol 8 writes rather than calls (§4) — and from that point:
|
||||
|
||||
- No operator installs UOFiddler.
|
||||
- No operator runs `dotnet build` on a converter.
|
||||
@@ -440,10 +455,10 @@ bulk-fill-everything switch rather than assuming on-demand is the only mode.
|
||||
actually carries hue 33. The cross product of 49,150 statics and ~3,000 hues is not a set anyone
|
||||
enumerates.
|
||||
|
||||
**Gump art is out of scope for Protocol 8** — see §4; it is the one kind whose vendored decoder
|
||||
demonstrably corrupts the process, and A2 would have to reimplement it against the UOP-with-extra-
|
||||
field layout that broke it. It is additive to add later under this same key scheme
|
||||
(`gump/<id>`), which is the point of §5.
|
||||
**Gump art is out of scope for Protocol 8, and that is now a safety rule rather than a priority
|
||||
call** — §4.1. It is the only decoder that reaches the `hasExtra: true` branch, and that branch
|
||||
corrupts the process on the second id. Adding gump art later means fixing that path first,
|
||||
deliberately; it is additive under the same key scheme (`gump/<id>`), which is the point of §5.
|
||||
|
||||
---
|
||||
|
||||
@@ -510,7 +525,7 @@ disagree, so a split bump means the next bundle silently fails to compose.
|
||||
|
||||
| Repo | Work |
|
||||
|---|---|
|
||||
| `servuo-plugins/` | The decoders (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` |
|
||||
| `servuo-plugins/` | Extraction over ServUO's own `Ultima` (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` |
|
||||
| `link/` | Six command families forwarded, the REST surface, **the inbound line cap (§3.3)**, `PROTOCOL_VERSION` |
|
||||
| `module-uo/` | Client calls, asset store, the atlas source backend (§10), cliloc ingest, admin surface |
|
||||
| `website/` | None expected — `ctx.uploads` already suffices (§12) |
|
||||
@@ -525,7 +540,7 @@ disagree, so a split bump means the next bundle silently fails to compose.
|
||||
|
||||
| # | Scope | Repos |
|
||||
|---|---|---|
|
||||
| 0 | Spike: A2's decoders against this machine's client — statics, land, one body, the Mythic cliloc — proving output matches UOFiddler's byte for byte | servuo-plugins |
|
||||
| 0 | Spike: the vendored decoders driven **from inside a running ServUO**, over a deliberately patched client — statics, land, bodies, and the Mythic cliloc against UOFiddler's output. What it is looking for is a fault on a path we call (§4.2) | servuo-plugins |
|
||||
| 1 | The transport: `assets.sources`, flow control, the sidecar line cap, `EXTRACTOR_VERSION`, protocol bump | servuo-plugins, link |
|
||||
| 2 | Clilocs end to end; retire the converter and `UOFIDDLER.md` §Part 1 | all |
|
||||
| 3 | Body resolution (§8) + the 1,144-body catalogue; `shard_spawn_creatures.art` filled | servuo-plugins, module-uo |
|
||||
@@ -535,18 +550,25 @@ disagree, so a split bump means the next bundle silently fails to compose.
|
||||
| 7 | Admin surface, Import/Update, approve/reject, activity log | module-uo |
|
||||
| 8 | Docs pass across five repos; live walk on the real rig | docs |
|
||||
|
||||
Phase 0 exists because §4 chose to own the decoders, and the honest way to hold that choice is to
|
||||
prove byte-identical output *before* building six phases on top of it.
|
||||
Phase 0 exists because §4 chose to call code that can take the shard down if it is wrong, and the
|
||||
honest way to hold that choice is to try to break it on purpose — in the real host process, against
|
||||
a client that has been patched — *before* building seven phases on top of it. The probes behind
|
||||
§1.1 were run from PowerShell against a stock client; neither of those is the environment this will
|
||||
actually run in.
|
||||
|
||||
---
|
||||
|
||||
## 17. Decisions still open
|
||||
|
||||
1. **§4: A2 (own decoders) versus A1 (vendored `Ultima` in a child process).** A2 is recommended
|
||||
and the phase plan assumes it. It costs a spike and ~600–900 lines; it buys the crash class, the
|
||||
libgdiplus dependency, the UOP gap and the vendor-drift risk all going away at once.
|
||||
2. **§11: gump art deferred.** Confirm that paperdoll and equipment gump art is genuinely
|
||||
out of scope for 8, given it is the one kind whose existing decoder crashes.
|
||||
1. **§4: settled 2026-09-10 — call ServUO's vendored `Ultima`.** No decoders reimplemented. The
|
||||
three accepted costs are in §4.2; the crash is confined to the `hasExtra: true` branch that only
|
||||
`Gumps` reaches, and nothing here calls `Gumps`.
|
||||
2. **§17.1's consequence: Linux/Mono shards.** §4.2 item 2 leaves `System.Drawing` in the decode
|
||||
path, so a non-Windows shard needs **libgdiplus** installed or art extraction fails there. Three
|
||||
ways to answer it, and this is the one real question left: document it as a prerequisite in
|
||||
`SHARD_PREREQS.md` and let it fail loudly; have the installer's `doctor` detect and report it;
|
||||
or fall back to no-art on that platform with a status the panel explains. Phase 0 should
|
||||
establish which failure it actually is before we pick.
|
||||
3. **§5.1/§5.2: direction — settled 2026-09-10.** Player character bodies use index 0, everything
|
||||
else index 1, direction is not in the key, and the player-body set is enumerated from
|
||||
`Race.AllRaces` rather than hardcoded. Nothing outstanding; recorded here because it changes
|
||||
|
||||
Reference in New Issue
Block a user