RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(link): call ServUO's decoders — the crash is on a branch we never take

Browse Source 
Decision: no decoders are reimplemented. Extraction goes through ServUO's own
vendored Ultima, which Scripts.csproj already references, so the art half of
this protocol is plumbing and the Mythic cliloc reader (§9) becomes the only
decoder Protocol 8 writes rather than calls.

What makes that safe rather than merely cheap is a distinction §1.1 did not
draw. All three decoders build a FileIndex, but only Gumps passes
hasExtra: true — and FileIndex.cs's own comment says that branch exists FOR
gumpartlegacy.uop, the one UOP layout with an extra field. Art passes
hasExtra: false and probed 49,150 statics plus 16,384 land tiles with zero
faults; Animations touches no UOP at all and probed 1,144 bodies clean. The
access violation is a bug on a branch exactly one decoder reaches, and that
decoder was already out of scope. So "nothing calls Ultima.Gumps" is now a
safety rule, and adding gump art later means fixing that path first.

§4.2 records the three costs this accepts: six of twelve stock player bodies
have no art (UOP-only, not reachable by calling the existing code differently),
System.Drawing stays in the decode path, and we inherit whatever Ultima a shard
vendors — EXTRACTOR_VERSION already covers the last one.

Phase 0 changes shape with it. It was going to prove new decoders byte-identical;
it now tries to BREAK the vendored ones on purpose, from inside a running ServUO
against a deliberately patched client, because the probes behind §1.1 ran in
PowerShell against a stock client and neither is the real environment.

§17 is down to one real question: libgdiplus on Linux/Mono shards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
claude
2026-09-10 01:11:13 -05:00
parent b62d0b6307
commit 44039e83d5
1 changed files with 73 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files

124
link/v8.md
View File

@@ -139,46 +139,57 @@ would exceed the cap. Base64 costs 33%; the budget must be stated in encoded byt
---
## 4. The decoders: whose code, and the crash that decides it
## 4. The decoders are ServUO's own — decided, and the crash is narrower than it looked
§1.1 found that `Gumps.GetGump(2)` does not fail — it **corrupts the process**. An
`AccessViolationException` from `unsafe` pointer code is a corrupted-state exception; on .NET
Framework 4.8 it is *not catchable* by an ordinary `try/catch`. In-process, on a live shard, that
is a shard crash with players on it.
**We call ServUO's vendored `Ultima` (decided 2026-09-10).** No decoders are reimplemented.
`overlay/Scripts/Scripts.csproj:39` already references the project, so the art half of this protocol
costs plumbing rather than pixel code, and only §9's cliloc decompressor is written from scratch.
Art and animation probed clean across ~66,000 and ~2,000 ids respectively. That is reassuring and
it is not a guarantee: the inputs we have not tested are exactly the ones that matter — a shard's
own **patched or custom** client files, which is the population this feature exists to serve.
The reason that is safe, rather than merely cheap, is a distinction §1.1 did not draw at first.
Two ways to hold this safely:
### 4.1 The crash lives on one code path, and nothing we call uses it
**A1 — call ServUO's vendored `Ultima`, isolate the fault.** Fastest to build, and proven for the
kinds we need. Requires running the decode where a fault costs one batch rather than the shard:
a short-lived child process. That means a binary to deploy, which the overlay (deployed as
*source*, compiled by ServUO at boot) has no mechanism for.
`Gumps.GetGump(2)` does not fail — it **corrupts the process**: `AccessViolationException`, exit
`0xC0000005`. That is a corrupted-state exception, uncatchable by an ordinary `try/catch` on .NET
Framework 4.8, so in-process on a live shard it is a crash with players on it. That much is
alarming, and on its own it looked like an argument against using this library at all.
**A2 — own bounds-checked decoders in the overlay.** Roughly 600–900 lines of ordinary safe C#:
`FileIndex` (~150), the RLE frame decoder (~60, and it is the same loop, writing to a `ushort[]`
instead of through a `LockBits` pointer), the static-art decoder, `Body.def`/`Bodyconv.def`, hue
application, and a minimal PNG writer over `System.IO.Compression.DeflateStream` (~80).
It is not, because of how the three decoders construct their `FileIndex`:
**A2 is recommended**, for four reasons that compound:
| Decoder | UOP file | `hasExtra` | Probed |
|---|---|---|---|
| `Art` | `artLegacyMUL.uop` | **false** | 49,150 statics + 16,384 land tiles, **0 faults** |
| `Animations` | *none — legacy `anim*.mul` only* | — | 1,144 bodies, **0 faults** |
| `Gumps` | `gumpartLegacyMUL.uop` | **true** | **faults on the second id** |
1. **It removes the crash class**, rather than containing it.
2. **It removes `System.Drawing` entirely.** ServUO's `Frame` decoder writes ARGB1555 straight
into a `Bitmap` via `LockBits`, so System.Drawing is in the *decode*, not just the encode — a
Linux/Mono shard needs libgdiplus even to read a sprite. Writing our own pixels and our own PNG
makes the feature portable by construction.
3. **It removes the UOP gap, which lands squarely on the player bodies.** ServUO's vendored
`Animations` reads legacy `anim*.mul` only, never `AnimationFrame*.uop`. This client has a full
195 MB `anim.mul` so most creatures resolve — but **six of the twelve stock player-character
bodies return nothing**, including every gargoyle and both human ghosts (§5.2). A missing
playable race in an asset store built for "player models and everything" is not a caveat, it is
a defect, and it is in the highest-attention part of the scope.
4. **We are already in this business.** §9 writes a cliloc decompressor from scratch regardless.
`FileIndex.cs`'s own comment says the extra-field handling exists *for* `gumpartlegacy.uop` — it is
the one UOP layout carrying an extra field, and `hasExtra: true` is the branch written to cope with
it. **Gumps is the only caller that sets it.** So the fault is not a general fragility in this
library's `unsafe` code; it is a bug on a branch that exactly one decoder reaches, and that decoder
is already out of scope (§11).
A2 also ends the dependency on whatever version of `Ultima` a given ServUO happens to vendor,
which is the kind of thing that silently changes under a shard upgrade.
The rule this turns into is a safety rule, not a preference: **nothing in this protocol calls
`Ultima.Gumps`.** Adding gump art later means fixing or replacing that path first, deliberately,
not discovering it in production.
### 4.2 What the decision accepts
Three costs come with it, all known and none of them blocking:
1. **Six of the twelve stock player-character bodies have no art** — both human ghosts and every
gargoyle body (§5.2). `Animations` never reads `AnimationFrame*.uop`, and UOP animation is a
different container with its own `AnimationSequence.uop`, not a mirror of the mul layout, so
this is not reachable by calling the existing code differently. Those bodies degrade to no
image, which is the same state every creature is in today.
2. **`System.Drawing` is a hard dependency, in the decode and not just the encode.** `Frame`
writes ARGB1555 straight through a `LockBits` pointer, so a Linux/Mono shard needs libgdiplus
to read a sprite at all. See §17.
3. **We inherit whatever `Ultima` a given ServUO vendors**, which can change under a shard upgrade.
`EXTRACTOR_VERSION` (§7) is the mitigation: it already counts as drift, so a shard whose library
changed re-derives on the next import.
The residual risk that remains is a patched or custom client tripping an out-of-bounds read on a
path we *do* call. §16's phase 0 is where that gets exercised rather than assumed.
---
@@ -270,10 +281,14 @@ the **opposite order** to the other two races (695 male, 694 female), and a shar
| Elf male/female (605, 606) | **Every gargoyle body (666, 667, 694, 695)** |
| Elf ghosts (607, 608) | |
Six of twelve, including a whole playable race. So the one part of the asset scope with the most
attention on it — the player character, head-on — is precisely the part the vendored `Animations`
serves worst, and §4's recommendation to own the decoders is what fixes it. This is the single
strongest piece of evidence for that choice.
Six of twelve, including a whole playable race — and §4 accepts that rather than reimplementing
`Animations` to reach `AnimationFrame*.uop`. Those bodies render without an image, exactly as every
creature does today.
Two things follow for the build. The catalogue must **not** treat a missing player body as an
error — it is the expected answer for half the set, and a status screen that flags six failures on
every import will teach an operator to ignore it. And `shard_spawn_creatures.art` staying NULL has
to remain a first-class state everywhere it is consumed, which it already is.
---
@@ -356,8 +371,8 @@ Every modern client ships `Cliloc.*` in the Mythic compressed container — this
the shard's own `VendorSearch.GetItemName` is already inert.
**UOFiddler is released under the Beerware licence**, so porting its decompressor into our
GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C#, alongside §4's decoders,
and from that point:
GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C# — the **only** decoder
Protocol 8 writes rather than calls (§4) — and from that point:
- No operator installs UOFiddler.
- No operator runs `dotnet build` on a converter.
@@ -440,10 +455,10 @@ bulk-fill-everything switch rather than assuming on-demand is the only mode.
actually carries hue 33. The cross product of 49,150 statics and ~3,000 hues is not a set anyone
enumerates.
**Gump art is out of scope for Protocol 8** — see §4; it is the one kind whose vendored decoder
demonstrably corrupts the process, and A2 would have to reimplement it against the UOP-with-extra-
field layout that broke it. It is additive to add later under this same key scheme
(`gump/<id>`), which is the point of §5.
**Gump art is out of scope for Protocol 8, and that is now a safety rule rather than a priority
call** — §4.1. It is the only decoder that reaches the `hasExtra: true` branch, and that branch
corrupts the process on the second id. Adding gump art later means fixing that path first,
deliberately; it is additive under the same key scheme (`gump/<id>`), which is the point of §5.
---
@@ -510,7 +525,7 @@ disagree, so a split bump means the next bundle silently fails to compose.
| Repo | Work |
|---|---|
| `servuo-plugins/` | The decoders (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` |
| `servuo-plugins/` | Extraction over ServUO's own `Ultima` (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` |
| `link/` | Six command families forwarded, the REST surface, **the inbound line cap (§3.3)**, `PROTOCOL_VERSION` |
| `module-uo/` | Client calls, asset store, the atlas source backend (§10), cliloc ingest, admin surface |
| `website/` | None expected — `ctx.uploads` already suffices (§12) |
@@ -525,7 +540,7 @@ disagree, so a split bump means the next bundle silently fails to compose.
| # | Scope | Repos |
|---|---|---|
| 0 | Spike: A2's decoders against this machine's client — statics, land, one body, the Mythic cliloc — proving output matches UOFiddler's byte for byte | servuo-plugins |
| 0 | Spike: the vendored decoders driven **from inside a running ServUO**, over a deliberately patched client — statics, land, bodies, and the Mythic cliloc against UOFiddler's output. What it is looking for is a fault on a path we call (§4.2) | servuo-plugins |
| 1 | The transport: `assets.sources`, flow control, the sidecar line cap, `EXTRACTOR_VERSION`, protocol bump | servuo-plugins, link |
| 2 | Clilocs end to end; retire the converter and `UOFIDDLER.md` §Part 1 | all |
| 3 | Body resolution (§8) + the 1,144-body catalogue; `shard_spawn_creatures.art` filled | servuo-plugins, module-uo |
@@ -535,18 +550,25 @@ disagree, so a split bump means the next bundle silently fails to compose.
| 7 | Admin surface, Import/Update, approve/reject, activity log | module-uo |
| 8 | Docs pass across five repos; live walk on the real rig | docs |
Phase 0 exists because §4 chose to own the decoders, and the honest way to hold that choice is to
prove byte-identical output *before* building six phases on top of it.
Phase 0 exists because §4 chose to call code that can take the shard down if it is wrong, and the
honest way to hold that choice is to try to break it on purpose — in the real host process, against
a client that has been patched — *before* building seven phases on top of it. The probes behind
§1.1 were run from PowerShell against a stock client; neither of those is the environment this will
actually run in.
---
## 17. Decisions still open
1. **§4: A2 (own decoders) versus A1 (vendored `Ultima` in a child process).** A2 is recommended
and the phase plan assumes it. It costs a spike and ~600–900 lines; it buys the crash class, the
libgdiplus dependency, the UOP gap and the vendor-drift risk all going away at once.
2. **§11: gump art deferred.** Confirm that paperdoll and equipment gump art is genuinely
out of scope for 8, given it is the one kind whose existing decoder crashes.
1. **§4: settled 2026-09-10 — call ServUO's vendored `Ultima`.** No decoders reimplemented. The
three accepted costs are in §4.2; the crash is confined to the `hasExtra: true` branch that only
`Gumps` reaches, and nothing here calls `Gumps`.
2. **§17.1's consequence: Linux/Mono shards.** §4.2 item 2 leaves `System.Drawing` in the decode
path, so a non-Windows shard needs **libgdiplus** installed or art extraction fails there. Three
ways to answer it, and this is the one real question left: document it as a prerequisite in
`SHARD_PREREQS.md` and let it fail loudly; have the installer's `doctor` detect and report it;
or fall back to no-art on that platform with a status the panel explains. Phase 0 should
establish which failure it actually is before we pick.
3. **§5.1/§5.2: direction — settled 2026-09-10.** Player character bodies use index 0, everything
else index 1, direction is not in the key, and the player-body set is enumerated from
`Race.AllRaces` rather than hardcoded. Nothing outstanding; recorded here because it changes