RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(link): call ServUO's decoders — the crash is on a branch we never take

Browse Source 
Decision: no decoders are reimplemented. Extraction goes through ServUO's own
vendored Ultima, which Scripts.csproj already references, so the art half of
this protocol is plumbing and the Mythic cliloc reader (§9) becomes the only
decoder Protocol 8 writes rather than calls.

What makes that safe rather than merely cheap is a distinction §1.1 did not
draw. All three decoders build a FileIndex, but only Gumps passes
hasExtra: true — and FileIndex.cs's own comment says that branch exists FOR
gumpartlegacy.uop, the one UOP layout with an extra field. Art passes
hasExtra: false and probed 49,150 statics plus 16,384 land tiles with zero
faults; Animations touches no UOP at all and probed 1,144 bodies clean. The
access violation is a bug on a branch exactly one decoder reaches, and that
decoder was already out of scope. So "nothing calls Ultima.Gumps" is now a
safety rule, and adding gump art later means fixing that path first.

§4.2 records the three costs this accepts: six of twelve stock player bodies
have no art (UOP-only, not reachable by calling the existing code differently),
System.Drawing stays in the decode path, and we inherit whatever Ultima a shard
vendors — EXTRACTOR_VERSION already covers the last one.

Phase 0 changes shape with it. It was going to prove new decoders byte-identical;
it now tries to BREAK the vendored ones on purpose, from inside a running ServUO
against a deliberately patched client, because the probes behind §1.1 ran in
PowerShell against a stock client and neither is the real environment.

§17 is down to one real question: libgdiplus on Linux/Mono shards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
claude
2026-09-10 01:11:13 -05:00
parent b62d0b6307
commit 44039e83d5
1 changed files with 73 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files

124
link/v8.md
View File

@@ -139,46 +139,57 @@ would exceed the cap. Base64 costs 33%; the budget must be stated in encoded byt
--- ---
## 4. The decoders: whose code, and the crash that decides it ## 4. The decoders are ServUO's own — decided, and the crash is narrower than it looked
§1.1 found that `Gumps.GetGump(2)` does not fail — it **corrupts the process**. An **We call ServUO's vendored `Ultima` (decided 2026-09-10).** No decoders are reimplemented.
`AccessViolationException` from `unsafe` pointer code is a corrupted-state exception; on .NET `overlay/Scripts/Scripts.csproj:39` already references the project, so the art half of this protocol
Framework 4.8 it is *not catchable* by an ordinary `try/catch`. In-process, on a live shard, that costs plumbing rather than pixel code, and only §9's cliloc decompressor is written from scratch.
is a shard crash with players on it.
Art and animation probed clean across ~66,000 and ~2,000 ids respectively. That is reassuring and The reason that is safe, rather than merely cheap, is a distinction §1.1 did not draw at first.
it is not a guarantee: the inputs we have not tested are exactly the ones that matter — a shard's
own **patched or custom** client files, which is the population this feature exists to serve.
Two ways to hold this safely: ### 4.1 The crash lives on one code path, and nothing we call uses it
**A1 — call ServUO's vendored `Ultima`, isolate the fault.** Fastest to build, and proven for the `Gumps.GetGump(2)` does not fail — it **corrupts the process**: `AccessViolationException`, exit
kinds we need. Requires running the decode where a fault costs one batch rather than the shard: `0xC0000005`. That is a corrupted-state exception, uncatchable by an ordinary `try/catch` on .NET
a short-lived child process. That means a binary to deploy, which the overlay (deployed as Framework 4.8, so in-process on a live shard it is a crash with players on it. That much is
*source*, compiled by ServUO at boot) has no mechanism for. alarming, and on its own it looked like an argument against using this library at all.
**A2 — own bounds-checked decoders in the overlay.** Roughly 600–900 lines of ordinary safe C#: It is not, because of how the three decoders construct their `FileIndex`:
`FileIndex` (~150), the RLE frame decoder (~60, and it is the same loop, writing to a `ushort[]`
instead of through a `LockBits` pointer), the static-art decoder, `Body.def`/`Bodyconv.def`, hue
application, and a minimal PNG writer over `System.IO.Compression.DeflateStream` (~80).
**A2 is recommended**, for four reasons that compound: | Decoder | UOP file | `hasExtra` | Probed |
|---|---|---|---|
| `Art` | `artLegacyMUL.uop` | **false** | 49,150 statics + 16,384 land tiles, **0 faults** |
| `Animations` | *none — legacy `anim*.mul` only* | — | 1,144 bodies, **0 faults** |
| `Gumps` | `gumpartLegacyMUL.uop` | **true** | **faults on the second id** |
1. **It removes the crash class**, rather than containing it. `FileIndex.cs`'s own comment says the extra-field handling exists *for* `gumpartlegacy.uop` — it is
2. **It removes `System.Drawing` entirely.** ServUO's `Frame` decoder writes ARGB1555 straight the one UOP layout carrying an extra field, and `hasExtra: true` is the branch written to cope with
into a `Bitmap` via `LockBits`, so System.Drawing is in the *decode*, not just the encode — a it. **Gumps is the only caller that sets it.** So the fault is not a general fragility in this
Linux/Mono shard needs libgdiplus even to read a sprite. Writing our own pixels and our own PNG library's `unsafe` code; it is a bug on a branch that exactly one decoder reaches, and that decoder
makes the feature portable by construction. is already out of scope (§11).
3. **It removes the UOP gap, which lands squarely on the player bodies.** ServUO's vendored
`Animations` reads legacy `anim*.mul` only, never `AnimationFrame*.uop`. This client has a full
195 MB `anim.mul` so most creatures resolve — but **six of the twelve stock player-character
bodies return nothing**, including every gargoyle and both human ghosts (§5.2). A missing
playable race in an asset store built for "player models and everything" is not a caveat, it is
a defect, and it is in the highest-attention part of the scope.
4. **We are already in this business.** §9 writes a cliloc decompressor from scratch regardless.
A2 also ends the dependency on whatever version of `Ultima` a given ServUO happens to vendor, The rule this turns into is a safety rule, not a preference: **nothing in this protocol calls
which is the kind of thing that silently changes under a shard upgrade. `Ultima.Gumps`.** Adding gump art later means fixing or replacing that path first, deliberately,
not discovering it in production.
### 4.2 What the decision accepts
Three costs come with it, all known and none of them blocking:
1. **Six of the twelve stock player-character bodies have no art** — both human ghosts and every
gargoyle body (§5.2). `Animations` never reads `AnimationFrame*.uop`, and UOP animation is a
different container with its own `AnimationSequence.uop`, not a mirror of the mul layout, so
this is not reachable by calling the existing code differently. Those bodies degrade to no
image, which is the same state every creature is in today.
2. **`System.Drawing` is a hard dependency, in the decode and not just the encode.** `Frame`
writes ARGB1555 straight through a `LockBits` pointer, so a Linux/Mono shard needs libgdiplus
to read a sprite at all. See §17.
3. **We inherit whatever `Ultima` a given ServUO vendors**, which can change under a shard upgrade.
`EXTRACTOR_VERSION` (§7) is the mitigation: it already counts as drift, so a shard whose library
changed re-derives on the next import.
The residual risk that remains is a patched or custom client tripping an out-of-bounds read on a
path we *do* call. §16's phase 0 is where that gets exercised rather than assumed.
--- ---
@@ -270,10 +281,14 @@ the **opposite order** to the other two races (695 male, 694 female), and a shar
| Elf male/female (605, 606) | **Every gargoyle body (666, 667, 694, 695)** | | Elf male/female (605, 606) | **Every gargoyle body (666, 667, 694, 695)** |
| Elf ghosts (607, 608) | | | Elf ghosts (607, 608) | |
Six of twelve, including a whole playable race. So the one part of the asset scope with the most Six of twelve, including a whole playable race — and §4 accepts that rather than reimplementing
attention on it — the player character, head-on — is precisely the part the vendored `Animations` `Animations` to reach `AnimationFrame*.uop`. Those bodies render without an image, exactly as every
serves worst, and §4's recommendation to own the decoders is what fixes it. This is the single creature does today.
strongest piece of evidence for that choice.
Two things follow for the build. The catalogue must **not** treat a missing player body as an
error — it is the expected answer for half the set, and a status screen that flags six failures on
every import will teach an operator to ignore it. And `shard_spawn_creatures.art` staying NULL has
to remain a first-class state everywhere it is consumed, which it already is.
--- ---
@@ -356,8 +371,8 @@ Every modern client ships `Cliloc.*` in the Mythic compressed container — this
the shard's own `VendorSearch.GetItemName` is already inert. the shard's own `VendorSearch.GetItemName` is already inert.
**UOFiddler is released under the Beerware licence**, so porting its decompressor into our **UOFiddler is released under the Beerware licence**, so porting its decompressor into our
GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C#, alongside §4's decoders, GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C# — the **only** decoder
and from that point: Protocol 8 writes rather than calls (§4) — and from that point:
- No operator installs UOFiddler. - No operator installs UOFiddler.
- No operator runs `dotnet build` on a converter. - No operator runs `dotnet build` on a converter.
@@ -440,10 +455,10 @@ bulk-fill-everything switch rather than assuming on-demand is the only mode.
actually carries hue 33. The cross product of 49,150 statics and ~3,000 hues is not a set anyone actually carries hue 33. The cross product of 49,150 statics and ~3,000 hues is not a set anyone
enumerates. enumerates.
**Gump art is out of scope for Protocol 8** — see §4; it is the one kind whose vendored decoder **Gump art is out of scope for Protocol 8, and that is now a safety rule rather than a priority
demonstrably corrupts the process, and A2 would have to reimplement it against the UOP-with-extra- call** — §4.1. It is the only decoder that reaches the `hasExtra: true` branch, and that branch
field layout that broke it. It is additive to add later under this same key scheme corrupts the process on the second id. Adding gump art later means fixing that path first,
(`gump/<id>`), which is the point of §5. deliberately; it is additive under the same key scheme (`gump/<id>`), which is the point of §5.
--- ---
@@ -510,7 +525,7 @@ disagree, so a split bump means the next bundle silently fails to compose.
| Repo | Work | | Repo | Work |
|---|---| |---|---|
| `servuo-plugins/` | The decoders (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` | | `servuo-plugins/` | Extraction over ServUO's own `Ultima` (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` |
| `link/` | Six command families forwarded, the REST surface, **the inbound line cap (§3.3)**, `PROTOCOL_VERSION` | | `link/` | Six command families forwarded, the REST surface, **the inbound line cap (§3.3)**, `PROTOCOL_VERSION` |
| `module-uo/` | Client calls, asset store, the atlas source backend (§10), cliloc ingest, admin surface | | `module-uo/` | Client calls, asset store, the atlas source backend (§10), cliloc ingest, admin surface |
| `website/` | None expected — `ctx.uploads` already suffices (§12) | | `website/` | None expected — `ctx.uploads` already suffices (§12) |
@@ -525,7 +540,7 @@ disagree, so a split bump means the next bundle silently fails to compose.
| # | Scope | Repos | | # | Scope | Repos |
|---|---|---| |---|---|---|
| 0 | Spike: A2's decoders against this machine's client — statics, land, one body, the Mythic cliloc — proving output matches UOFiddler's byte for byte | servuo-plugins | | 0 | Spike: the vendored decoders driven **from inside a running ServUO**, over a deliberately patched client — statics, land, bodies, and the Mythic cliloc against UOFiddler's output. What it is looking for is a fault on a path we call (§4.2) | servuo-plugins |
| 1 | The transport: `assets.sources`, flow control, the sidecar line cap, `EXTRACTOR_VERSION`, protocol bump | servuo-plugins, link | | 1 | The transport: `assets.sources`, flow control, the sidecar line cap, `EXTRACTOR_VERSION`, protocol bump | servuo-plugins, link |
| 2 | Clilocs end to end; retire the converter and `UOFIDDLER.md` §Part 1 | all | | 2 | Clilocs end to end; retire the converter and `UOFIDDLER.md` §Part 1 | all |
| 3 | Body resolution (§8) + the 1,144-body catalogue; `shard_spawn_creatures.art` filled | servuo-plugins, module-uo | | 3 | Body resolution (§8) + the 1,144-body catalogue; `shard_spawn_creatures.art` filled | servuo-plugins, module-uo |
@@ -535,18 +550,25 @@ disagree, so a split bump means the next bundle silently fails to compose.
| 7 | Admin surface, Import/Update, approve/reject, activity log | module-uo | | 7 | Admin surface, Import/Update, approve/reject, activity log | module-uo |
| 8 | Docs pass across five repos; live walk on the real rig | docs | | 8 | Docs pass across five repos; live walk on the real rig | docs |
Phase 0 exists because §4 chose to own the decoders, and the honest way to hold that choice is to Phase 0 exists because §4 chose to call code that can take the shard down if it is wrong, and the
prove byte-identical output *before* building six phases on top of it. honest way to hold that choice is to try to break it on purpose — in the real host process, against
a client that has been patched — *before* building seven phases on top of it. The probes behind
§1.1 were run from PowerShell against a stock client; neither of those is the environment this will
actually run in.
--- ---
## 17. Decisions still open ## 17. Decisions still open
1. **§4: A2 (own decoders) versus A1 (vendored `Ultima` in a child process).** A2 is recommended 1. **§4: settled 2026-09-10 — call ServUO's vendored `Ultima`.** No decoders reimplemented. The
and the phase plan assumes it. It costs a spike and ~600–900 lines; it buys the crash class, the three accepted costs are in §4.2; the crash is confined to the `hasExtra: true` branch that only
libgdiplus dependency, the UOP gap and the vendor-drift risk all going away at once. `Gumps` reaches, and nothing here calls `Gumps`.
2. **§11: gump art deferred.** Confirm that paperdoll and equipment gump art is genuinely 2. **§17.1's consequence: Linux/Mono shards.** §4.2 item 2 leaves `System.Drawing` in the decode
out of scope for 8, given it is the one kind whose existing decoder crashes. path, so a non-Windows shard needs **libgdiplus** installed or art extraction fails there. Three
ways to answer it, and this is the one real question left: document it as a prerequisite in
`SHARD_PREREQS.md` and let it fail loudly; have the installer's `doctor` detect and report it;
or fall back to no-art on that platform with a status the panel explains. Phase 0 should
establish which failure it actually is before we pick.
3. **§5.1/§5.2: direction — settled 2026-09-10.** Player character bodies use index 0, everything 3. **§5.1/§5.2: direction — settled 2026-09-10.** Player character bodies use index 0, everything
else index 1, direction is not in the key, and the player-body set is enumerated from else index 1, direction is not in the key, and the player-body set is enumerated from
`Race.AllRaces` rather than hardcoded. Nothing outstanding; recorded here because it changes `Race.AllRaces` rather than hardcoded. Nothing outstanding; recorded here because it changes