diff --git a/link/v8.md b/link/v8.md index ee34cfa..35c2797 100644 --- a/link/v8.md +++ b/link/v8.md @@ -139,46 +139,57 @@ would exceed the cap. Base64 costs 33%; the budget must be stated in encoded byt --- -## 4. The decoders: whose code, and the crash that decides it +## 4. The decoders are ServUO's own — decided, and the crash is narrower than it looked -§1.1 found that `Gumps.GetGump(2)` does not fail — it **corrupts the process**. An -`AccessViolationException` from `unsafe` pointer code is a corrupted-state exception; on .NET -Framework 4.8 it is *not catchable* by an ordinary `try/catch`. In-process, on a live shard, that -is a shard crash with players on it. +**We call ServUO's vendored `Ultima` (decided 2026-09-10).** No decoders are reimplemented. +`overlay/Scripts/Scripts.csproj:39` already references the project, so the art half of this protocol +costs plumbing rather than pixel code, and only §9's cliloc decompressor is written from scratch. -Art and animation probed clean across ~66,000 and ~2,000 ids respectively. That is reassuring and -it is not a guarantee: the inputs we have not tested are exactly the ones that matter — a shard's -own **patched or custom** client files, which is the population this feature exists to serve. +The reason that is safe, rather than merely cheap, is a distinction §1.1 did not draw at first. -Two ways to hold this safely: +### 4.1 The crash lives on one code path, and nothing we call uses it -**A1 — call ServUO's vendored `Ultima`, isolate the fault.** Fastest to build, and proven for the -kinds we need. Requires running the decode where a fault costs one batch rather than the shard: -a short-lived child process. That means a binary to deploy, which the overlay (deployed as -*source*, compiled by ServUO at boot) has no mechanism for. +`Gumps.GetGump(2)` does not fail — it **corrupts the process**: `AccessViolationException`, exit +`0xC0000005`. That is a corrupted-state exception, uncatchable by an ordinary `try/catch` on .NET +Framework 4.8, so in-process on a live shard it is a crash with players on it. That much is +alarming, and on its own it looked like an argument against using this library at all. -**A2 — own bounds-checked decoders in the overlay.** Roughly 600–900 lines of ordinary safe C#: -`FileIndex` (~150), the RLE frame decoder (~60, and it is the same loop, writing to a `ushort[]` -instead of through a `LockBits` pointer), the static-art decoder, `Body.def`/`Bodyconv.def`, hue -application, and a minimal PNG writer over `System.IO.Compression.DeflateStream` (~80). +It is not, because of how the three decoders construct their `FileIndex`: -**A2 is recommended**, for four reasons that compound: +| Decoder | UOP file | `hasExtra` | Probed | +|---|---|---|---| +| `Art` | `artLegacyMUL.uop` | **false** | 49,150 statics + 16,384 land tiles, **0 faults** | +| `Animations` | *none — legacy `anim*.mul` only* | — | 1,144 bodies, **0 faults** | +| `Gumps` | `gumpartLegacyMUL.uop` | **true** | **faults on the second id** | -1. **It removes the crash class**, rather than containing it. -2. **It removes `System.Drawing` entirely.** ServUO's `Frame` decoder writes ARGB1555 straight - into a `Bitmap` via `LockBits`, so System.Drawing is in the *decode*, not just the encode — a - Linux/Mono shard needs libgdiplus even to read a sprite. Writing our own pixels and our own PNG - makes the feature portable by construction. -3. **It removes the UOP gap, which lands squarely on the player bodies.** ServUO's vendored - `Animations` reads legacy `anim*.mul` only, never `AnimationFrame*.uop`. This client has a full - 195 MB `anim.mul` so most creatures resolve — but **six of the twelve stock player-character - bodies return nothing**, including every gargoyle and both human ghosts (§5.2). A missing - playable race in an asset store built for "player models and everything" is not a caveat, it is - a defect, and it is in the highest-attention part of the scope. -4. **We are already in this business.** §9 writes a cliloc decompressor from scratch regardless. +`FileIndex.cs`'s own comment says the extra-field handling exists *for* `gumpartlegacy.uop` — it is +the one UOP layout carrying an extra field, and `hasExtra: true` is the branch written to cope with +it. **Gumps is the only caller that sets it.** So the fault is not a general fragility in this +library's `unsafe` code; it is a bug on a branch that exactly one decoder reaches, and that decoder +is already out of scope (§11). -A2 also ends the dependency on whatever version of `Ultima` a given ServUO happens to vendor, -which is the kind of thing that silently changes under a shard upgrade. +The rule this turns into is a safety rule, not a preference: **nothing in this protocol calls +`Ultima.Gumps`.** Adding gump art later means fixing or replacing that path first, deliberately, +not discovering it in production. + +### 4.2 What the decision accepts + +Three costs come with it, all known and none of them blocking: + +1. **Six of the twelve stock player-character bodies have no art** — both human ghosts and every + gargoyle body (§5.2). `Animations` never reads `AnimationFrame*.uop`, and UOP animation is a + different container with its own `AnimationSequence.uop`, not a mirror of the mul layout, so + this is not reachable by calling the existing code differently. Those bodies degrade to no + image, which is the same state every creature is in today. +2. **`System.Drawing` is a hard dependency, in the decode and not just the encode.** `Frame` + writes ARGB1555 straight through a `LockBits` pointer, so a Linux/Mono shard needs libgdiplus + to read a sprite at all. See §17. +3. **We inherit whatever `Ultima` a given ServUO vendors**, which can change under a shard upgrade. + `EXTRACTOR_VERSION` (§7) is the mitigation: it already counts as drift, so a shard whose library + changed re-derives on the next import. + +The residual risk that remains is a patched or custom client tripping an out-of-bounds read on a +path we *do* call. §16's phase 0 is where that gets exercised rather than assumed. --- @@ -270,10 +281,14 @@ the **opposite order** to the other two races (695 male, 694 female), and a shar | Elf male/female (605, 606) | **Every gargoyle body (666, 667, 694, 695)** | | Elf ghosts (607, 608) | | -Six of twelve, including a whole playable race. So the one part of the asset scope with the most -attention on it — the player character, head-on — is precisely the part the vendored `Animations` -serves worst, and §4's recommendation to own the decoders is what fixes it. This is the single -strongest piece of evidence for that choice. +Six of twelve, including a whole playable race — and §4 accepts that rather than reimplementing +`Animations` to reach `AnimationFrame*.uop`. Those bodies render without an image, exactly as every +creature does today. + +Two things follow for the build. The catalogue must **not** treat a missing player body as an +error — it is the expected answer for half the set, and a status screen that flags six failures on +every import will teach an operator to ignore it. And `shard_spawn_creatures.art` staying NULL has +to remain a first-class state everywhere it is consumed, which it already is. --- @@ -356,8 +371,8 @@ Every modern client ships `Cliloc.*` in the Mythic compressed container — this the shard's own `VendorSearch.GetItemName` is already inert. **UOFiddler is released under the Beerware licence**, so porting its decompressor into our -GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C#, alongside §4's decoders, -and from that point: +GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C# — the **only** decoder +Protocol 8 writes rather than calls (§4) — and from that point: - No operator installs UOFiddler. - No operator runs `dotnet build` on a converter. @@ -440,10 +455,10 @@ bulk-fill-everything switch rather than assuming on-demand is the only mode. actually carries hue 33. The cross product of 49,150 statics and ~3,000 hues is not a set anyone enumerates. -**Gump art is out of scope for Protocol 8** — see §4; it is the one kind whose vendored decoder -demonstrably corrupts the process, and A2 would have to reimplement it against the UOP-with-extra- -field layout that broke it. It is additive to add later under this same key scheme -(`gump/`), which is the point of §5. +**Gump art is out of scope for Protocol 8, and that is now a safety rule rather than a priority +call** — §4.1. It is the only decoder that reaches the `hasExtra: true` branch, and that branch +corrupts the process on the second id. Adding gump art later means fixing that path first, +deliberately; it is additive under the same key scheme (`gump/`), which is the point of §5. --- @@ -510,7 +525,7 @@ disagree, so a split bump means the next bundle silently fails to compose. | Repo | Work | |---|---| -| `servuo-plugins/` | The decoders (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` | +| `servuo-plugins/` | Extraction over ServUO's own `Ultima` (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` | | `link/` | Six command families forwarded, the REST surface, **the inbound line cap (§3.3)**, `PROTOCOL_VERSION` | | `module-uo/` | Client calls, asset store, the atlas source backend (§10), cliloc ingest, admin surface | | `website/` | None expected — `ctx.uploads` already suffices (§12) | @@ -525,7 +540,7 @@ disagree, so a split bump means the next bundle silently fails to compose. | # | Scope | Repos | |---|---|---| -| 0 | Spike: A2's decoders against this machine's client — statics, land, one body, the Mythic cliloc — proving output matches UOFiddler's byte for byte | servuo-plugins | +| 0 | Spike: the vendored decoders driven **from inside a running ServUO**, over a deliberately patched client — statics, land, bodies, and the Mythic cliloc against UOFiddler's output. What it is looking for is a fault on a path we call (§4.2) | servuo-plugins | | 1 | The transport: `assets.sources`, flow control, the sidecar line cap, `EXTRACTOR_VERSION`, protocol bump | servuo-plugins, link | | 2 | Clilocs end to end; retire the converter and `UOFIDDLER.md` §Part 1 | all | | 3 | Body resolution (§8) + the 1,144-body catalogue; `shard_spawn_creatures.art` filled | servuo-plugins, module-uo | @@ -535,18 +550,25 @@ disagree, so a split bump means the next bundle silently fails to compose. | 7 | Admin surface, Import/Update, approve/reject, activity log | module-uo | | 8 | Docs pass across five repos; live walk on the real rig | docs | -Phase 0 exists because §4 chose to own the decoders, and the honest way to hold that choice is to -prove byte-identical output *before* building six phases on top of it. +Phase 0 exists because §4 chose to call code that can take the shard down if it is wrong, and the +honest way to hold that choice is to try to break it on purpose — in the real host process, against +a client that has been patched — *before* building seven phases on top of it. The probes behind +§1.1 were run from PowerShell against a stock client; neither of those is the environment this will +actually run in. --- ## 17. Decisions still open -1. **§4: A2 (own decoders) versus A1 (vendored `Ultima` in a child process).** A2 is recommended - and the phase plan assumes it. It costs a spike and ~600–900 lines; it buys the crash class, the - libgdiplus dependency, the UOP gap and the vendor-drift risk all going away at once. -2. **§11: gump art deferred.** Confirm that paperdoll and equipment gump art is genuinely - out of scope for 8, given it is the one kind whose existing decoder crashes. +1. **§4: settled 2026-09-10 — call ServUO's vendored `Ultima`.** No decoders reimplemented. The + three accepted costs are in §4.2; the crash is confined to the `hasExtra: true` branch that only + `Gumps` reaches, and nothing here calls `Gumps`. +2. **§17.1's consequence: Linux/Mono shards.** §4.2 item 2 leaves `System.Drawing` in the decode + path, so a non-Windows shard needs **libgdiplus** installed or art extraction fails there. Three + ways to answer it, and this is the one real question left: document it as a prerequisite in + `SHARD_PREREQS.md` and let it fail loudly; have the installer's `doctor` detect and report it; + or fall back to no-art on that platform with a status the panel explains. Phase 0 should + establish which failure it actually is before we pick. 3. **§5.1/§5.2: direction — settled 2026-09-10.** Player character bodies use index 0, everything else index 1, direction is not in the key, and the player-body set is enumerated from `Race.AllRaces` rather than hardcoded. Nothing outstanding; recorded here because it changes