feat(asset-bridge): phase 0 spike — the decoders, from inside a live shard #27

wtclaude · 2026-09-10T07:59:43Z

wtclaude commented

2026-09-10 07:59:43 +00:00

Asset Bridge phase 0 (docs/link/v8.md §16). Docs half: RunicGateway/docs#235.

§4 chose to call ServUO's vendored Ultima rather than reimplement it, and the evidence was a PowerShell probe against a stock client — neither the process nor the client the extractor will run in. This runs the same decoders from inside a running ServUO 57.4, against a client broken in 21 catalogued ways.

§4's decision stands. Nothing faulted on a path this protocol calls. But the spike was looking for the wrong kind of failure.

The finding: 22,102 wrong pictures on a stock, unmodified client

statics 0..65535   ok 39,189   WRONG PICTURES (empty record)  9,962   threw 16,385
land    0..16383   ok  4,244   WRONG PICTURES (empty record) 12,140

Those ids have an index entry of lookup 0, length 0 — no record at all. FileIndex.Seek rejects lookup < 0 and length < 0, and zero is neither, so it treats an empty slot as a hit. LoadStatic then decodes 0 bytes into m_StreamBuffer — reused, only ever grown, filled by a stream.Read whose return value is discarded — so the id renders whatever the previously-decoded asset left behind.

It is specific to the UOP path: artidx.mul stores -1 for an absent record, while an unmapped UOP slot is a zeroed struct. That is why the earlier probe counted 32,766 of these as "ok" — they decode, they raise nothing, and no success count can tell them from art.

A bulk import that trusted the library would have written 22,102 duplicate images into the site under ids that have no art.

The validator, measured both ways

BridgeAssetValidator prototypes the validate-before-calling response chosen for §4.2's residual risk. Against the patched client it refused all eight record-level defects — seven of which the library rendered without raising anything:

id	defect	the library
`static/4104`	lookup past EOF	returns nothing — `Seek` does check the record's start
`static/4105`	starts 64 B before EOF, declares 8,192	renders the previous asset — `Seek` never checks the end
`static/4109`	header declares 8000×8000	allocates it — ~128 MB from two bytes; the field can ask for 8 GB
`static/4111`	row table 60,000 words outside a 512-byte record	renders — `LoadStatic`'s guards bound the write, nothing bounds the read
`static/4131`	verdata lookup past verdata.mul's own end	renders — `Verdata.Seek` is bounds-checked nowhere
`land/256`	512-byte land record	renders — `LoadLand` reads a fixed 2,024 B regardless

Against the stock client it refused nothing across 49,151 statics and 16,384 land tiles. That second number is what makes the boundary defensible — a checker that refuses real art is worse than no checker.

The UOP wins outright, and it cost a run to learn

FileIndex's UOP constructor ends with a bare MulPath = uopPath. artLegacyMUL.uop wins and art.mul/artidx.mul are never opened on a current client. The first run of this probe refused 34,299 perfectly good statics for "declaring 10533x2085" because it bounded UOP offsets against art.mul — and every one of those refusals looked like a real finding. It also means a custom-art shard adding graphics to art.mul while the UOP is present gets nothing, silently.

The gump crash, reproduced where it counts

One Ultima.Gumps.GetGump(2) from inside the shard and the ServUO process disappeared — no exception line, no catch reached, no console output. The report ends mid-section and checkpoint.txt reading gump 2 is the entire record, which is why the checkpoint is written before the call. §4.1's "nothing calls Ultima.Gumps" is now earned rather than argued.

§9 is proven

123,490 entries in 218 ms (55,986 blank, 67,504 would be stored)
vs UOFiddler: 123,490 identical, 0 differ, 0 only ours, 0 only theirs

No UOFiddler installed, no dotnet build, no 5 MB file copied to a server. The reference is what makes it a test: a subtly wrong inverse-BWT coder still yields a plausible table, and a row count would sail past it.

What's here

All under tools/, so nothing is deployed — deploy.ps1 copies only overlay/.

tools/scaffolding/BridgeAssetProbe.cs — the sweep and BridgeAssetValidator. Runs off the Core thread (§8's split, rehearsed), snapshots Race.AllRaces on it, and checkpoints the id it is about to touch.
tools/scaffolding/BridgeMythicCliloc.cs — the §9 reader in net48 C#, ported from UOFiddler (Beerware) with every file-derived index bounds-checked, which upstream's blanket catch does not do. Phase 2 promotes it into overlay/.
tools/patch_client.ps1 — builds the patched client in five tiers (nouop, verdata, customart, corrupt, bodyconv). Hashes every file it touches in the source before and after and aborts on a change; catalogues all 21 defects to a manifest beside the copy.
an assetprobe verb on BridgeRigDriver, so stock and patched run against one boot.
the full write-up in tools/scaffolding/README.md.

Left for phase 1, deliberately

The animation path has no validator. The patched client's verdata entry for body 34 points past verdata.mul's end and the wolf still "decoded" — counted among the 1,144 successes, silently rendering something else. GetAnimation also allocates new int[frameCount] from a file-supplied int. v8.md §16 now names this in phase 1.

Also reproduced exactly, from a different process against the same files: 1,144 decodable bodies, 904 empty, 0 faults, and 6 of the 12 player-character bodies absent. Note the gargoyle ghost bodies resolve to file type 1, not 5.

No protocol version change here — phase 0 adds no wire surface.

AI-assisted — Claude Code (Opus 5)

🤖 Generated with Claude Code

https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4

**Asset Bridge phase 0** ([`docs/link/v8.md` §16](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/main/link/v8.md)). Docs half: RunicGateway/docs#235. §4 chose to **call** ServUO's vendored `Ultima` rather than reimplement it, and the evidence was a PowerShell probe against a stock client — neither the process nor the client the extractor will run in. This runs the same decoders from inside a running ServUO 57.4, against a client broken in 21 catalogued ways. **§4's decision stands.** Nothing faulted on a path this protocol calls. But the spike was looking for the wrong kind of failure. ## The finding: 22,102 wrong pictures on a *stock, unmodified* client ``` statics 0..65535 ok 39,189 WRONG PICTURES (empty record) 9,962 threw 16,385 land 0..16383 ok 4,244 WRONG PICTURES (empty record) 12,140 ``` Those ids have an index entry of `lookup 0, length 0` — **no record at all**. `FileIndex.Seek` rejects `lookup < 0` and `length < 0`, and zero is neither, so it treats an empty slot as a hit. `LoadStatic` then decodes 0 bytes into `m_StreamBuffer` — reused, only ever grown, filled by a `stream.Read` whose return value is discarded — so the id renders **whatever the previously-decoded asset left behind**. It is specific to the UOP path: `artidx.mul` stores `-1` for an absent record, while an unmapped UOP slot is a zeroed struct. That is why the earlier probe counted 32,766 of these as "ok" — they decode, they raise nothing, and no success count can tell them from art. A bulk import that trusted the library would have written 22,102 duplicate images into the site under ids that have no art. ## The validator, measured both ways `BridgeAssetValidator` prototypes the validate-before-calling response chosen for §4.2's residual risk. Against the patched client it refused all eight record-level defects — **seven of which the library rendered without raising anything**: | id | defect | the library | |---|---|---| | `static/4104` | lookup past EOF | returns nothing — `Seek` does check the record's **start** | | `static/4105` | starts 64 B before EOF, declares 8,192 | **renders the previous asset** — `Seek` never checks the **end** | | `static/4109` | header declares 8000×8000 | **allocates it** — ~128 MB from two bytes; the field can ask for 8 GB | | `static/4111` | row table 60,000 words outside a 512-byte record | renders — `LoadStatic`'s guards bound the *write*, nothing bounds the *read* | | `static/4131` | verdata lookup past verdata.mul's own end | renders — **`Verdata.Seek` is bounds-checked nowhere** | | `land/256` | 512-byte land record | renders — `LoadLand` reads a fixed 2,024 B regardless | Against the **stock** client it refused **nothing** across 49,151 statics and 16,384 land tiles. That second number is what makes the boundary defensible — a checker that refuses real art is worse than no checker. ## The UOP wins outright, and it cost a run to learn `FileIndex`'s UOP constructor ends with a bare `MulPath = uopPath`. **`artLegacyMUL.uop` wins and `art.mul`/`artidx.mul` are never opened on a current client.** The first run of this probe refused **34,299 perfectly good statics** for "declaring 10533x2085" because it bounded UOP offsets against `art.mul` — and every one of those refusals looked like a real finding. It also means a custom-art shard adding graphics to `art.mul` while the UOP is present gets nothing, silently. ## The gump crash, reproduced where it counts One `Ultima.Gumps.GetGump(2)` from inside the shard and **the ServUO process disappeared** — no exception line, no `catch` reached, no console output. The report ends mid-section and `checkpoint.txt` reading `gump 2` is the entire record, which is why the checkpoint is written *before* the call. §4.1's "nothing calls `Ultima.Gumps`" is now earned rather than argued. ## §9 is proven ``` 123,490 entries in 218 ms (55,986 blank, 67,504 would be stored) vs UOFiddler: 123,490 identical, 0 differ, 0 only ours, 0 only theirs ``` No UOFiddler installed, no `dotnet build`, no 5 MB file copied to a server. The reference is what makes it a test: a subtly wrong inverse-BWT coder still yields a plausible table, and a row count would sail past it. ## What's here All under `tools/`, so **nothing is deployed** — `deploy.ps1` copies only `overlay/`. - **`tools/scaffolding/BridgeAssetProbe.cs`** — the sweep and `BridgeAssetValidator`. Runs off the Core thread (§8's split, rehearsed), snapshots `Race.AllRaces` on it, and checkpoints the id it is *about* to touch. - **`tools/scaffolding/BridgeMythicCliloc.cs`** — the §9 reader in net48 C#, ported from UOFiddler (Beerware) with every file-derived index bounds-checked, which upstream's blanket `catch` does not do. **Phase 2 promotes it into `overlay/`.** - **`tools/patch_client.ps1`** — builds the patched client in five tiers (`nouop`, `verdata`, `customart`, `corrupt`, `bodyconv`). Hashes every file it touches **in the source** before and after and aborts on a change; catalogues all 21 defects to a manifest beside the copy. - an `assetprobe` verb on `BridgeRigDriver`, so stock and patched run against **one boot**. - the full write-up in `tools/scaffolding/README.md`. ## Left for phase 1, deliberately **The animation path has no validator.** The patched client's verdata entry for body 34 points past verdata.mul's end and the wolf still "decoded" — counted among the 1,144 successes, silently rendering something else. `GetAnimation` also allocates `new int[frameCount]` from a file-supplied int. v8.md §16 now names this in phase 1. Also reproduced exactly, from a different process against the same files: 1,144 decodable bodies, 904 empty, 0 faults, and 6 of the 12 player-character bodies absent. Note the gargoyle *ghost* bodies resolve to file type **1**, not 5. No protocol version change here — phase 0 adds no wire surface. - [x] AI-assisted — Claude Code (Opus 5) 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4

wtclaude referenced this issue from a commit

2026-09-10 08:00:01 +00:00

docs(link): phase 0 ran, and the fault it found is not a crash

wtclaude referenced this pull request from RunicGateway/docs

2026-09-10 08:00:22 +00:00

docs(link): phase 0 ran, and the fault it found is not a crash #235

wtclaude changed target branch from main to edge

2026-09-10 08:05:49 +00:00

wtclaude added 1 commit 2026-09-10 08:05:49 +00:00

feat(asset-bridge): phase 0 spike — the decoders, from inside a live shard 0ce92152a1

docs/link/v8.md §16 phase 0. §4 chose to CALL ServUO's vendored `Ultima`
rather than reimplement it, on the evidence of a PowerShell probe against a
stock client — neither the process nor the client the extractor will run in.
This runs the same decoders from inside a running ServUO 57.4 against a
client broken in 21 catalogued ways, and it found more than a crash.

Adds, all under tools/ and therefore never deployed:

  * BridgeAssetProbe.cs — the sweep, plus BridgeAssetValidator, a prototype
    of the validate-before-calling response chosen for §4.2's residual risk.
    Runs off the Core thread, snapshots Race.AllRaces on it, and writes the
    id it is ABOUT to touch to a checkpoint file before every call.
  * BridgeMythicCliloc.cs — the §9 Mythic cliloc reader in net48 C#, ported
    from UOFiddler (Beerware) with every file-derived index bounds-checked.
    Phase 2 promotes this into overlay/.
  * patch_client.ps1 — builds the patched client in five tiers. Hashes every
    file it touches in the SOURCE before and after and aborts on a change.
  * an `assetprobe` verb on BridgeRigDriver, so stock and patched can be run
    against one boot rather than two shard processes.

The findings are written up in tools/scaffolding/README.md. The four that
change what phase 1 has to build:

  * FileIndex's UOP constructor ends `MulPath = uopPath`, so artLegacyMUL.uop
    wins outright and art.mul/artidx.mul are never opened on a current
    client. A validator bounding offsets against art.mul is not approximate,
    it is nonsense — the first run refused 34,299 good statics on that
    mistake, and every refusal looked like a real finding.

  * 22,102 WRONG PICTURES on a stock, unmodified client. Empty UOP index
    slots read `lookup 0, length 0`; Seek treats that as a hit, and
    LoadStatic decodes zero bytes into a shared buffer it reuses, only ever
    grows, and fills from a Read whose return value is discarded — so the id
    renders the previously-decoded asset. The mul path does not do this
    (artidx stores -1), which is why the earlier probe counted 32,766 of
    them as "ok". A bulk import that trusted the library would have written
    22,102 duplicate images under ids that have no art.

  * The validator caught all 8 record-level defects — 7 of which the library
    rendered without raising anything, including a verdata lookup past
    verdata.mul's own end (Verdata.Seek is bounds-checked nowhere) and an
    8000x8000 bitmap allocated from two bytes in a file. It refused NOTHING
    across 49,151 statics and 16,384 land tiles on the stock client, which
    is the number that makes the boundary defensible.

  * §4.1's crash reproduces in-process: one Ultima.Gumps.GetGump(2) and the
    ServUO process disappeared — no catch reached, no console line, the
    checkpoint file the only record. "Nothing calls Ultima.Gumps" is now an
    earned safety rule.

§9 is proven: 123,490 entries in 218 ms, byte-identical to UOFiddler's own
output, with no UOFiddler installed and nothing copied to a server.

Not covered, and named as phase 1 work: the animation path has no validator
at all, and the patched wolf decoded something else in silence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4

whitlocktech merged commit 1b7edebd31 into edge

2026-09-10 08:07:56 +00:00

whitlocktech deleted branch feat/asset-bridge-p0

2026-09-10 08:07:57 +00:00

whitlocktech referenced this issue from a commit

2026-09-10 08:07:58 +00:00

Merge pull request 'feat(asset-bridge): phase 0 spike — the decoders, from inside a live shard' (#27) from feat/asset-bridge-p0 into edge

wtclaude referenced this pull request

2026-09-14 22:37:41 +00:00

feat(bridge): protocol 8 — the shard reads its own client files (Asset Bridge cutover, 1 of 5) #36

Sign in to join this conversation.