servuo-plugins

RunicGateway/servuo-plugins

Fork 0

Commit Graph

Author	SHA1	Message	Date
wtclaude	73b07eed22	feat(asset-bridge): the cliloc table, decompressed on the shard (Phase 2) The shard reads its own client's `Cliloc.enu` and serves it over the bridge, so the operator stops installing UOFiddler, building a converter against its `Ultima.dll`, and copying a 5 MB file to the web host every time they patch. `BridgeCliloc.cs` is the one decoder protocol 8 writes rather than calls (docs/link/v8.md §4, §9): a port of UOFiddler's `MythicDecompress` + `MoveToFront` — Beerware, so clean to bring into a GPL-3.0-or-later tree — rewritten against plain arrays, because the upstream is `Span<T>` / `ArrayPool<T>` / `BinaryPrimitives` code and ServUO targets `net48`. The algorithm is deliberately unchanged, including the parts that read oddly. The three-region count/cursor/end table and the symbol-table shifts are upstream's, because a tidier rewrite of somebody else's format decoder is a chance to be subtly wrong in a way that produces plausible text. Two bounds checks were added and they are the only behavioural difference: the upstream indexes its payload unchecked, which is safe for a file the client wrote and is not safe for a file this shard was handed. Measured on a stock client: 4,989,921 bytes read, decompressed and parsed in 290 ms, yielding 67,496 non-blank rows in id order. That number is the acceptance test — it is what UOFiddler's own DLL produced from this same client through the converter this phase deletes, so an independent implementation agrees to the row. Zero U+FFFD; the 696 non-ASCII rows carry correct curly quotes; the longest row is a 12,149-character EULA, which is why the record length is read unsigned. Blanks never reach the wire — ~56,000 of the 123,490 entries are empty strings the client reserves, and the website discards them at import anyway. Also on this plane: * `assets.error` gains a `code`. Phase 1 chose between 403 and 400 by looking for the word "disabled" in an operator-facing sentence, which makes prose load-bearing; `DISABLED` / `NOT_FOUND` / `UNREADABLE` / `UNAVAILABLE` / `BAD_REQUEST` say it directly. * `Accept` and `Fail` are internal rather than private, because the asset plane's single slot and its refusal frame are shared by every family on it. The cursor is a cliloc NUMBER, not an offset: the decoded table is cached for five idle minutes and released after the last page, so it can be dropped and rebuilt between two pages of one import, and an index would then silently mean something else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4	2026-09-10 11:12:43 -05:00
wtclaude	c89e818dbf	feat(asset-bridge): the transport, and the 357 wrong pictures it found Asset Bridge phase 1 (docs/link/v8.md §16). Sidecar half: RunicGateway/link#41. Docs half: RunicGateway/docs#236. The transport for protocol 8, plus phase 0's validator promoted into the overlay and extended to animations — which is where the interesting part is. ## 357 of the 1,144 "decodable" bodies are wrong pictures, on a STOCK client Phase 0 measured the art path and left the animation half unbuilt. It has the same defect, and it is worse: `GetAnimation` decodes through `new MemoryStream(m_StreamBuffer, false)` — the whole shared buffer, not the `length` bytes just read into it — so a truncated or absent record does not even hit end-of-stream. It sails on into the previous animation's bytes. Measured directly, because no count could tell: \| Decode body 320 (`lookup 22638982, length 0`) straight after… \| Comes back \| \|---\|---\| \| body 12, the dragon \| the dragon, 176x167, identical hash \| \| body 34, the wolf \| the wolf's dimensions, 35x34 \| \| body 400, the human male \| the human, 27x63, identical hash \| The catalogue is 787 bodies, not 1,144. Importing the other 357 would have written duplicate creature portraits into the site showing whichever body the walk decoded before them. The record walk refused 0 real bodies on the stock client — the false-refusal measurement §4.5 says the boundary depends on. ## And four of the twelve player bodies, not six §5.2 listed the elf ghosts (607, 608) as decoding. Their index entry is `length 0`; what came back was the elf female at her exact dimensions, because 606 is what the walk decoded immediately before. Confirmed the same way — 607 after the dragon is the dragon. Phase 4's UOP decoder now covers eight ids rather than six. ## What is here - `overlay/Scripts/Custom/Bridge/BridgeAssets.cs` — the plane. Accepts on the Core thread, hands off to a dedicated asset worker, returns immediately. Three rules, all answering a specific failure: - one slot, second request answered `bridge.busy` (425). `Emit`'s queue is bounded in lines, so 10,000 queued 200 KB replies is 2 GB of shard memory; the bound that holds is flow control, on the side where the memory is. - byte budgets (`AssetBatchBytes`, 512 KiB) under the sidecar's new 1 MiB cap. The factor of two is load-bearing: a page always admits its first item, so it may overshoot by one, and the headroom is what makes that land on the wire. - replies, never events — no `reqId`, no answer. An uncorrelated frame is an event by definition, and §3.1 is why none of this may be one. - `PageBuilder` — one paging envelope (`more`/`cursor`/`cut`) for all five families that will page, defined before the first one needs it. `cut` matters: "short page" has three meanings and only `end` means finished. - `assets.sources` — stage 1 of the import gate, its first user. - `BridgeAssetValidator.cs` — promoted from `tools/`, plus `ResolveAnimation` (the never-sweep-file-types rule as code, with no loop and no fallback), `AnimationRecordSane` and the frame walk. - `EXTRACTOR_VERSION`, `overlay.toml` protocol 7 → 8, `AssetsEnabled`. ## Hashing had to come off the request path §6's gate is (size, mtime) first, hash only when those differ. The first call has nothing cached, so that still means hashing 1.06 GB — inside the sidecar's 10 s reply timeout it does not fit. So hashes are computed on their own thread (deliberately not the single-slot worker, which would answer every status poll `bridge.busy` for the whole pass) and the reply carries `hashing`/`complete`. Measured on the real rig: first call instant with `sha256: null`, second call 44 ms with every hash present. ## Verified on the wire, not just compiled Real ServUO 57.4 + the real sidecar + the real client. `GET /assets/sources` → 200, `X-UOLink-Version: 8`, `imaging: {ok: true}`, and §4.6's diagnostic firing on a live client: `artDataFile: artlegacymul.uop`, with `art.mul` and `artidx.mul` both carrying `shadowedBy`. Live events kept flowing through the new capped reader with no warnings. Not exercised live: the disabled-plane 403 and the busy 425 (both unit-tested on the sidecar side; the shard halves are a config read and a lock). - [x] AI-assisted — Claude Code (Opus 5) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4	2026-09-10 08:32:18 -05:00

Author

SHA1

Message

Date

wtclaude

73b07eed22

feat(asset-bridge): the cliloc table, decompressed on the shard (Phase 2)

The shard reads its own client's `Cliloc.enu` and serves it over the bridge, so
the operator stops installing UOFiddler, building a converter against its
`Ultima.dll`, and copying a 5 MB file to the web host every time they patch.

`BridgeCliloc.cs` is the one decoder protocol 8 writes rather than calls
(docs/link/v8.md §4, §9): a port of UOFiddler's `MythicDecompress` +
`MoveToFront` — Beerware, so clean to bring into a GPL-3.0-or-later tree —
rewritten against plain arrays, because the upstream is `Span<T>` /
`ArrayPool<T>` / `BinaryPrimitives` code and ServUO targets `net48`.

The algorithm is deliberately unchanged, including the parts that read oddly.
The three-region count/cursor/end table and the symbol-table shifts are
upstream's, because a tidier rewrite of somebody else's format decoder is a
chance to be subtly wrong in a way that produces plausible text. Two bounds
checks were added and they are the only behavioural difference: the upstream
indexes its payload unchecked, which is safe for a file the client wrote and is
not safe for a file this shard was handed.

Measured on a stock client: 4,989,921 bytes read, decompressed and parsed in
**290 ms**, yielding **67,496** non-blank rows in id order. That number is the
acceptance test — it is what UOFiddler's own DLL produced from this same client
through the converter this phase deletes, so an independent implementation
agrees to the row. Zero U+FFFD; the 696 non-ASCII rows carry correct curly
quotes; the longest row is a 12,149-character EULA, which is why the record
length is read unsigned.

Blanks never reach the wire — ~56,000 of the 123,490 entries are empty strings
the client reserves, and the website discards them at import anyway.

Also on this plane:

  * `assets.error` gains a `code`. Phase 1 chose between 403 and 400 by looking
    for the word "disabled" in an operator-facing sentence, which makes prose
    load-bearing; `DISABLED` / `NOT_FOUND` / `UNREADABLE` / `UNAVAILABLE` /
    `BAD_REQUEST` say it directly.
  * `Accept` and `Fail` are internal rather than private, because the asset
    plane's single slot and its refusal frame are shared by every family on it.

The cursor is a cliloc NUMBER, not an offset: the decoded table is cached for
five idle minutes and released after the last page, so it can be dropped and
rebuilt between two pages of one import, and an index would then silently mean
something else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4

2026-09-10 11:12:43 -05:00

wtclaude

c89e818dbf

feat(asset-bridge): the transport, and the 357 wrong pictures it found

Asset Bridge phase 1 (docs/link/v8.md §16). Sidecar half: RunicGateway/link#41.
Docs half: RunicGateway/docs#236.

The transport for protocol 8, plus phase 0's validator promoted into the overlay
and extended to animations — which is where the interesting part is.

## 357 of the 1,144 "decodable" bodies are wrong pictures, on a STOCK client

Phase 0 measured the art path and left the animation half unbuilt. It has the same
defect, and it is worse: `GetAnimation` decodes through
`new MemoryStream(m_StreamBuffer, false)` — the whole shared buffer, not the
`length` bytes just read into it — so a truncated or absent record does not even hit
end-of-stream. It sails on into the previous animation's bytes.

Measured directly, because no count could tell:

| Decode body 320 (`lookup 22638982, length 0`) straight after… | Comes back |
|---|---|
| body 12, the dragon | the dragon, 176x167, identical hash |
| body 34, the wolf | the wolf's dimensions, 35x34 |
| body 400, the human male | the human, 27x63, identical hash |

The catalogue is **787 bodies, not 1,144**. Importing the other 357 would have written
duplicate creature portraits into the site showing whichever body the walk decoded
before them.

The record walk refused **0** real bodies on the stock client — the false-refusal
measurement §4.5 says the boundary depends on.

## And four of the twelve player bodies, not six

§5.2 listed the elf ghosts (607, 608) as decoding. Their index entry is `length 0`;
what came back was the elf female at her exact dimensions, because 606 is what the
walk decoded immediately before. Confirmed the same way — 607 after the dragon is
the dragon. Phase 4's UOP decoder now covers eight ids rather than six.

## What is here

- **`overlay/Scripts/Custom/Bridge/BridgeAssets.cs`** — the plane. Accepts on the Core
thread, hands off to a dedicated asset worker, returns immediately. Three rules, all
answering a specific failure:
- **one slot**, second request answered `bridge.busy` (425). `Emit`'s queue is bounded
in *lines*, so 10,000 queued 200 KB replies is 2 GB of shard memory; the bound that
holds is flow control, on the side where the memory is.
- **byte budgets** (`AssetBatchBytes`, 512 KiB) under the sidecar's new 1 MiB cap. The
factor of two is load-bearing: a page always admits its first item, so it may
overshoot by one, and the headroom is what makes that land on the wire.
- **replies, never events** — no `reqId`, no answer. An uncorrelated frame is an event
by definition, and §3.1 is why none of this may be one.
- **`PageBuilder`** — one paging envelope (`more`/`cursor`/`cut`) for all five families
that will page, defined before the first one needs it. `cut` matters: "short page" has
three meanings and only `end` means finished.
- **`assets.sources`** — stage 1 of the import gate, its first user.
- **`BridgeAssetValidator.cs`** — promoted from `tools/`, plus `ResolveAnimation` (the
never-sweep-file-types rule as code, with no loop and no fallback),
`AnimationRecordSane` and the frame walk.
- **`EXTRACTOR_VERSION`**, **`overlay.toml` protocol 7 → 8**, `AssetsEnabled`.

## Hashing had to come off the request path

§6's gate is (size, mtime) first, hash only when those differ. The first call has nothing
cached, so that still means hashing 1.06 GB — inside the sidecar's 10 s reply timeout it
does not fit. So hashes are computed on their own thread (deliberately not the single-slot
worker, which would answer every status poll `bridge.busy` for the whole pass) and the
reply carries `hashing`/`complete`.

Measured on the real rig: first call instant with `sha256: null`, second call **44 ms**
with every hash present.

## Verified on the wire, not just compiled

Real ServUO 57.4 + the real sidecar + the real client. `GET /assets/sources` → 200,
`X-UOLink-Version: 8`, `imaging: {ok: true}`, and §4.6's diagnostic firing on a live
client: `artDataFile: artlegacymul.uop`, with `art.mul` and `artidx.mul` both carrying
`shadowedBy`. Live events kept flowing through the new capped reader with no warnings.

Not exercised live: the disabled-plane 403 and the busy 425 (both unit-tested on the
sidecar side; the shard halves are a config read and a lock).

- [x] AI-assisted — Claude Code (Opus 5)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4

2026-09-10 08:32:18 -05:00

2 Commits