452be696df2f049e50fd3665b72e976d2ec72008
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| efbd45685c |
feat(asset-bridge): the UOP animation reader, and 235 bodies the legacy path cannot see (Phase 4)
ServUO's vendored `Ultima.Animations` reads legacy `anim*.mul` only -- it builds its
five FileIndexes with the constructor that passes `uopFile: null` -- so everything a
modern client moved into `AnimationFrame*.uop` is invisible to it. This adds the one
reader docs/link/v8.md 4.3 reserved for phase 4, and wires it in as a fallback beneath
the legacy path.
What it actually recovers is not what the plan expected, and the difference was
measured before any of this was written:
- Of the EIGHT player-character bodies 4.8 assigned to this phase, only TWO are in
the client at all: gargoyles 666 and 667, in AnimationFrame3.uop. The six ghost
bodies (human 402/403, elf 607/608, gargoyle 694/695) are in no package. The five
packages hold 10,724 entries between them and the
`build/animationlegacyframe/%06d/%02d.bin` name scheme claims every one, so there
is no other naming they could be hiding under.
- The same fallback reaches 233 further bodies the catalogue had nothing for, so the
working set goes from 787 to 1,022 (57 Monster, 26 Animal, 97 Equipment, 50
unlisted, 3 Human, 2 Sea). The catalogue was already 366 Equipment bodies before
this, so its character does not change.
Decided with the org lead before building: the fallback applies to every body rather
than to player bodies alone; ghost ids leave the player-body set entirely (no client
has art for any of them, and listing them only advertised keys that cannot exist); the
UOP path gets its own PNG encoder rather than Bitmap.Save; and a host without
libgdiplus keeps the flat NO_IMAGING refusal rather than serving a partial catalogue.
Three things about the reader:
- It is not the never-sweep rule being broken. That rule exists because a legacy
index is addressed by POSITION, so asking the wrong file returns a giant spider
that decodes cleanly. A UOP entry is addressed by the hash of a name containing the
body id, and the payload declares that id again for `Group.TryOpen` to check, so a
hit is proof of identity. Measured: no hash appears in two packages.
- Validate-as-we-go replaces validate-before-calling, because here we ARE the
library: the block chain is bounded against the file, the record against the file,
the inflated length against the declared one, the frame table against the payload,
and every run header against both the record's remaining bytes and the bitmap it
writes into. Measured the way 4.5 was -- across every UOP body on a stock client it
refuses nothing that carries art. The one body it refuses (286) declares a 0x0
frame, which the vendored decoder treats as absent too.
- No System.Drawing anywhere in it, which is what 4.4 promised: the decode fills a
ushort[] of ARGB1555 and BridgePng encodes that directly (zlib around net48's
raw-deflate-only DeflateStream, CRC32, one IDAT, filter 0).
EXTRACTOR_VERSION 1 -> 2: every client file is byte-identical and the answer is
different, which is exactly what that number exists to say. The UOP packages join
`assets.sources` and the catalogue id, so patching one is drift; `Ultima.Files` cannot
resolve them (its table predates UOP animations) so BridgeUop.FindClientFile does it,
case-insensitively by enumeration for Linux hosts. Manifest and fetch rows carry a new
`source` field (`legacy` / `uop`).
Protocol stays 8 -- no message shape changed, only fields added.
Measured on the live rig (real sidecar, real ServUO, this machine's client):
1,022 rows in ONE page, 1,409 ms cold; six player bodies, all six with art for the
first time (400/401/605/606 legacy, 666/667 uop), all at direction 0; 1,016 at
direction 1; the six ghost bodies absent; 45 duplicate-hash groups of which exactly one
is new, bodies 1531/1532, two distinct records whose first frames match -- legitimate,
and provable only because each payload declares its own body id. The gargoyles were
rendered and looked at, because 4.3's whole point is that this failure mode produces
confident, wrong pictures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
|
|||
| 64c0ec00b1 |
feat(asset-bridge): the body catalogue and slug to body id (Phase 3)
Two request families, and they run on opposite threads on purpose.
`assets.bodies` (BridgeBodies) answers the question only code inside ServUO
can: the atlas knows a creature by the class name in Spawns/*.xml, the client
knows it by a body id, and nothing in the tree declares the mapping. Construct
the type, read Body.BodyID, Delete(). That is world mutation, so it answers on
the CORE thread and is the one family here that does not take the asset
worker's slot -- and the batch is capped at 100 names, REFUSED rather than
truncated, because a truncated answer is indistinguishable from a complete one
from the website's side.
`assets.manifest` / `assets.fetch` (BridgeCatalog) are the catalogue, on the
worker. The manifest carries { key, sha256, bytes, width, height } and no
pixels, so an Update fetches only what moved; the fetch carries base64 PNG.
The scan keeps the bytes it hashed rather than decoding all 787 sprites twice.
Three things worth stating about the shapes:
- It pages on the WALL CLOCK as well as on bytes. The rows are ~90 bytes and
the whole catalogue is one page by the byte budget, but building it means
decoding hundreds of sprites against a 10 s reply timeout.
- `catalog` is derived from the client files (sizes, mtimes, both direction
settings, EXTRACTOR_VERSION), not minted per build -- the cache is released
when idle, and a fresh id per build would force a restart mid-import although
nothing about the client moved.
- ARGB1555 is expanded to 32bpp here rather than handed to GDI+, because what
it does with a one-bit alpha channel varies by platform and a black rectangle
behind every sprite would pass any test that only checked the bytes decoded.
Nothing trusts the library's success. Every body goes through CheckEntry and
AnimationRecordSane before it is decoded, which is what keeps the 357 bodies
whose index entry reads `length 0` -- and which the decoder hands back the
PREVIOUS creature's bitmap for -- out of the catalogue.
Walked on a live shard: 787 rows in one 734 ms page; bodies 320, 607, 666 all
absent rather than wrong; 783 at direction 1 and 4 at direction 0; all 455 stock
creature classes resolved at ~190 ms per 100 with zero mobiles leaked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
|
|||
| 73b07eed22 |
feat(asset-bridge): the cliloc table, decompressed on the shard (Phase 2)
The shard reads its own client's `Cliloc.enu` and serves it over the bridge, so
the operator stops installing UOFiddler, building a converter against its
`Ultima.dll`, and copying a 5 MB file to the web host every time they patch.
`BridgeCliloc.cs` is the one decoder protocol 8 writes rather than calls
(docs/link/v8.md §4, §9): a port of UOFiddler's `MythicDecompress` +
`MoveToFront` — Beerware, so clean to bring into a GPL-3.0-or-later tree —
rewritten against plain arrays, because the upstream is `Span<T>` /
`ArrayPool<T>` / `BinaryPrimitives` code and ServUO targets `net48`.
The algorithm is deliberately unchanged, including the parts that read oddly.
The three-region count/cursor/end table and the symbol-table shifts are
upstream's, because a tidier rewrite of somebody else's format decoder is a
chance to be subtly wrong in a way that produces plausible text. Two bounds
checks were added and they are the only behavioural difference: the upstream
indexes its payload unchecked, which is safe for a file the client wrote and is
not safe for a file this shard was handed.
Measured on a stock client: 4,989,921 bytes read, decompressed and parsed in
**290 ms**, yielding **67,496** non-blank rows in id order. That number is the
acceptance test — it is what UOFiddler's own DLL produced from this same client
through the converter this phase deletes, so an independent implementation
agrees to the row. Zero U+FFFD; the 696 non-ASCII rows carry correct curly
quotes; the longest row is a 12,149-character EULA, which is why the record
length is read unsigned.
Blanks never reach the wire — ~56,000 of the 123,490 entries are empty strings
the client reserves, and the website discards them at import anyway.
Also on this plane:
* `assets.error` gains a `code`. Phase 1 chose between 403 and 400 by looking
for the word "disabled" in an operator-facing sentence, which makes prose
load-bearing; `DISABLED` / `NOT_FOUND` / `UNREADABLE` / `UNAVAILABLE` /
`BAD_REQUEST` say it directly.
* `Accept` and `Fail` are internal rather than private, because the asset
plane's single slot and its refusal frame are shared by every family on it.
The cursor is a cliloc NUMBER, not an offset: the decoded table is cached for
five idle minutes and released after the last page, so it can be dropped and
rebuilt between two pages of one import, and an index would then silently mean
something else.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
|
|||
| c89e818dbf |
feat(asset-bridge): the transport, and the 357 wrong pictures it found
Asset Bridge phase 1 (docs/link/v8.md §16). Sidecar half: RunicGateway/link#41. Docs half: RunicGateway/docs#236. The transport for protocol 8, plus phase 0's validator promoted into the overlay and extended to animations — which is where the interesting part is. ## 357 of the 1,144 "decodable" bodies are wrong pictures, on a STOCK client Phase 0 measured the art path and left the animation half unbuilt. It has the same defect, and it is worse: `GetAnimation` decodes through `new MemoryStream(m_StreamBuffer, false)` — the whole shared buffer, not the `length` bytes just read into it — so a truncated or absent record does not even hit end-of-stream. It sails on into the previous animation's bytes. Measured directly, because no count could tell: | Decode body 320 (`lookup 22638982, length 0`) straight after… | Comes back | |---|---| | body 12, the dragon | the dragon, 176x167, identical hash | | body 34, the wolf | the wolf's dimensions, 35x34 | | body 400, the human male | the human, 27x63, identical hash | The catalogue is **787 bodies, not 1,144**. Importing the other 357 would have written duplicate creature portraits into the site showing whichever body the walk decoded before them. The record walk refused **0** real bodies on the stock client — the false-refusal measurement §4.5 says the boundary depends on. ## And four of the twelve player bodies, not six §5.2 listed the elf ghosts (607, 608) as decoding. Their index entry is `length 0`; what came back was the elf female at her exact dimensions, because 606 is what the walk decoded immediately before. Confirmed the same way — 607 after the dragon is the dragon. Phase 4's UOP decoder now covers eight ids rather than six. ## What is here - **`overlay/Scripts/Custom/Bridge/BridgeAssets.cs`** — the plane. Accepts on the Core thread, hands off to a dedicated asset worker, returns immediately. Three rules, all answering a specific failure: - **one slot**, second request answered `bridge.busy` (425). `Emit`'s queue is bounded in *lines*, so 10,000 queued 200 KB replies is 2 GB of shard memory; the bound that holds is flow control, on the side where the memory is. - **byte budgets** (`AssetBatchBytes`, 512 KiB) under the sidecar's new 1 MiB cap. The factor of two is load-bearing: a page always admits its first item, so it may overshoot by one, and the headroom is what makes that land on the wire. - **replies, never events** — no `reqId`, no answer. An uncorrelated frame is an event by definition, and §3.1 is why none of this may be one. - **`PageBuilder`** — one paging envelope (`more`/`cursor`/`cut`) for all five families that will page, defined before the first one needs it. `cut` matters: "short page" has three meanings and only `end` means finished. - **`assets.sources`** — stage 1 of the import gate, its first user. - **`BridgeAssetValidator.cs`** — promoted from `tools/`, plus `ResolveAnimation` (the never-sweep-file-types rule as code, with no loop and no fallback), `AnimationRecordSane` and the frame walk. - **`EXTRACTOR_VERSION`**, **`overlay.toml` protocol 7 → 8**, `AssetsEnabled`. ## Hashing had to come off the request path §6's gate is (size, mtime) first, hash only when those differ. The first call has nothing cached, so that still means hashing 1.06 GB — inside the sidecar's 10 s reply timeout it does not fit. So hashes are computed on their own thread (deliberately not the single-slot worker, which would answer every status poll `bridge.busy` for the whole pass) and the reply carries `hashing`/`complete`. Measured on the real rig: first call instant with `sha256: null`, second call **44 ms** with every hash present. ## Verified on the wire, not just compiled Real ServUO 57.4 + the real sidecar + the real client. `GET /assets/sources` → 200, `X-UOLink-Version: 8`, `imaging: {ok: true}`, and §4.6's diagnostic firing on a live client: `artDataFile: artlegacymul.uop`, with `art.mul` and `artidx.mul` both carrying `shadowedBy`. Live events kept flowing through the new capped reader with no warnings. Not exercised live: the disabled-plane 403 and the busy 425 (both unit-tested on the sidecar side; the shard halves are a config read and a lock). - [x] AI-assisted — Claude Code (Opus 5) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4 |
