Phase 3 is built and walked on a live shard. What the walk measured, and the
two places the design of record needed correcting:
§8.1, new: the catalogue is 787 exactly as §4.8 predicted, and the whole scan
of bodies 1-2047 takes 734 ms cold -- so the wall-clock paging §11 designed
never fires on this client. Every §4.8/§5.2 prediction held when the bytes were
rendered and LOOKED at: 320, 607, 608 and 666 come back absent rather than as
another creature's picture, and the direction split is 783 at index 1 against 4
at index 0 -- four player bodies, not six.
44 of the 787 hashes are shared by two or three bodies, which is the exact
signature of the wrong-picture bug, so it was chased rather than assumed. It is
the client's own Body.def aliasing (83 {1}, 84 {1}, 106 {12, 59}), and the check
that settles it is at the source: Translate(ref body, ref hue) rewrites `body`
only when bit 31 is set, unlike the one-argument overload -- and ResolveAnimation
calls that same two-argument overload, so validator and decoder resolve the
identical record.
§12.1, new: **§12 is right about the outcome and wrong about the mechanism.**
`shard_spawn_creatures` is emptied and refilled by every atlas refresh, and a
refresh runs on every boot -- so an imported filename written to that row is
destroyed by an ordinary re-parse of the ServUO tree, and the next Update finds
the client files unchanged and never restores it. Three tables outside that
blast radius, and the atlas import re-derives `art` on the way past.
§14: **§16 listed phase 3 as servuo-plugins + module-uo and that was wrong.**
web.rs routes every command explicitly, so `link` is in the phase. Corrected in
both places.
UOFIDDLER.md is DELETED, two phases earlier than §9.1 predicted -- creature art
was the only thing still on it. SPAWN_ATLAS.md §Artwork is rewritten around the
import, keeping the operator's own map as the thing that wins; the module's
SCHEMA.md gains the three tables and API.md the two admin routes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
1151 lines
70 KiB
Markdown
1151 lines
70 KiB
Markdown
# Asset Bridge (Protocol 8) — client assets without UOFiddler
|
||
|
||
**Call it the Asset Bridge.** That is the name for this work everywhere — commits, PR titles,
|
||
branches (`feat/asset-bridge-p<n>`), and conversation. The protocol number is **8** and this file is
|
||
`docs/link/v8.md`.
|
||
|
||
**Status:** Design of record, **phase 0 complete**. Approved in principle 2026-09-09
|
||
(architecture, asset scope, built-in cliloc decoder, atlas cleanup); refined 2026-09-10 (one
|
||
direction not five, ServUO's own decoders, the UOP reader for player bodies, the libgdiplus
|
||
requirement). The phase 0 spike ran on 2026-09-10 and its findings are §4.5-§4.7 -- §4's decision
|
||
stands, and the response to a malformed record is now measured rather than proposed. **Nothing in
|
||
§17 is open.**
|
||
**Supersedes the manual half of:** `../website/UOFIDDLER.md` (deleted in phase 3, once creature
|
||
art stopped needing it), [`../website/CLILOCS.md`](../website/CLILOCS.md) §Converting,
|
||
[`../website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md) §Artwork and §Configuring the tree.
|
||
|
||
Two features on this platform read data that only exists inside a UO client, and today both reach
|
||
the site by hand: the operator installs UOFiddler, converts `Cliloc.enu` on their own desktop,
|
||
exports sprites one at a time from a GUI, hand-writes a slug→filename JSON map, and copies the
|
||
result to the server. A third — the spawn atlas — avoids UOFiddler but pays a different price: the
|
||
**website** must be able to read the shard's ServUO tree directly, over a bind mount or a shared
|
||
volume.
|
||
|
||
This protocol deletes all three arrangements. The shard already has everything, and the bridge
|
||
already goes to the website.
|
||
|
||
---
|
||
|
||
## 1. The premise, which turns out to be free
|
||
|
||
**A ServUO shard cannot boot without a UO client installation.** It reads maps, statics, tiledata
|
||
and multis out of `.mul`/`.uop` files, and `Config/DataPath.cfg` is where an operator declares
|
||
where those live — *required* on Linux, auto-detected from the registry on Windows. At runtime the
|
||
resolved directories sit in `Server.Core.DataDirectories`, a public static the plugin can read on
|
||
any shard, with no new configuration and nothing for an operator to set up.
|
||
|
||
So the files the operator has been converting on their desktop are already on the shard host, in a
|
||
directory the shard already knows the path of, in a process the bridge already runs inside.
|
||
|
||
Everything below follows from that.
|
||
|
||
### 1.1 What was measured, not assumed
|
||
|
||
Against this machine's ServUO 57.4 tree (`C:\Users\colby\Desktop\ServUO`) and client
|
||
(`D:\Games\Electronic Arts\Ultima Online Classic`, 3.5 GB), loading **ServUO's own
|
||
`Ultima.dll`** — the assembly `overlay/Scripts/Scripts.csproj:39` already carries a
|
||
`<ProjectReference>` to:
|
||
|
||
| Call | Result |
|
||
|---|---|
|
||
| `Art.GetStatic(0…16383)` | 16,384 decoded, 0 errors |
|
||
| `Art.GetStatic(16384…65535)` | 32,766 decoded, 1 empty, 16,385 clean out-of-range errors |
|
||
| `Art.GetLand(0…16383)` | 16,384 decoded, 0 errors |
|
||
| `Animations.GetAnimation(0…2047, 0, 1)` | **1,144** bodies with a decodable first frame, 904 empty, 0 errors — of which **357 are wrong pictures**; see §4.8 |
|
||
| `Hues.GetHue(33)` | loads |
|
||
| `Bitmap.Save(…, Png)` | 852-byte PNG from one creature frame |
|
||
| `Gumps.GetGump(2)` | **hard crash** — `AccessViolationException`, process exit `0xC0000005` |
|
||
| `new StringList("enu", "Cliloc.enu")` | throws — `Non-negative number required` |
|
||
|
||
Two of those rows are load-bearing and are dealt with in §4 and §9. The rest say the same thing:
|
||
**most of the extraction this protocol needs is already implemented, already compiled, and already
|
||
referenced by the plugin's own build.**
|
||
|
||
**Every "decoded" count in that table is an upper bound, not a measurement**, and phases 0 and 1
|
||
spent themselves establishing by how much. `Ultima` reports success for records that do not exist
|
||
(§4.5, §4.8), so the honest reading of the animation row is 787 bodies with art, 357 that return
|
||
the previously-decoded body's bitmap, and 904 that return nothing. The static and land rows are
|
||
overstated the same way. This is not a table to size anything from any more — §11 is.
|
||
|
||
Depth, for §11's sizing: body 400 (human male) has **35 actions × 5 directions = 1,050 frames**.
|
||
One body. §5.1 cuts that by exactly 5×.
|
||
|
||
---
|
||
|
||
## 2. Architecture: the shard extracts, the sidecar forwards, the website decides
|
||
|
||
```
|
||
UO client files (operator's own, on the shard host)
|
||
│ read by the plugin, off the Core thread
|
||
▼
|
||
ServUO shard (servuo-plugins/) ← decodes; resolves body ids; hashes
|
||
│ loopback JSON, request/reply, one batch outstanding at a time
|
||
▼
|
||
uo-link sidecar (link/) ← forwards bytes; decides nothing
|
||
│ REST, bearer-token auth, X-UOLink-Version: 8
|
||
▼
|
||
website (module-uo/) ← stores, names, gates, serves
|
||
```
|
||
|
||
This is deliberately the *only* arrangement that keeps
|
||
[the bridge's standing rules](PLAN.md) intact:
|
||
|
||
- **The sidecar stays a dumb forwarder.** It moves opaque assets and decides nothing about them —
|
||
no audience, no projection, no capability advertisement. Putting the decoders in Rust would have
|
||
meant the sidecar deciding what an asset *is*, on top of re-deriving in Rust what is already
|
||
compiled next door in C#.
|
||
- **Access control stays on the website**, which has the auth machinery and the admin forms.
|
||
- **The shard is still never network-reachable.** Nothing here opens a port; the plugin answers
|
||
requests on the connection it already dialled out on.
|
||
|
||
### 2.1 Why not the sidecar, and why not the operator's desktop
|
||
|
||
A Rust extractor in the sidecar would need ports of: the Mythic cliloc decompressor, `FileIndex`
|
||
(including UOP), the ARGB1555 run-length frame decoder, `Body.def`/`Bodyconv.def` translation,
|
||
`Hues.mul`, and a PNG encoder — weeks of work to re-derive what §1.1 shows already runs. It also
|
||
cannot do §8: resolving a creature slug to a body id requires being inside ServUO.
|
||
|
||
Automating on the operator's desktop (shipping the converter with the installer) removes UOFiddler
|
||
but keeps a manual step and still cannot do §8. It was considered and rejected.
|
||
|
||
---
|
||
|
||
## 3. The transport, and the three traps in it
|
||
|
||
### 3.1 Assets go over the request/reply path, never the event path
|
||
|
||
`link/sidecar/src/app.rs:122` persists **every** non-`pong` event into the SQLite store *and*
|
||
broadcasts it to every WebSocket subscriber. An asset stream on that path would grow the sidecar's
|
||
store without bound and fan megabytes out to every connected client, forever.
|
||
|
||
`rpc.rs`'s `try_route` consumes a correlated reply and `continue`s **before** either of those
|
||
happens. So an asset batch is a reply, not an event. This is not a new mechanism — it is the one
|
||
`char.request`, `account.roster` and `vendor.snapshot` already use.
|
||
|
||
### 3.2 One batch outstanding, always
|
||
|
||
`BridgeLink.Emit()` enqueues onto a **bounded drop-oldest** queue (`Bridge.QueueCap`, default
|
||
10,000). It counts **lines, not bytes** — a design that is correct for live events and dangerous
|
||
for bulk transfer, because 10,000 queued 200 KB replies is 2 GB of shard memory.
|
||
|
||
The rule that makes this safe is flow control, not a bigger queue: **the website requests batch
|
||
*n+1* only after batch *n* has arrived.** Queue depth stays at approximately one. A dropped or
|
||
lost reply simply times out and the batch is re-requested, which is safe because reading a client
|
||
file is idempotent and has no world side effects.
|
||
|
||
**Phase 1 made that a rule the shard enforces rather than one the website is trusted to follow.**
|
||
The asset plane has a **single slot**: a request arriving while one is in flight is answered
|
||
`bridge.busy` — which the sidecar already maps to 425 — and runs nothing. The bound belongs on the
|
||
side where the memory actually is; a documented convention would have held right up until the
|
||
first website bug, and its failure mode is the 2 GB above.
|
||
|
||
The alternative considered was serialising in the sidecar, so a second caller waits instead of
|
||
being refused. It was rejected because a waiter spends the website's own 12 s timeout doing
|
||
nothing, and because it leaves the shard itself unguarded against anything that is not that one
|
||
sidecar.
|
||
|
||
What this costs, and it is deliberate: **a status poll shares the slot with a batch**, so polling
|
||
during a long import is answered 425 until the batch lands. That is honest — this plane really
|
||
does do one thing at a time — and it is why the one genuinely long operation on it, hashing 343 MB
|
||
of client files, is explicitly *not* a job on this worker (§6).
|
||
|
||
### 3.3 The size ceilings are already fixed, and one of them is missing
|
||
|
||
| Limit | Value | Where |
|
||
|---|---|---|
|
||
| Sidecar waits for a shard reply | **10 s** | `rpc.rs` `REPLY_TIMEOUT` |
|
||
| Website waits for the sidecar | **12 s** | `module-uo/server/utils/uoLinkClient.js` `TIMEOUT_MS` |
|
||
| Sidecar → shard line | 1 MiB | `BridgeLink.cs:283` |
|
||
| **Shard → sidecar line** | **1 MiB** | `shard.rs` `MAX_INBOUND_LINE_BYTES` — **added in phase 1**; it was unbounded |
|
||
| **Batch budget** | **512 KiB encoded** | `Bridge.AssetBatchBytes` — **added in phase 1** |
|
||
|
||
The first two bound a batch: it must decode, encode, serialise and cross the wire inside ten
|
||
seconds. The last is a gap this protocol must close — an unbounded `read_line` facing a component
|
||
that is now deliberately sending large lines is a memory-exhaustion shape we would be inventing
|
||
ourselves. **Protocol 8 adds an explicit inbound line cap to the sidecar**, set above the largest
|
||
legal batch and rejecting rather than buffering past it.
|
||
|
||
Batches are therefore sized by bytes, not by count, with the emitter cutting a batch short when it
|
||
would exceed the cap. Base64 costs 33%; the budget must be stated in encoded bytes.
|
||
|
||
**The two numbers, settled in phase 1: a 512 KiB batch budget under a 1 MiB line cap.** The cap is
|
||
symmetric with the one the shard has always applied to its own inbound lines, so both directions
|
||
of this link read the same. The factor of two between them is load-bearing rather than cautious: a
|
||
page always admits its **first** item even when that item alone exceeds the budget, because the
|
||
alternative is an oversized item being skipped for the budget on every page forever and its family
|
||
never making progress. The headroom is what makes that overshoot land on the wire instead of being
|
||
rejected by the cap.
|
||
|
||
An over-long line is **discarded and the connection kept**, which is what `BridgeLink.cs` has
|
||
always done in the other direction. Tearing the link down would take the live event feed with it
|
||
over a single malformed frame, and the reply that was lost simply times out and is re-requested —
|
||
safe, because reading a client file is idempotent.
|
||
|
||
### 3.4 One paging envelope, defined once
|
||
|
||
Five of the families in §14 page: clilocs (phase 2), the body catalogue (3), statics and land (5),
|
||
deep animation keys (6) and the ServUO tree files (7). Left to themselves that is five chances to
|
||
invent five slightly different shapes, and the website would have to learn each one — so phase 1
|
||
defines the envelope before the first family needs it, and `assets.sources` is its first user even
|
||
though it has nothing to page.
|
||
|
||
```
|
||
"files": [ … ], // the array, named by the family
|
||
"more": true, // ask again, with this cursor
|
||
"cursor": "s:4104", // opaque to everyone but the shard; absent when more is false
|
||
"cut": "budget" // budget | end | limit — WHY the page stopped
|
||
```
|
||
|
||
`cut` is the field that is easy to leave out and expensive not to have. "This page is short" has
|
||
three different meanings — the source ran out (`end`), the byte budget was spent (`budget`), or
|
||
the family stopped at a limit of its own (`limit`) — and **only the first means the import is
|
||
finished.** A website that had to infer completion from an item count would resume from the wrong
|
||
place the first time a page happened to land exactly on a boundary.
|
||
|
||
The cursor is deliberately **opaque and shard-defined**. The shard is the only side that knows how
|
||
its own walk is ordered, and a cursor the website could parse is a cursor the website would
|
||
eventually construct.
|
||
|
||
The budget is counted in **UTF-8 bytes**, not characters and not items. Characters would undercount
|
||
every non-ASCII byte in a cliloc row, and the ceiling this has to live inside — §3.3's line cap —
|
||
is measured in bytes.
|
||
|
||
---
|
||
|
||
## 4. The decoders are ServUO's own — decided, and the crash is narrower than it looked
|
||
|
||
**We call ServUO's vendored `Ultima` (decided 2026-09-10).** No decoders are reimplemented.
|
||
`overlay/Scripts/Scripts.csproj:39` already references the project, so the art half of this protocol
|
||
costs plumbing rather than pixel code, and only §9's cliloc decompressor is written from scratch.
|
||
|
||
The reason that is safe, rather than merely cheap, is a distinction §1.1 did not draw at first.
|
||
|
||
### 4.1 The crash lives on one code path, and nothing we call uses it
|
||
|
||
`Gumps.GetGump(2)` does not fail — it **corrupts the process**: `AccessViolationException`, exit
|
||
`0xC0000005`. That is a corrupted-state exception, uncatchable by an ordinary `try/catch` on .NET
|
||
Framework 4.8, so in-process on a live shard it is a crash with players on it. That much is
|
||
alarming, and on its own it looked like an argument against using this library at all.
|
||
|
||
It is not, because of how the three decoders construct their `FileIndex`:
|
||
|
||
| Decoder | UOP file | `hasExtra` | Probed |
|
||
|---|---|---|---|
|
||
| `Art` | `artLegacyMUL.uop` | **false** | 49,150 statics + 16,384 land tiles, **0 faults** |
|
||
| `Animations` | *none — legacy `anim*.mul` only* | — | 1,144 bodies, **0 faults** |
|
||
| `Gumps` | `gumpartLegacyMUL.uop` | **true** | **faults on the second id** |
|
||
|
||
`FileIndex.cs`'s own comment says the extra-field handling exists *for* `gumpartlegacy.uop` — it is
|
||
the one UOP layout carrying an extra field, and `hasExtra: true` is the branch written to cope with
|
||
it. **Gumps is the only caller that sets it.** So the fault is not a general fragility in this
|
||
library's `unsafe` code; it is a bug on a branch that exactly one decoder reaches, and that decoder
|
||
is already out of scope (§11).
|
||
|
||
The rule this turns into is a safety rule, not a preference: **nothing in this protocol calls
|
||
`Ultima.Gumps`.** Adding gump art later means fixing or replacing that path first, deliberately,
|
||
not discovering it in production.
|
||
|
||
### 4.2 What the decision accepts
|
||
|
||
Three costs come with it, all known and none of them blocking:
|
||
|
||
1. **Six of the twelve stock player-character bodies have no art on this path** — both human ghosts
|
||
and every gargoyle body (§5.2). `Animations` never reads `AnimationFrame*.uop`. **This one is
|
||
not merely accepted: §4.3 adds a decoder for it**, because it is the player character and the
|
||
scope says player models.
|
||
2. **`System.Drawing` is a hard dependency, in the decode and not just the encode.** `Frame`
|
||
writes ARGB1555 straight through a `LockBits` pointer, so a Linux shard needs **libgdiplus** to
|
||
read a sprite at all. That is a stated prerequisite — §4.4.
|
||
3. **We inherit whatever `Ultima` a given ServUO vendors**, which can change under a shard upgrade.
|
||
`EXTRACTOR_VERSION` (§7) is the mitigation: it already counts as drift, so a shard whose library
|
||
changed re-derives on the next import.
|
||
|
||
The residual risk that remains is a patched or custom client tripping an out-of-bounds read on a
|
||
path we *do* call. §16's phase 0 is where that gets exercised rather than assumed.
|
||
|
||
### 4.3 One decoder we do write: UOP animation, for the player bodies
|
||
|
||
The six missing bodies are the player character, and the scope is player models, so they get a
|
||
decoder rather than a caveat. It is deliberately the **narrowest possible** addition: a reader for
|
||
`AnimationFrame*.uop`, used *only* for bodies the legacy path cannot resolve. Everything the
|
||
vendored code can already decode keeps going through the vendored code.
|
||
|
||
This client ships `AnimationFrame1/2/3/4/6.uop` (107, 118, 253, 115 and 24 MB) plus
|
||
`AnimationSequence.uop`. ServUO's `FileIndex` already contains a UOP reader — but `Animations`
|
||
never constructs one, and the UOP animation *payload* is its own format rather than a repackaged
|
||
mul record, so wiring the existing `FileIndex` in is not sufficient. Two license-compatible
|
||
references exist to work from: ClassicUO's animation loader (GPL-3, and we are GPL-3-or-later) and
|
||
UOFiddler 4.22 (Beerware, already established in §9).
|
||
|
||
**The trap this must not fall into, and it is why the phase exists at all.** `Bodyconv.def` maps
|
||
gargoyle 666 to `anim5`, and `BodyConverter.Convert` faithfully returns fileType 5 — where the
|
||
client has nothing. Asking the *other* anim files for index 666 does not fail. It returns 175
|
||
decodable action/direction combinations of **a giant spider**, because something unrelated occupies
|
||
that index in `anim2.mul`; fileTypes 3 and 4 return misaligned colour fragments. Rendered and
|
||
confirmed.
|
||
|
||
So the extractor takes `BodyConverter.Convert`'s answer and, if that yields nothing, reports
|
||
nothing. **It must never sweep file types looking for a hit** — that does not find missing art, it
|
||
silently puts a spider on the gargoyle page, and no error is raised anywhere. A "0 rows" outcome is
|
||
correct behaviour; a plausible wrong picture is the failure this protocol most needs to avoid,
|
||
because nothing downstream can detect it.
|
||
|
||
### 4.4 Requirement: a Linux shard host needs libgdiplus
|
||
|
||
**Stated prerequisite, not a soft recommendation.** ServUO targets `net48`, so on Linux it runs under
|
||
Mono, and Mono's `System.Drawing` is a thin layer over **libgdiplus**. §4.2 put `System.Drawing` in
|
||
the decode path, so without that library a Linux shard cannot extract art at all — the cliloc table
|
||
(§9) and the atlas files (§10) are unaffected, since neither touches pixels.
|
||
|
||
**Windows shard hosts need nothing.** `System.Drawing` ships with .NET Framework. This is a
|
||
Linux-only prerequisite and most shards will never read this section.
|
||
|
||
| Host | Get it with |
|
||
|---|---|
|
||
| Debian / Ubuntu | `sudo apt-get install libgdiplus` — in Debian since bullseye (6.0.4) and bookworm/trixie (6.1), and in Ubuntu universe |
|
||
| Fedora / RHEL | `sudo dnf install libgdiplus` (EPEL or the Mono repository) |
|
||
| Docker | `RUN apt-get update && apt-get install -y libgdiplus` in the shard image |
|
||
| Alpine, or a distro with no package | Build from source — see the repository below. This is the awkward case and is worth avoiding by choosing a Debian-based image |
|
||
|
||
Upstream is <https://github.com/mono/libgdiplus>, with the Mono project's own page at
|
||
<https://www.mono-project.com/docs/gui/libgdiplus/>.
|
||
|
||
**One thing to know before depending on it: that repository was archived in March 2025 and is
|
||
read-only.** Distributions still package and patch it, so `apt-get install libgdiplus` is a normal,
|
||
supported thing to do today — but upstream is not maintained. It is the strongest long-term argument
|
||
for eventually moving extraction off `System.Drawing`, and phase 4's UOP reader (§4.3) is written
|
||
without it precisely so that door stays open. It does not change the decision now.
|
||
|
||
**How its absence must present.** Never a stack trace and never a 500. Missing libgdiplus is a
|
||
named, actionable outcome in the same family as the cliloc reader's `COMPRESSED`:
|
||
|
||
```
|
||
status: unavailable
|
||
code: NO_IMAGING
|
||
reason: This shard host cannot render images — Mono's System.Drawing needs libgdiplus.
|
||
Install it (apt-get install libgdiplus) and re-run the import. Cliloc and
|
||
atlas import are unaffected.
|
||
```
|
||
|
||
The installer's `doctor` checks for it and reports it alongside its other host checks, so an
|
||
operator learns about this while setting the shard up rather than from an empty bestiary weeks
|
||
later.
|
||
|
||
---
|
||
|
||
### 4.5 What phase 0 measured, and the rule it produced
|
||
|
||
Phase 0 ran §4's decoders from inside a live ServUO 57.4 against a client broken in 21 catalogued
|
||
ways ([`servuo-plugins/tools/scaffolding/README.md`](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/src/branch/main/tools/scaffolding/README.md)
|
||
carries the full results). §4's decision **stands** — nothing faulted on a path this protocol
|
||
calls, and §9's cliloc reader reproduced UOFiddler's 123,490-entry table byte for byte in 218 ms.
|
||
|
||
But the spike was looking for the wrong kind of failure, and found a worse one.
|
||
|
||
**A malformed record does not usually throw. It renders the previous asset.** `LoadStatic` and
|
||
`LoadLand` decode out of `m_StreamBuffer`, which is reused across calls, only ever grown, and
|
||
filled by a `stream.Read` whose return value is discarded. So a record that is short, absent or
|
||
out of bounds produces a real bitmap of whatever was decoded last — reported as success by every
|
||
count in the library and undetectable by anything downstream.
|
||
|
||
On the **stock, unmodified** client on this machine that is **22,102 ids**: 9,962 statics and
|
||
12,140 land tiles whose index entry reads `lookup 0, length 0`. `FileIndex.Seek` rejects
|
||
`lookup < 0` and `length < 0`, and zero is neither, so it treats an empty slot as a hit. A bulk
|
||
import that trusted the library would have written 22,102 duplicate images into the site under
|
||
ids that have no art. §1.1's "32,766 decoded" was counting these.
|
||
|
||
**This is specific to the UOP path** (see §4.6), because `artidx.mul` stores `-1` for an absent
|
||
record where an unmapped UOP slot is a zeroed struct.
|
||
|
||
So the rule, and it is the deliverable phase 1 inherits:
|
||
|
||
> **Validate before calling.** The extractor judges an index entry — and, for statics, the record
|
||
> header and row table behind it — *before* handing the id to `Ultima`. A record that fails is
|
||
> reported as absent, never decoded.
|
||
|
||
The checks, all of which phase 0 implemented and measured as `BridgeAssetValidator`:
|
||
|
||
| Check | The shape it stops |
|
||
|---|---|
|
||
| `lookup >= 0`, `length > 0` | the 22,102 empty slots above |
|
||
| `lookup + length <= <data file>.Length` | **`Seek` checks that a record *starts* inside the file and never that it *ends* inside it**; a short read then decodes the previous asset |
|
||
| the same bound against `verdata.mul` for a patched entry | `Verdata.Seek` is bounds-checked **nowhere at all** |
|
||
| land records are at least 2,024 bytes | `LoadLand` reads exactly that many whatever the length says |
|
||
| declared width and height within a ceiling | `LoadStatic` allocates `new Bitmap(width, height)` from two bytes in the file — phase 0 got a ~128 MB allocation out of an edit, and the same field can ask for 8 GB |
|
||
| walk the row table and every run, bounded | `LoadStatic`'s two guards bound the **write** into the bitmap and nothing bounds the **read** out of the record |
|
||
|
||
Measured against the patched client, this refused all eight record-level defects, seven of which
|
||
the library rendered without raising anything. Measured against the stock client it refused
|
||
**nothing** across 49,151 statics and 16,384 land tiles. That second number is the one that makes
|
||
the boundary defensible: a checker that refuses real art would be worse than no checker.
|
||
|
||
Two more ways an id with no art yields a picture, both of which the extractor must handle itself:
|
||
`Art.GetStatic(id, false)` **throws** `IndexOutOfRangeException` above the index's own ceiling
|
||
(16,385 ids in a full sweep), and `Art.GetStatic(id)` with the default `checkmaxid: true` is worse
|
||
— `GetLegalItemID` maps an out-of-range id to **0** and returns **item 0's picture**. Take the
|
||
ceiling from the index that was opened, and pass `checkmaxid: false` so an overrun is loud.
|
||
|
||
**The animation path has none of this yet**, and phase 0 proved it needs it: the patched client's
|
||
verdata entry for body 34 points past `verdata.mul`'s end and the wolf still "decoded", counted
|
||
among the 1,144 successes while rendering something else. `GetAnimation` additionally allocates
|
||
`new int[frameCount]` straight from a file-supplied int. Extending the validator to animations is
|
||
phase 1 work, not a phase 9 tidy-up.
|
||
|
||
### 4.6 The UOP wins outright, and `art.mul` is never opened
|
||
|
||
`FileIndex`'s UOP constructor ends with a bare `MulPath = uopPath`. **When `artLegacyMUL.uop` is
|
||
present it wins, and `art.mul` / `artidx.mul` are not opened at all.** Every current client ships
|
||
the UOP, so this is the normal case and not an edge one.
|
||
|
||
It matters twice.
|
||
|
||
**For us:** an index entry's `lookup` is an offset into whichever file `FileIndex` resolved, so any
|
||
bound taken against `art.mul` while the index holds UOP offsets is not approximate — it is
|
||
meaningless. Phase 0's first run refused **34,299 perfectly good statics** for "declaring
|
||
10533x2085" on exactly that mistake, and every one of those refusals read like a real finding. The
|
||
extractor must resolve the data file with `FileIndex`'s own precedence, not by name.
|
||
|
||
**For operators:** a shard that adds custom art to `art.mul` while the UOP is still in place gets
|
||
nothing, silently. Worth a line in the admin surface's diagnostics (§14) rather than leaving an
|
||
operator to discover it as "my art did not import".
|
||
|
||
### 4.7 The gump crash, reproduced where it counts
|
||
|
||
§4.1 argued from source that the access violation lives on the `hasExtra: true` branch only `Gumps`
|
||
reaches. Phase 0 called `Ultima.Gumps.GetGump(2)` **once, from inside a running shard**: the ServUO
|
||
process disappeared. No exception line, no `catch` reached, no shutdown, nothing in the console —
|
||
the probe's checkpoint file, written before the call, was the entire record of what happened.
|
||
|
||
`AccessViolationException` is a corrupted-state exception that .NET Framework 4.8 does not deliver
|
||
to ordinary handlers, so on a live shard this is a crash with players on it and **there is no
|
||
in-process defence**. §4.1's "nothing in this protocol calls `Ultima.Gumps`" is therefore an earned
|
||
safety rule rather than a scoping preference, and adding gump art later means fixing that path
|
||
first, deliberately.
|
||
|
||
---
|
||
|
||
### 4.8 What phase 1 measured: the animation path has the same defect, and it is worse
|
||
|
||
§4.5 ended by saying the animation path had no validator and that extending it was phase 1's work
|
||
rather than a phase 9 tidy-up. Phase 1 built it, ran it, and the reason that sentence was right is
|
||
larger than the verdata entry that prompted it.
|
||
|
||
**357 of the 1,144 "decodable" bodies are wrong pictures, on the stock client.** Their index entry
|
||
reads `length 0` — no record at all — and `GetAnimation` returns a real bitmap anyway, for the same
|
||
reason `LoadStatic` does: `m_StreamBuffer` is reused, only ever grown, and filled by a
|
||
`stream.Read` whose return value is discarded.
|
||
|
||
Measured directly, because a count could not tell:
|
||
|
||
| Decode body 320 (`lookup 22638982, length 0`) straight after… | What comes back |
|
||
|---|---|
|
||
| body 12, the dragon | the dragon's bitmap, 176×167, identical hash |
|
||
| body 34, the wolf | the wolf's dimensions, 35×34 |
|
||
| body 400, the human male | the human's bitmap, 27×63, identical hash |
|
||
|
||
That is not a near miss or a misaligned fragment. Body 320 has no art, and it renders **whichever
|
||
creature was decoded before it** — which means the picture a bestiary page got would depend on the
|
||
order the importer happened to walk the catalogue in.
|
||
|
||
So the working set is **787 bodies, not 1,144**:
|
||
|
||
| | Bodies 0–2047, direction 1, first frame |
|
||
|---|---|
|
||
| Real art | **787** |
|
||
| Wrong pictures (empty record, library returned a bitmap) | **357** |
|
||
| Absent, and the library agreed | 903 |
|
||
| `bodyconv` resolves nowhere, nothing swept (§4.3) | 1 |
|
||
| Refused by the record walk | **0** |
|
||
| Threw | 0 |
|
||
|
||
**That last-but-one row is the number that matters as much as the first.** The record-level
|
||
animation checks — palette, frame count, frame table, and every run header walked against both the
|
||
record's own length and the bitmap it locks — refused **nothing** across every real body on a stock
|
||
client. §4.5's rule holds: a checker that refuses real art is worse than no checker, and this one
|
||
does not.
|
||
|
||
#### The player bodies: four of twelve, not six
|
||
|
||
§5.2's table was built from the library's answer alone, and two of the six bodies it listed as
|
||
decoding do not have art:
|
||
|
||
| Body | Index entry | Library | Actually |
|
||
|---|---|---|---|
|
||
| Human male/female (400, 401) | real | 24×64, 24×63 | **art** |
|
||
| Elf male/female (605, 606) | real | 24×64, 24×63 | **art** |
|
||
| **Elf ghosts (607, 608)** | `lookup 27221378, length 0` | 24×63 | **the previous body's picture** |
|
||
| Human ghosts (402, 403) | `lookup -1` | nothing | honestly absent |
|
||
| Gargoyle (666, 667, 694, 695) | `lookup -1` | nothing | honestly absent |
|
||
|
||
Confirmed the same way: body 607 decoded after the dragon is the dragon, after the wolf is the
|
||
wolf, after the human male is the human male. Its 24×63 was the elf *female*'s dimensions, because
|
||
606 is what the catalogue walk decoded immediately before it.
|
||
|
||
Two consequences, both of which change work elsewhere in this document:
|
||
|
||
- **§4.3's UOP decoder covers eight player bodies, not six.** The elf ghosts join the human ghosts
|
||
and the four gargoyle bodies. Phase 4's scope grows by two ids and its argument does not change.
|
||
- **The `lookup -1` / `length 0` distinction is the whole difference between an honest absence and
|
||
a wrong picture**, and it is not visible from outside the index. `artidx.mul` and the legacy
|
||
`anim*.idx` write `-1` for a record that is not there; an empty UOP slot, and evidently a
|
||
deliberately blanked legacy entry, is a zeroed struct. `FileIndex.Seek` rejects the first and
|
||
accepts the second, and the second is 357 creatures and two playable ghosts.
|
||
|
||
#### Why this could not have been found any other way
|
||
|
||
Phase 0 ran this exact sweep and reported "1,144 decoded, 904 empty, 0 faults", and every one of
|
||
those numbers is true. The library raised nothing, returned bitmaps of plausible sizes, and agreed
|
||
with itself. Nothing downstream of the decode — not a count, not an exception, not a hash of the
|
||
output, not a look at one picture in isolation — distinguishes body 320's dragon from body 12's.
|
||
The only things that did were **validating the index entry before the call** and **decoding the
|
||
same id twice after different neighbours**.
|
||
|
||
That is the same method note §4.3 ended on, and this is its second confirmed catch. Any time a UO
|
||
file lookup is addressed by index, a success count is evidence of nothing.
|
||
|
||
---
|
||
|
||
## 5. Addressing: one key for every asset
|
||
|
||
Every asset the bridge can serve is named by a single string key, and the key is the cache key,
|
||
the hash key, the filename stem and the manifest row id:
|
||
|
||
```
|
||
static/3922 one item graphic
|
||
static/3922/h33 the same graphic, hue 33 applied
|
||
land/3 one land tile
|
||
body/34/a0 creature body 34, action 0, first frame
|
||
body/400/a0/f0..f9 human male, action 0, all ten frames
|
||
cliloc/enu the whole converted string table (not an image)
|
||
tree/Spawns/Trammel.xml a ServUO tree file (§10)
|
||
```
|
||
|
||
Three properties this shape buys:
|
||
|
||
- **Hue is part of the key, not a transform.** `itemId` and `hue` are already on the wire together
|
||
(`BridgeMarket.cs:582`, `BridgeProfile.cs:314`), so a marketplace listing already knows the exact
|
||
key for its own picture. Applying hues website-side would mean shipping `Hues.mul` semantics into
|
||
Node for no gain.
|
||
- **Depth is expressible without being mandatory.** `body/400/a0` and `body/400/a0/f0..f9` are the
|
||
same addressing scheme at two depths, which is what lets §11 bulk-import thumbnails and fetch full
|
||
animations on demand without a second protocol.
|
||
- **Nothing in the key is client-version-specific**, so a client patch changes an asset's *bytes*,
|
||
not its name — which is what makes §7's delta work.
|
||
|
||
### 5.1 There is no direction segment, because only one direction is wanted
|
||
|
||
Bodies are stored in **five** directions and the client mirrors three of them to reach eight. Only
|
||
one is needed here, so **direction is fixed by the extractor and is not part of the key**. Leaving
|
||
it in would advertise a choice nobody is going to vary and would five-fold every count in §11 for
|
||
nothing.
|
||
|
||
**Which one depends on whether the body is a player character:**
|
||
|
||
| Body | Direction | Why |
|
||
|---|---|---|
|
||
| A player character body | **0** — head-on, facing the viewer | A character is a portrait; it should look at you |
|
||
| Everything else | **1** — front three-quarter | The view that actually reads as a creature (see the caveat below) |
|
||
|
||
Which index is which was determined by **rendering all five** for a human, a wolf and a dragon
|
||
rather than from a table, because the answer is not obvious and the small-thumbnail version of the
|
||
same test suggested the exact opposite:
|
||
|
||
| Index | View |
|
||
|---|---|
|
||
| **0** | **Head-on, facing the viewer** — face, chest and front legs visible |
|
||
| 1 | Front three-quarter |
|
||
| 2 | Full side profile |
|
||
| 3 | Rear three-quarter |
|
||
| 4 | Directly away — back of the head, and a quadruped's tail toward the camera |
|
||
|
||
The caveat the render made obvious is what produced the split: **index 0 is the least legible view
|
||
for four-legged and long-bodied creatures.** A wolf seen head-on is a dark blob; the same wolf at
|
||
index 1 is unmistakably a wolf, which is also why UOFiddler's own thumbnail list picks that view. A
|
||
humanoid has no such problem — it reads fine head-on, and head-on is what a character portrait
|
||
wants.
|
||
|
||
Both indices stay **configuration values** (defaulting to 0 and 1), so changing the catalogue's mind
|
||
later is a setting and a re-import, not a protocol change.
|
||
|
||
### 5.2 "Player character body" is asked of the shard, never hardcoded
|
||
|
||
`Server.Race.AllRaces` gives every registered race, and each carries `MaleBody`, `FemaleBody`,
|
||
`MaleGhostBody` and `FemaleGhostBody`. The plugin enumerates those four ids per race and that set —
|
||
nothing else — takes index 0. On stock ServUO 57.4 that is twelve ids:
|
||
|
||
| Race | Male | Female | Male ghost | Female ghost |
|
||
|---|---|---|---|---|
|
||
| Human | 400 | 401 | 402 | 403 |
|
||
| Elf | 605 | 606 | 607 | 608 |
|
||
| Gargoyle | 666 | 667 | **695** | **694** |
|
||
|
||
This is the §8 argument again in miniature: only code inside ServUO can answer it, and asking is
|
||
the only thing that works on a shard with a custom race. Two details make the case that a
|
||
hardcoded list would have been wrong — `RaceDefinitions.cs` passes the gargoyle's ghost bodies in
|
||
the **opposite order** to the other two races (695 male, 694 female), and a shard that calls
|
||
`RegisterRace` adds ids no table of ours would contain.
|
||
|
||
**Two thirds of that set has no art in ServUO's vendored library.** Measured — and re-measured in
|
||
phase 1, which moved two rows across (§4.8):
|
||
|
||
| Has art | Does not |
|
||
|---|---|
|
||
| Human male/female (400, 401) | **Human ghosts (402, 403)** — `lookup -1` |
|
||
| Elf male/female (605, 606) | **Elf ghosts (607, 608)** — `length 0`, and the library returns the previously-decoded body's bitmap |
|
||
| | **Every gargoyle body (666, 667, 694, 695)** — `lookup -1` |
|
||
|
||
**Four of twelve, not six.** The elf ghosts were counted as decoding for as long as the library's
|
||
own answer was the measurement: their index entry has no record, and what came back was whatever
|
||
the catalogue walk had decoded the moment before — the elf female, at her exact dimensions. All
|
||
eight are UOP-only and **§4.3 adds a decoder for them** — verified genuinely absent from the legacy
|
||
files rather than merely mis-addressed, by rendering what the other anim files hold at those
|
||
indices and finding spiders.
|
||
|
||
Until that phase lands, and on any client that lacks them entirely, the catalogue must **not** treat
|
||
a missing player body as an error. It is the expected answer for two thirds of the set, and a
|
||
status screen that flags eight failures on every import teaches an operator to ignore it.
|
||
`shard_spawn_creatures.art` staying NULL remains a first-class state everywhere it is consumed,
|
||
which it already is.
|
||
|
||
---
|
||
|
||
## 6. The manifest, and what the two buttons actually do
|
||
|
||
Two stages, which is where **Import** and **Update** come from.
|
||
|
||
**Stage 1 — the source gate.** The shard reports a manifest of the client files themselves:
|
||
size, mtime and content hash of `Cliloc.enu`, `anim*.idx`/`anim*.mul`, `art.mul`/`artidx.mul`,
|
||
`Body.def`, `Bodyconv.def`, `Hues.mul`. Unchanged since the last import, and nothing else happens.
|
||
This is the same hash gate the spawn atlas and the cliloc table already use, and for the same
|
||
reason: the normal case is a restart that changed nothing, and it must cost nothing.
|
||
|
||
`anim.mul` is 195 MB and `art.mul` is 148 MB, so the gate is **(size, mtime) first, content hash
|
||
only when those differ** — a full hash of 343 MB on every status poll would make the admin panel
|
||
feel broken.
|
||
|
||
**Phase 1 found that the rule is not sufficient on its own, because of §3.3's other ceiling.** The
|
||
first time those hashes are needed there is nothing cached to compare against, so "hash only when
|
||
(size, mtime) differ" still means hashing 343 MB — inside a 10 s reply timeout it will not fit,
|
||
and the call would 504 rather than answer. So the shard's hashes are computed **off the request
|
||
path entirely**:
|
||
|
||
- `assets.sources` always answers immediately, with `size` and `mtime` for every file and
|
||
`sha256: null` for any file whose hash is not cached against exactly that (size, mtime).
|
||
- A file with a missing hash starts a background pass on its **own** thread — deliberately not a
|
||
job on §3.2's single-slot worker, which would answer every status poll `bridge.busy` for the
|
||
minutes the pass takes, at exactly the moment an operator is watching the panel.
|
||
- The reply carries `hashing` and `complete`, so the website knows to poll again rather than to
|
||
treat a null hash as a changed file.
|
||
|
||
The gate is unchanged; what changed is that "the normal case must cost nothing" now also means
|
||
"and the abnormal case must not time out".
|
||
|
||
**Stage 2 — the asset manifest.** For the working set (§11), the shard streams
|
||
`[{ key, sha256, bytes }]` — no pixels. The website diffs that against what it holds and requests
|
||
**only the keys whose hash changed**.
|
||
|
||
- **Update** = stage 1, then stage 2, then fetch the diff.
|
||
- **Import** = the same path with the diff skipped and every key fetched.
|
||
- **A key that has vanished** from the manifest is staged for review, never applied silently —
|
||
the same rule, and the same reasoning, as a vanished cliloc source or a disappearing atlas
|
||
facet. An unmounted volume and a deliberate client downgrade look identical from here.
|
||
|
||
Clilocs are the exception and stay a **whole-table replace** whenever the file hash changes:
|
||
the measured cost is 663 ms for 67,496 rows, so per-entry deltas would be complexity bought for
|
||
nothing.
|
||
|
||
---
|
||
|
||
## 7. The parser version applies here too
|
||
|
||
`spawnAtlasSource.js` carries `PARSER_VERSION` (currently 5) and the cliloc source carries its own,
|
||
both counted as drift so that a corrected parse reaches an install whose files never change. The
|
||
asset pipeline inherits the rule and needs it more, not less: a fixed hue application or a
|
||
corrected frame offset changes the bytes we derive from files that are byte-identical.
|
||
|
||
**`EXTRACTOR_VERSION` lives in the plugin**, because the plugin is what derives the bytes, and it
|
||
is folded into stage 1's gate. Bumping it makes every asset drift, which is correct.
|
||
|
||
---
|
||
|
||
## 8. Body ids: the part only the shard can do
|
||
|
||
The atlas knows creatures by **slug**, derived from type names in `Spawns/*.xml`. The client knows
|
||
them by **body id**. Nothing in the ServUO tree declares the mapping as data — today an operator
|
||
bridges it by grepping `Scripts/Mobiles/Normal/<Name>.cs` for `Body =`, which appears variously as
|
||
a decimal, as hex (`0xD1`), as `Utility.RandomList(35, 36)`, and as an `m_IDs[]` table.
|
||
|
||
Inside ServUO the problem does not exist. `BridgeWorld.cs:350` already does exactly the required
|
||
thing for a different feature:
|
||
|
||
```csharp
|
||
var type = ScriptCompiler.FindTypeByName(name, true);
|
||
var creature = Activator.CreateInstance(type) as BaseCreature;
|
||
```
|
||
|
||
Construct, read `creature.Body.BodyID`, `Delete()`. Authoritative, no source parsing, and correct
|
||
for custom creatures a grep would never find.
|
||
|
||
**This pass must run on the Core thread** — it constructs and deletes mobiles, which is world
|
||
mutation — while the decode in §4 must run **off** it. That split is the one genuinely new
|
||
threading shape in this protocol, and it is why slug→body resolution is its own request kind with
|
||
its own (small) batch size rather than a step inside asset extraction.
|
||
|
||
Constructing arbitrary creature types has side effects: constructors pack items, set skills, start
|
||
timers. The mitigations are per-type `try`/`catch`, immediate `Delete()`, small batches, and the
|
||
fact that the whole pass is admin-triggered rather than something that runs at boot.
|
||
|
||
### 8.1 What phase 3 measured
|
||
|
||
**Built and walked 2026-09-10** against a live ServUO with a real world (43,000 mobiles, 210,000
|
||
items) and the real sidecar.
|
||
|
||
**The catalogue is 787, exactly as §4.8 predicted**, and it arrives in one page: the whole scan of
|
||
bodies 1–2047 — index validation, 787 decodes, 787 PNG encodes and 787 SHA-256s — took **734 ms**
|
||
cold. That is well inside the 3 s scan budget, so the wall-clock paging §11 was designed for never
|
||
fired on this client. It stays, because the budget is what keeps a slower host or a larger family
|
||
inside the 10 s reply timeout rather than producing replies that are always thrown away.
|
||
|
||
Every prediction in §4.8 and §5.2 held when the bytes were actually rendered and looked at:
|
||
|
||
| Asked for | Answer | Why it matters |
|
||
|---|---|---|
|
||
| `body/320/a0` (`length 0`) | **absent** | The 357-class bug. The library would have returned the previously-decoded creature |
|
||
| `body/607/a0`, `body/608/a0` (elf ghosts) | **absent** | §4.8's two rows that moved; the library returns the elf female here |
|
||
| `body/666/a0` (gargoyle → `anim5`) | **absent** | The spider trap. Nothing swept, nothing found |
|
||
| `body/400/a0/…` deeper key | **unsupported** | Well-formed under §5, not served until phase 6 |
|
||
| direction distribution across 787 | **783 at index 1, 4 at index 0** | Four player bodies, not six — §4.8 again, from `Race.AllRaces` rather than a table |
|
||
|
||
**44 of the 787 hashes are shared by two or three bodies, and that is correct.** It is the exact
|
||
signature the wrong-picture bug produces, so it was chased rather than assumed: the client's own
|
||
`Body.def` says `83 {1}`, `84 {1}`, `138 {7}`, `139 {7}`, `106 {12, 59}`, and the sharing groups
|
||
match those lines rather than being runs of consecutive ids (which is what a reused stream buffer
|
||
produces). The distinguishing check is at the source: `Animations.Translate(ref body, ref hue)`
|
||
rewrites `body` **only when bit 31 of the table entry is set**, unlike the one-argument overload
|
||
which always does — and `BridgeAssetValidator.ResolveAnimation` calls that same two-argument
|
||
overload, matching `GetAnimation(…, preserveHue: false, …)`. Validator and decoder therefore
|
||
resolve the identical record, which is the property the whole §4.5 design rests on.
|
||
|
||
**The Core-thread pass costs about 190 ms per 100 types.** All 455 stock `Scripts/Mobiles/Normal`
|
||
classes were constructed and deleted in five chunks, producing every status the protocol defines
|
||
(`ok` 436, `unknown` 11, `notCreature` 4, `failed` 4). The world's mobile count went from 43,000 at
|
||
boot to 42,924 afterwards and its item count fell too — so `Delete()` is reclaiming the packed
|
||
inventory as well as the mobile, and nothing leaked.
|
||
|
||
The three refusals answer as designed end to end: an unknown `family` is **400**, a stale `catalog`
|
||
on a fetch is **422**, and 101 types in one `assets.bodies` is **400** naming the cap.
|
||
|
||
---
|
||
|
||
## 9. The cliloc decompressor is ours now
|
||
|
||
Every modern client ships `Cliloc.*` in the Mythic compressed container — this machine's
|
||
`Cliloc.enu` is 4,989,921 bytes beginning `E8 79 67 8E`, high byte `0x8E`. ServUO's bundled
|
||
`Ultima.StringList` implements only the plain layout and throws on it (§1.1), which is also why
|
||
the shard's own `VendorSearch.GetItemName` is already inert.
|
||
|
||
**UOFiddler is released under the Beerware licence**, so porting its decompressor into our
|
||
GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C# — the **only** decoder
|
||
Protocol 8 writes rather than calls (§4) — and from that point:
|
||
|
||
- No operator installs UOFiddler.
|
||
- No operator runs `dotnet build` on a converter.
|
||
- No operator copies a 5 MB file to a server.
|
||
- `website/server/tools/cliloc-export/` is retired, and `UOFIDDLER.md` is deleted rather than
|
||
rewritten. (Phase 2 deleted its Part 1; **phase 3 deleted the page**, a phase earlier than that
|
||
section predicted, because creature art was the only thing left on it.)
|
||
|
||
### 9.1 What phase 2 built, and what the port cost
|
||
|
||
**Built 2026-09-10.** `BridgeCliloc.cs` in the overlay, `GET /cliloc` in the sidecar, and a paging
|
||
walk in module-uo that merges the shard's table under the `custom/` overlays.
|
||
|
||
The port is `MythicDecompress` + `MoveToFront` rewritten against plain arrays: the upstream is
|
||
`Span<T>` / `ArrayPool<T>` / `BinaryPrimitives` code and ServUO targets `net48`, which has none of
|
||
them without a package this tree does not vendor. The algorithm is unchanged, **including the parts
|
||
that read oddly** — the three-region count/cursor/end table and the symbol-table shifts are
|
||
upstream's, deliberately, because a tidier rewrite of somebody else's format decoder is a chance to
|
||
be subtly wrong in a way that produces plausible text.
|
||
|
||
Two bounds checks were added, and they are the only behavioural change: the upstream indexes its
|
||
payload without checking, which is safe for a file the client wrote and is not safe for a file this
|
||
shard was handed. A truncated container now reports `UNREADABLE` with the byte it wanted, instead of
|
||
throwing an `IndexOutOfRangeException` from inside a decoder.
|
||
|
||
**Measured on this machine's stock client:** 4,989,921 bytes read, decompressed and parsed in
|
||
**290 ms**, yielding **67,496** non-blank rows in id order, ~5.4 MB on the wire, ~11 pages.
|
||
|
||
That 67,496 is the acceptance test, and it is worth saying why it is a strong one: it is the number
|
||
[`../website/CLILOCS.md`](../website/CLILOCS.md) already recorded for this same client, measured
|
||
through **UOFiddler's own `Ultima.dll`** by the converter this phase deletes. An independent
|
||
implementation agreeing to the row is not something a subtly-wrong decoder produces. Checked
|
||
alongside it: zero U+FFFD, so the UTF-8 survived; 696 rows with non-ASCII text, spot-checked as
|
||
correct curly quotes; the longest row is a 12,149-character EULA, which is also why the record
|
||
length is read **unsigned** (a signed read turns anything over 32 KB negative).
|
||
|
||
**The shard drops blanks before they reach the wire.** ~56,000 of the 123,490 entries are empty
|
||
strings the client reserves, the website discards them at import anyway, and sending them would
|
||
double the transfer for data thrown away on arrival.
|
||
|
||
**Language is a parameter (`enu` by default) but not a free one.** `Ultima.Files` resolves only the
|
||
names in its own table, which for clilocs is `enu`, `deu`, `custom1` and `custom2`; anything else is
|
||
refused `NOT_FOUND` rather than answered with an empty table. `custom1`/`custom2` are the
|
||
client-side custom cliloc files a shard ships to its players — readable here, and deliberately not
|
||
wired into the website's import, because `custom/` on the site is the supported answer for
|
||
shard-added ids.
|
||
|
||
**`assets.error` gained a `code`.** Phase 1 chose between 403 and 400 by looking for the word
|
||
"disabled" in the operator-facing sentence, which makes prose load-bearing; the codes are
|
||
`DISABLED` (403), `NOT_FOUND` (404), `UNREADABLE` (422), `UNAVAILABLE` (503) and `BAD_REQUEST`
|
||
(400). The substring check survives as a fallback, because an overlay and a sidecar are deployed
|
||
separately and a phase-1 shard must keep its 403.
|
||
|
||
### 9.2 Where the base comes from now, and when it is read
|
||
|
||
**The shard wins whenever uo-link is configured and enabled.** No mode setting: there is no version
|
||
of that question an operator benefits from answering. A file on disk remains the source only where
|
||
there is no shard link, plus a one-off explicit path — the deprecated pipeline, kept for installs
|
||
with no bridge and for development.
|
||
|
||
**Boot no longer imports on the bridge.** The file path could hash 5 MB locally and skip in 14 ms; a
|
||
shard round trip in the boot sequence would be spent answering "no" on every restart but the one
|
||
after a client patch — and patching a client is an operator action, so importing became one:
|
||
Admin → Shard → Import. Whatever table is loaded keeps serving until then.
|
||
|
||
Three things about the walk are worth recording because each is a way a shard can hand back a table
|
||
that *looks* complete:
|
||
|
||
- **Only `cut: "end"` finishes it** (§3.4). A short page can equally be a spent budget.
|
||
- **The cursor must advance**, or the walk stops rather than spinning.
|
||
- **Every page echoes the source's size and mtime.** A client patched mid-import is refused
|
||
outright (`SOURCE_CHANGED`) rather than stitched together — half of what arrived came from a file
|
||
that no longer exists and nothing later can tell which half.
|
||
|
||
**The base is exempt from the vanished-source rule**, and that is an upgrade detail rather than a
|
||
design preference: an install that used the file pipeline carries its base file's label in the
|
||
stored fingerprint, and on the bridge that label is *supposed* to disappear. Counting it as vanished
|
||
would make the first import after the upgrade demand approval for a change the upgrade itself made.
|
||
Overlays keep the rule in full.
|
||
|
||
**module-uo's protocol pin moved 7 → 8 in this phase**, which is the third declaration site §15
|
||
names and the one nothing enforces. Phase 1 moved the sidecar and the overlay together because the
|
||
installer refuses to pair a mismatched bundle; this one had to be moved by hand, in the phase that
|
||
first calls a protocol-8 route — the same trap that left the pin at 5 for two phases of the Event
|
||
System while every REST call was answered `409`.
|
||
|
||
**What survives untouched is the `custom/` overlay mechanism.** Shard-added items carry cliloc ids
|
||
no client table has, and ServUO has no server-side notion of a custom cliloc — that is a real gap
|
||
in the *game*, not an artefact of the manual pipeline, and `CLILOCS.md`'s reasoning for it stands.
|
||
The base table now arrives over the bridge; overlays still come from a directory the site reads.
|
||
Measured on the live shard: 16,434 cliloc ids referenced by the script tree, 37 absent from stock.
|
||
|
||
---
|
||
|
||
## 10. The atlas stops needing a shared filesystem
|
||
|
||
Today `SPAWN_ATLAS.md` requires the **website** to read the ServUO tree — "same host, a bind mount,
|
||
or a shared volume". That is the one place the platform's own rule (only the sidecar bridges the
|
||
shard) is broken, and it is broken by the component that faces the internet.
|
||
|
||
The same transport closes it. `spawnAtlasSource.js` already labels every file it reads with a
|
||
portable key:
|
||
|
||
| Label | Count (stock 57.4) |
|
||
|---|---|
|
||
| `Data/Regions.xml` | 1 |
|
||
| `Data/Locations/*.xml` | 6 |
|
||
| `Spawns/*.xml` | 13, ~10.5 MB |
|
||
| `Config/ChampionSpawns.xml` | 1 |
|
||
| `Data/Decoration/**` | tree |
|
||
|
||
So the shard serves `tree/<label>` → bytes over the same batched request/reply path, and
|
||
`spawnAtlasSource.js` gains a second backend behind its existing interface: **filesystem** (today,
|
||
kept for same-host installs and for development) or **sidecar** (new, and the default once
|
||
configured).
|
||
|
||
**The parsers do not move.** `spawnAtlasParse.js` is pure, fs-free and CI-covered without a ServUO
|
||
tree, and every quirk it handles — the two respawn delay units, `:OBJ=` splitting, facet-name
|
||
reconciliation, the XmlSpawner directive stripping — stays exactly where it is. The shard sends
|
||
bytes; the website still decides what they mean. That is the same division as §2, and it keeps the
|
||
sidecar a forwarder here too.
|
||
|
||
`SERVUO_PATH` and the `spawn_atlas_servuo_path` setting remain, and select the filesystem backend.
|
||
|
||
---
|
||
|
||
## 11. What is bulk and what is on demand
|
||
|
||
The scope approved is creature art, item art, player models "and everything", against a future
|
||
project. §1.1's measurements make the sizing question concrete:
|
||
|
||
| Kind | Addressable | Bulk? |
|
||
|---|---|---|
|
||
| Item statics | **~49,150** | No — on demand, cached, keyed by `itemId` (+ hue) |
|
||
| Land tiles | **16,384** | No — on demand |
|
||
| Creature/player bodies, first frame | **787** (§4.8 — *not* the 1,144 the library reports) | **Yes** — this is the catalogue |
|
||
| One body, every action, one direction | **210 frames** (body 400); 96–210 measured across six bodies | No — on demand, per body |
|
||
| All bodies, every action, one direction | **~119,000 frames**, ~117 MB | No — but no longer unthinkable |
|
||
| The same at five directions | ~865,000 frames | Not built (§5.1) |
|
||
| Cliloc table | 123,490 entries → 67,496 rows | **Yes** — whole-table replace |
|
||
| ServUO tree files (§10) | ~21 files, ~10.6 MB | **Yes** |
|
||
|
||
**The working set is one thumbnail per body, plus the atlas's own creatures.** 787 sprites at
|
||
roughly a kilobyte each is under 1 MB — trivial to import, trivial to re-hash, and it is the set
|
||
that makes a bestiary, a marketplace listing and a character sheet render.
|
||
|
||
Phase 1 took that count down from 1,144, and the 357 it removed are the important part: those are
|
||
ids with **no art** that the library returns a picture for (§4.8). Importing them would have
|
||
written 357 duplicate creature portraits into the site, each one showing whichever body the walk
|
||
happened to decode before it. The count went down; what the catalogue is worth went up.
|
||
|
||
Everything deeper is the *same protocol at a deeper key* (§5), fetched on demand and cached. That
|
||
is what serves the future project without exporting 3.5 GB of someone else's copyrighted client
|
||
into a database: a viewer that wants body 400's full walk cycle asks for `body/400/a2/f0..f9` and
|
||
gets it, once, and it is cached from then on.
|
||
|
||
Because §5.1 dropped four of the five directions, a **complete** one-direction animation set for
|
||
every body is now ~119,000 frames rather than ~865,000 — around 117 MB (~173,000 before §4.8 cut
|
||
the body count to the ones that have art; both figures carry the same measured ~151-frame average
|
||
per body). That is still not the
|
||
default and still not something to import before anything asks for it, but it has moved from
|
||
"never" to "a thing an operator could reasonably choose", and phase 6 should leave room for a
|
||
bulk-fill-everything switch rather than assuming on-demand is the only mode.
|
||
|
||
**Hued variants are on demand, always.** `static/3922/h33` is generated when something on the wire
|
||
actually carries hue 33. The cross product of 49,150 statics and ~3,000 hues is not a set anyone
|
||
enumerates.
|
||
|
||
**Gump art is out of scope for Protocol 8, and that is now a safety rule rather than a priority
|
||
call** — §4.1. It is the only decoder that reaches the `hasExtra: true` branch, and that branch
|
||
corrupts the process on the second id. Adding gump art later means fixing that path first,
|
||
deliberately; it is additive under the same key scheme (`gump/<id>`), which is the point of §5.
|
||
|
||
---
|
||
|
||
## 12. Where it lands on the website
|
||
|
||
Images are written by module-uo into the upload directory. `ctx.uploads`
|
||
(`{ upload, UPLOAD_DIR, MIME_EXT }`) is **already** exposed to modules and
|
||
[`MODULE_API.md:626`](../website/MODULE_API.md) already names its consumer as "atlas art import",
|
||
so no `MODULE_API_VERSION` bump is needed to store them.
|
||
|
||
- `shard_spawn_creatures.art` stops being NULL-by-default and starts being filled by the import.
|
||
- A new asset table carries `key`, `sha256`, `bytes`, `width`, `height`, `imported_at` — the
|
||
manifest side of §6, and what makes an Update a diff rather than a re-download.
|
||
- **The operator-supplied `spawnAtlas.art.json` map stays supported** and continues to win over an
|
||
imported asset. An operator who has drawn their own creature portraits must not have them
|
||
overwritten by a sprite rip on the next Update.
|
||
|
||
Licensing is unchanged and the reasoning is unchanged: these are the operator's own client files,
|
||
extracted on their own host, for their own shard. Nothing is committed, nothing ships in a repo,
|
||
and nothing is redistributed. What changes is only that the extraction stopped requiring a GUI on a
|
||
desktop.
|
||
|
||
### 12.1 The one thing above that phase 3 had to build differently
|
||
|
||
"`shard_spawn_creatures.art` … starts being filled by the import" is right about the outcome and
|
||
wrong about the mechanism, and the difference is not cosmetic.
|
||
|
||
**That table is emptied and refilled by every atlas refresh** — `shardAtlas.db.js`'s `replaceAtlas`
|
||
DELETEs all six atlas tables inside one transaction — and a refresh runs on **every boot**. Before
|
||
protocol 8 that cost nothing, because `art` came from a file on disk and was simply re-read each
|
||
time. An imported sprite is the opposite: expensive to obtain, and gated on client-file hashes that
|
||
would say "unchanged" for weeks afterwards. Writing it onto the creature row would mean an ordinary
|
||
re-parse of the ServUO tree silently deleting every portrait, with the next Update reporting
|
||
*nothing to do* and never restoring them. Nothing would report a fault; the pictures would just be
|
||
gone.
|
||
|
||
So phase 3 built three tables outside that blast radius, and the atlas import reads them on the way
|
||
past (org lead, 2026-09-10):
|
||
|
||
| Table | Holds | Lifetime |
|
||
|---|---|---|
|
||
| `shard_assets` | `asset_key`, `sha256`, `bytes`, `width`, `height`, `body`, `direction`, `file` | Upserted per key; only an approved vanish deletes |
|
||
| `shard_creature_bodies` | `slug`, `type_name`, `body`, `status` | Replaced whole — it is derived from the atlas's creature list, so a slug that has left has no meaning |
|
||
| `shard_asset_meta` | The singleton an Update compares against | Replaced |
|
||
|
||
`replaceAtlas` now takes `{ ...derived, ...operatorMap }`, which is where "the operator's map wins"
|
||
is actually enforced — one spread, in one place, applied on every rebuild rather than only at
|
||
import time.
|
||
|
||
**Two details worth not rediscovering.** The derivation joins on the catalogue key
|
||
(`a.asset_key = CONCAT('body/', b.body, '/a0')`), not on `a.body = b.body`: today one body has
|
||
exactly one asset and the simpler join is correct, and it stops being correct the moment phase 6
|
||
adds `body/400/a2/f0`, at which point one slug matches dozens of rows and whichever the engine
|
||
returned last becomes the portrait. And the stored filename is **content-addressed**
|
||
(`uo-body-34-a0-<sha8>.png`), because a stable name overwritten in place leaves every browser and
|
||
CDN serving last month's client's sprite from cache with the database row perfectly correct.
|
||
|
||
`shard_creature_bodies` also answers §8 without a schema change on the atlas side:
|
||
`shard_spawn_creatures.name` already holds the ServUO **class name** — the atlas build picks the
|
||
winning spelling of the spawn type token rather than inventing a display label — so the import has
|
||
something `ScriptCompiler.FindTypeByName` will resolve without storing it twice.
|
||
|
||
---
|
||
|
||
## 13. Visibility
|
||
|
||
New surfaces over shard data are admin-toggleable with an operator-set audience, and this is no
|
||
exception. Asset serving is gated like every other shard read: a `requireFeature` gate, an
|
||
audience, and 404-not-403 when the feature is off, so a disabled feature does not advertise itself.
|
||
|
||
The default is the least surprising one: assets are as public as the page that uses them. A
|
||
bestiary that is already anonymous does not become staff-only because its pictures arrived over a
|
||
new pipe.
|
||
|
||
---
|
||
|
||
## 14. Routes and commands added
|
||
|
||
**Loopback (shard ↔ sidecar), all request/reply:**
|
||
|
||
| Command | Reply | Purpose | Built |
|
||
|---|---|---|---|
|
||
| `assets.sources` | `assets.sources.ok` | Stage 1: client file manifest + `EXTRACTOR_VERSION` | **phase 1** |
|
||
| `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes, width, height, body, direction}]`, paged | **phase 3** |
|
||
| `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged; base64 PNG per row | **phase 3** |
|
||
| `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) | **phase 3** |
|
||
| `cliloc.table` | `cliloc.table.ok` | The decompressed table, paged (`?lang=`, `?cursor=`) | **phase 2** |
|
||
| `tree.manifest` / `tree.fetch` | `.ok` | §10, the ServUO tree files | phase 7 |
|
||
|
||
Every one of them is refused outright when `Bridge.AssetsEnabled` is off, and every one of them
|
||
requires a `reqId` — a request without one is refused rather than answered, because an uncorrelated
|
||
reply is by definition an event, and §3.1 is the reason none of this may ever be one.
|
||
|
||
**Sidecar REST** mirrors those one for one under `/assets/*`, `/cliloc`, `/tree/*`, carrying
|
||
`X-UOLink-Version: 8` and forwarding verbatim. `GET /assets/sources` exists as of phase 1 and
|
||
`GET /cliloc` as of phase 2; their responder maps `bridge.busy` to **425** (flow control, and the
|
||
ordinary answer mid-import rather than a rare one) and reads the refusal's `code` for the rest —
|
||
**403** disabled, **404** no such file, **422** a file the shard cannot decode, **503** a shard that
|
||
cannot do this now (§9.1).
|
||
|
||
Phase 3 added the other three: `GET /assets/manifest?family=&cursor=`, `POST /assets/fetch` and
|
||
`POST /assets/bodies`. **The two POSTs are reads**, and the method is the request body rather than a
|
||
side effect — a few hundred asset keys do not belong in a query string. They are the only reads on
|
||
this link that take one. `422` gains a second meaning on this plane alongside "cannot decode": the
|
||
mid-import guard, a `catalog` that no longer describes the files on disk.
|
||
|
||
**§16 listed phase 3 as `servuo-plugins, module-uo` and that was wrong** — `web.rs` routes every
|
||
command explicitly and has no generic `/assets/*` forwarder, so `link` is in the phase too. The
|
||
table now says so.
|
||
|
||
**Website admin** (`Admin → Shard`, admin-only): status, **Import**, **Update**, approve/reject for
|
||
a vanished key, and the existing path settings. Every action to the admin activity log, as
|
||
`shard.assets.*`. Phase 3 shipped the two that make it reachable — `GET /admin/shard/assets` and
|
||
`POST /admin/shard/assets/import`, mirroring the cliloc pair — so the phase could be accepted on a
|
||
real rig; the full panel is phase 8 (org lead, 2026-09-10).
|
||
|
||
---
|
||
|
||
## 15. Cross-repo obligations
|
||
|
||
`PROTOCOL_VERSION` goes **7 → 8** in `link/sidecar/src/main.rs`, and in the **same PR**
|
||
`servuo-plugins/overlay.toml` — the installer refuses to pair a sidecar and an overlay that
|
||
disagree, so a split bump means the next bundle silently fails to compose.
|
||
|
||
| Repo | Work |
|
||
|---|---|
|
||
| `servuo-plugins/` | Extraction over ServUO's own `Ultima` (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` |
|
||
| `link/` | Six command families forwarded, the REST surface, **the inbound line cap (§3.3)**, `PROTOCOL_VERSION` |
|
||
| `module-uo/` | Client calls, asset store, the atlas source backend (§10), cliloc ingest, admin surface |
|
||
| `website/` | Almost none — `ctx.uploads` already suffices (§12). Phase 2 deleted `server/tools/cliloc-export/`, the converter this protocol retires |
|
||
| `docs/` | This file; rewrite `CLILOCS.md` §Converting and `SPAWN_ATLAS.md` §Artwork + §Configuring; **delete `UOFIDDLER.md`**; add the libgdiplus prerequisite to `SHARD_PREREQS.md` (§4.4) |
|
||
| `installer/` | A `doctor` check for libgdiplus on Linux hosts (§4.4). Bundle pairing already enforces §15 |
|
||
| `android-app/` | Consumes images by URL; no parity gate expected until a screen shows one |
|
||
| `integration-kit/` | A chapter note only — this is UO-specific and teaches nothing about the module contract |
|
||
|
||
---
|
||
|
||
## 16. Phases
|
||
|
||
| # | Scope | Repos |
|
||
|---|---|---|
|
||
| 0 | **DONE 2026-09-10.** Spike: the vendored decoders driven from inside a running ServUO, over a client broken 21 ways. §4 stands; the finding was 22,102 wrong pictures on a *stock* client, and the validator that answers them (§4.5-§4.7) | servuo-plugins |
|
||
| 1 | **DONE 2026-09-10.** The transport: `assets.sources`, the single-slot gate (§3.2), the paging envelope (§3.4), the sidecar line cap (§3.3), `EXTRACTOR_VERSION`, `NO_IMAGING`, protocol 7→8. Plus §4.5's validator promoted into the overlay and extended to animations — which found **357 wrong pictures in the body catalogue on a stock client** and cut it from 1,144 to 787 (§4.8) | servuo-plugins, link |
|
||
| 2 | **DONE 2026-09-10.** Clilocs end to end (§9.1, §9.2): the Mythic decompressor ported into the overlay, `cliloc.table` + `GET /cliloc`, the paging walk and the source switch on the website, module-uo's protocol pin 7→8. `cliloc-export/` deleted and `UOFIDDLER.md` §Part 1 with it. **67,496 rows, 290 ms, ~11 pages** — the same count UOFiddler's own DLL produced from this client | all |
|
||
| 3 | **DONE 2026-09-10.** Body resolution (§8) + the **787**-body catalogue (§4.8), `assets.manifest` / `assets.fetch` / `assets.bodies` and their REST mirrors, `shard_spawn_creatures.art` filled and rendered (§8.1, §12.1). **787 rows in one 734 ms page; 455 types resolved at ~190 ms per 100 on the Core thread; zero mobiles leaked.** `UOFIDDLER.md` deleted, two phases early | servuo-plugins, **link**, module-uo |
|
||
| 4 | **The UOP animation decoder (§4.3)** — the **eight** player bodies the legacy path cannot reach (§4.8 moved the elf ghosts across), the never-sweep rule already enforced in code by phase 1's `ResolveAnimation` | servuo-plugins |
|
||
| 5 | Item statics and land on demand, hued keys, the cache | servuo-plugins, module-uo |
|
||
| 6 | Deep animation keys (`body/<id>/a<n>/f<n>`) for the future project, plus the bulk-fill switch | servuo-plugins, module-uo |
|
||
| 7 | The atlas over the sidecar (§10); shared-filesystem requirement retired | module-uo |
|
||
| 8 | Admin surface, Import/Update, approve/reject, activity log | module-uo |
|
||
| 9 | Docs pass across five repos; live walk on the real rig | docs |
|
||
|
||
Phase 0 exists because §4 chose to call code that can take the shard down if it is wrong, and the
|
||
honest way to hold that choice is to try to break it on purpose — in the real host process, against
|
||
a client that has been patched — *before* building eight phases on top of it. The probes behind
|
||
§1.1 were run from PowerShell against a stock client; neither of those is the environment this will
|
||
actually run in.
|
||
|
||
Phase 4 sits **after** the catalogue rather than inside it on purpose. The catalogue is useful with
|
||
783 of its 787 bodies, the UOP reader is the one piece of genuinely new format work in this
|
||
protocol, and putting it on the critical path would hold up every website-side phase behind it. Its
|
||
acceptance test is not "it decodes" — it is that a gargoyle looks like a gargoyle, checked by eye,
|
||
because §4.3's whole point is that this failure mode produces confident, wrong pictures.
|
||
|
||
---
|
||
|
||
## 17. Decisions
|
||
|
||
Every item here is settled. Each is recorded because it changes numbers or obligations elsewhere
|
||
in the document.
|
||
|
||
1. **§4: settled 2026-09-10 — call ServUO's vendored `Ultima`,** with one exception added the same
|
||
day: **§4.3's UOP animation decoder**, scoped to the player bodies the legacy path cannot reach
|
||
and to nothing else (phase 4). The crash is confined to the `hasExtra: true` branch that only
|
||
`Gumps` reaches, and nothing here calls `Gumps`. **Phase 0 confirmed both halves in the real
|
||
host process** — nothing faulted on a path we call, and one `GetGump(2)` killed the shard
|
||
outright (§4.7).
|
||
2. **§4.4: settled 2026-09-10 — libgdiplus is a stated requirement** on Linux shard hosts, with
|
||
all three answers taken rather than one: it goes in `SHARD_PREREQS.md`, the installer's `doctor`
|
||
checks it, and its absence degrades to a named `NO_IMAGING` status instead of an error. Windows
|
||
hosts are unaffected.
|
||
3. **§5.1/§5.2: direction — settled 2026-09-10.** Player character bodies use index 0, everything
|
||
else index 1, direction is not in the key, and the player-body set is enumerated from
|
||
`Race.AllRaces` rather than hardcoded. Recorded here because it changes every count in §11.
|
||
Phase 0 reproduced the twelve stock ids and the six that do not decode, exactly.
|
||
4. **§13: the default audience — settled 2026-09-10. An asset inherits the audience of the page
|
||
that uses it.** A bestiary that is already anonymous keeps anonymous pictures; a staff-only
|
||
screen's art is staff-only. The operator can still set the policy explicitly, and the
|
||
`requireFeature` gate with its 404-not-403 behaviour is unchanged. The alternative — one flat
|
||
audience for all asset serving — was rejected because it necessarily disagrees with some page
|
||
that uses it, in one direction or the other, and the disagreement is silent either way.
|
||
5. **§4.5: the response to a malformed record — settled 2026-09-10, and now measured. Validate
|
||
before calling.** The extractor judges an index entry, and for statics the record behind it,
|
||
before handing the id to `Ultima`; a record that fails is reported absent and never decoded.
|
||
Chosen ahead of phase 0 over two alternatives — extracting in a child process (much stronger
|
||
containment, a much larger change to §2 and phase 1) and reversing §4 to write our own decoders
|
||
(weeks, per §2.1). Phase 0 then found the shape that settles it: the dangerous failure is not a
|
||
crash a child process would contain, it is a **wrong picture** that no containment strategy
|
||
would have caught, on **22,102 ids of a stock client**. See §4.5 for the checks and the
|
||
false-refusal measurement that says the boundary is in the right place.
|
||
6. **§3.2/§3.3/§3.4: the transport's three numbers and one shape — settled 2026-09-10, phase 1.**
|
||
A **512 KiB** batch budget under a **1 MiB** inbound line cap, with the factor of two load-bearing
|
||
(a page always admits its first item, so it may overshoot by one). Flow control is enforced
|
||
**on the shard**, as a single slot answering `bridge.busy`, rather than serialised in the
|
||
sidecar or left to the website as a convention — the bound belongs where the memory is. And one
|
||
**paging envelope** (`more` / `cursor` / `cut`) is defined now, with `assets.sources` as its
|
||
first user, rather than left for whichever family pages first to invent.
|
||
7. **§9: the cliloc pipeline's four shapes — settled 2026-09-10, phase 2.** All four were put to
|
||
the org lead before any of it was written, and two of them departed from the recommendation:
|
||
|
||
- **The wire carries rows, not bytes.** `[{n, f, t}]` inside the paging envelope, rather than a
|
||
reconstituted plain-binary file in base64. The container decompresses into *records*; the
|
||
plain layout is a file shape that would have had to be synthesised on the shard and re-parsed
|
||
on the website, at a 33% base64 premium, to reach the same place.
|
||
- **The shard omits blank entries** — ~56,000 of 123,490 — because the website discards them at
|
||
import anyway. The transfer halves and nothing observable changes.
|
||
- **The bridge always wins; the file upload is deprecated** (org lead, departing from the
|
||
proposed `auto`/`bridge`/`file` setting). There is no version of "which source?" an operator
|
||
benefits from answering, so there is no setting to answer it with. A file remains the source
|
||
only where there is no shard link, plus a one-off explicit path.
|
||
- **Import is admin-triggered on the bridge** (org lead, departing from the proposed boot
|
||
refresh). Boot does not call the shard at all — see §9.2 for why that is the right trade and
|
||
what it costs.
|
||
|
||
8. **§4.8: the catalogue is 787 bodies, not 1,144 — measured 2026-09-10, phase 1.** Recorded here
|
||
because it changes §11's sizing, phase 3's scope and phase 4's, and because of *how* it was
|
||
found: the animation path has §4.5's shared-buffer defect too, and 357 ids with no art were
|
||
returning the previously-decoded body's bitmap. The elf ghosts moved from "decodes" to "no art"
|
||
in §5.2 for the same reason, taking phase 4's set from six player bodies to eight.
|