Files
docs/link/v8.md
wtclaude 44039e83d5 docs(link): call ServUO's decoders — the crash is on a branch we never take
Decision: no decoders are reimplemented. Extraction goes through ServUO's own
vendored Ultima, which Scripts.csproj already references, so the art half of
this protocol is plumbing and the Mythic cliloc reader (§9) becomes the only
decoder Protocol 8 writes rather than calls.

What makes that safe rather than merely cheap is a distinction §1.1 did not
draw. All three decoders build a FileIndex, but only Gumps passes
hasExtra: true — and FileIndex.cs's own comment says that branch exists FOR
gumpartlegacy.uop, the one UOP layout with an extra field. Art passes
hasExtra: false and probed 49,150 statics plus 16,384 land tiles with zero
faults; Animations touches no UOP at all and probed 1,144 bodies clean. The
access violation is a bug on a branch exactly one decoder reaches, and that
decoder was already out of scope. So "nothing calls Ultima.Gumps" is now a
safety rule, and adding gump art later means fixing that path first.

§4.2 records the three costs this accepts: six of twelve stock player bodies
have no art (UOP-only, not reachable by calling the existing code differently),
System.Drawing stays in the decode path, and we inherit whatever Ultima a shard
vendors — EXTRACTOR_VERSION already covers the last one.

Phase 0 changes shape with it. It was going to prove new decoders byte-identical;
it now tries to BREAK the vendored ones on purpose, from inside a running ServUO
against a deliberately patched client, because the probes behind §1.1 ran in
PowerShell against a stock client and neither is the real environment.

§17 is down to one real question: libgdiplus on Linux/Mono shards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-10 01:11:13 -05:00

578 lines
31 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Protocol 8 — Client assets over the bridge
**Status:** Design of record. Approved in principle 2026-09-09 (architecture, asset scope,
built-in cliloc decoder, atlas cleanup); §17 lists what is still open.
**Supersedes the manual half of:** [`../website/UOFIDDLER.md`](../website/UOFIDDLER.md),
[`../website/CLILOCS.md`](../website/CLILOCS.md) §Converting,
[`../website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md) §Artwork and §Configuring the tree.
Two features on this platform read data that only exists inside a UO client, and today both reach
the site by hand: the operator installs UOFiddler, converts `Cliloc.enu` on their own desktop,
exports sprites one at a time from a GUI, hand-writes a slug→filename JSON map, and copies the
result to the server. A third — the spawn atlas — avoids UOFiddler but pays a different price: the
**website** must be able to read the shard's ServUO tree directly, over a bind mount or a shared
volume.
This protocol deletes all three arrangements. The shard already has everything, and the bridge
already goes to the website.
---
## 1. The premise, which turns out to be free
**A ServUO shard cannot boot without a UO client installation.** It reads maps, statics, tiledata
and multis out of `.mul`/`.uop` files, and `Config/DataPath.cfg` is where an operator declares
where those live — *required* on Linux, auto-detected from the registry on Windows. At runtime the
resolved directories sit in `Server.Core.DataDirectories`, a public static the plugin can read on
any shard, with no new configuration and nothing for an operator to set up.
So the files the operator has been converting on their desktop are already on the shard host, in a
directory the shard already knows the path of, in a process the bridge already runs inside.
Everything below follows from that.
### 1.1 What was measured, not assumed
Against this machine's ServUO 57.4 tree (`C:\Users\colby\Desktop\ServUO`) and client
(`D:\Games\Electronic Arts\Ultima Online Classic`, 3.5 GB), loading **ServUO's own
`Ultima.dll`** — the assembly `overlay/Scripts/Scripts.csproj:39` already carries a
`<ProjectReference>` to:
| Call | Result |
|---|---|
| `Art.GetStatic(0…16383)` | 16,384 decoded, 0 errors |
| `Art.GetStatic(16384…65535)` | 32,766 decoded, 1 empty, 16,385 clean out-of-range errors |
| `Art.GetLand(0…16383)` | 16,384 decoded, 0 errors |
| `Animations.GetAnimation(0…2047, 0, 1)` | **1,144** bodies with a decodable first frame, 904 empty, 0 errors |
| `Hues.GetHue(33)` | loads |
| `Bitmap.Save(…, Png)` | 852-byte PNG from one creature frame |
| `Gumps.GetGump(2)` | **hard crash**`AccessViolationException`, process exit `0xC0000005` |
| `new StringList("enu", "Cliloc.enu")` | throws — `Non-negative number required` |
Two of those rows are load-bearing and are dealt with in §4 and §9. The rest say the same thing:
**most of the extraction this protocol needs is already implemented, already compiled, and already
referenced by the plugin's own build.**
Depth, for §11's sizing: body 400 (human male) has **35 actions × 5 directions = 1,050 frames**.
One body. §5.1 cuts that by exactly 5×.
---
## 2. Architecture: the shard extracts, the sidecar forwards, the website decides
```
UO client files (operator's own, on the shard host)
│ read by the plugin, off the Core thread
ServUO shard (servuo-plugins/) ← decodes; resolves body ids; hashes
│ loopback JSON, request/reply, one batch outstanding at a time
uo-link sidecar (link/) ← forwards bytes; decides nothing
│ REST, bearer-token auth, X-UOLink-Version: 8
website (module-uo/) ← stores, names, gates, serves
```
This is deliberately the *only* arrangement that keeps
[the bridge's standing rules](PLAN.md) intact:
- **The sidecar stays a dumb forwarder.** It moves opaque assets and decides nothing about them —
no audience, no projection, no capability advertisement. Putting the decoders in Rust would have
meant the sidecar deciding what an asset *is*, on top of re-deriving in Rust what is already
compiled next door in C#.
- **Access control stays on the website**, which has the auth machinery and the admin forms.
- **The shard is still never network-reachable.** Nothing here opens a port; the plugin answers
requests on the connection it already dialled out on.
### 2.1 Why not the sidecar, and why not the operator's desktop
A Rust extractor in the sidecar would need ports of: the Mythic cliloc decompressor, `FileIndex`
(including UOP), the ARGB1555 run-length frame decoder, `Body.def`/`Bodyconv.def` translation,
`Hues.mul`, and a PNG encoder — weeks of work to re-derive what §1.1 shows already runs. It also
cannot do §8: resolving a creature slug to a body id requires being inside ServUO.
Automating on the operator's desktop (shipping the converter with the installer) removes UOFiddler
but keeps a manual step and still cannot do §8. It was considered and rejected.
---
## 3. The transport, and the three traps in it
### 3.1 Assets go over the request/reply path, never the event path
`link/sidecar/src/app.rs:122` persists **every** non-`pong` event into the SQLite store *and*
broadcasts it to every WebSocket subscriber. An asset stream on that path would grow the sidecar's
store without bound and fan megabytes out to every connected client, forever.
`rpc.rs`'s `try_route` consumes a correlated reply and `continue`s **before** either of those
happens. So an asset batch is a reply, not an event. This is not a new mechanism — it is the one
`char.request`, `account.roster` and `vendor.snapshot` already use.
### 3.2 One batch outstanding, always
`BridgeLink.Emit()` enqueues onto a **bounded drop-oldest** queue (`Bridge.QueueCap`, default
10,000). It counts **lines, not bytes** — a design that is correct for live events and dangerous
for bulk transfer, because 10,000 queued 200 KB replies is 2 GB of shard memory.
The rule that makes this safe is flow control, not a bigger queue: **the website requests batch
*n+1* only after batch *n* has arrived.** Queue depth stays at approximately one. A dropped or
lost reply simply times out and the batch is re-requested, which is safe because reading a client
file is idempotent and has no world side effects.
### 3.3 The size ceilings are already fixed, and one of them is missing
| Limit | Value | Where |
|---|---|---|
| Sidecar waits for a shard reply | **10 s** | `rpc.rs` `REPLY_TIMEOUT` |
| Website waits for the sidecar | **12 s** | `module-uo/server/utils/uoLinkClient.js` `TIMEOUT_MS` |
| Sidecar → shard line | 1 MiB | `BridgeLink.cs:283` |
| **Shard → sidecar line** | **none** | `shard.rs` uses `read_line` unbounded |
The first two bound a batch: it must decode, encode, serialise and cross the wire inside ten
seconds. The last is a gap this protocol must close — an unbounded `read_line` facing a component
that is now deliberately sending large lines is a memory-exhaustion shape we would be inventing
ourselves. **Protocol 8 adds an explicit inbound line cap to the sidecar**, set above the largest
legal batch and rejecting rather than buffering past it.
Batches are therefore sized by bytes, not by count, with the emitter cutting a batch short when it
would exceed the cap. Base64 costs 33%; the budget must be stated in encoded bytes.
---
## 4. The decoders are ServUO's own — decided, and the crash is narrower than it looked
**We call ServUO's vendored `Ultima` (decided 2026-09-10).** No decoders are reimplemented.
`overlay/Scripts/Scripts.csproj:39` already references the project, so the art half of this protocol
costs plumbing rather than pixel code, and only §9's cliloc decompressor is written from scratch.
The reason that is safe, rather than merely cheap, is a distinction §1.1 did not draw at first.
### 4.1 The crash lives on one code path, and nothing we call uses it
`Gumps.GetGump(2)` does not fail — it **corrupts the process**: `AccessViolationException`, exit
`0xC0000005`. That is a corrupted-state exception, uncatchable by an ordinary `try/catch` on .NET
Framework 4.8, so in-process on a live shard it is a crash with players on it. That much is
alarming, and on its own it looked like an argument against using this library at all.
It is not, because of how the three decoders construct their `FileIndex`:
| Decoder | UOP file | `hasExtra` | Probed |
|---|---|---|---|
| `Art` | `artLegacyMUL.uop` | **false** | 49,150 statics + 16,384 land tiles, **0 faults** |
| `Animations` | *none — legacy `anim*.mul` only* | — | 1,144 bodies, **0 faults** |
| `Gumps` | `gumpartLegacyMUL.uop` | **true** | **faults on the second id** |
`FileIndex.cs`'s own comment says the extra-field handling exists *for* `gumpartlegacy.uop` — it is
the one UOP layout carrying an extra field, and `hasExtra: true` is the branch written to cope with
it. **Gumps is the only caller that sets it.** So the fault is not a general fragility in this
library's `unsafe` code; it is a bug on a branch that exactly one decoder reaches, and that decoder
is already out of scope (§11).
The rule this turns into is a safety rule, not a preference: **nothing in this protocol calls
`Ultima.Gumps`.** Adding gump art later means fixing or replacing that path first, deliberately,
not discovering it in production.
### 4.2 What the decision accepts
Three costs come with it, all known and none of them blocking:
1. **Six of the twelve stock player-character bodies have no art** — both human ghosts and every
gargoyle body (§5.2). `Animations` never reads `AnimationFrame*.uop`, and UOP animation is a
different container with its own `AnimationSequence.uop`, not a mirror of the mul layout, so
this is not reachable by calling the existing code differently. Those bodies degrade to no
image, which is the same state every creature is in today.
2. **`System.Drawing` is a hard dependency, in the decode and not just the encode.** `Frame`
writes ARGB1555 straight through a `LockBits` pointer, so a Linux/Mono shard needs libgdiplus
to read a sprite at all. See §17.
3. **We inherit whatever `Ultima` a given ServUO vendors**, which can change under a shard upgrade.
`EXTRACTOR_VERSION` (§7) is the mitigation: it already counts as drift, so a shard whose library
changed re-derives on the next import.
The residual risk that remains is a patched or custom client tripping an out-of-bounds read on a
path we *do* call. §16's phase 0 is where that gets exercised rather than assumed.
---
## 5. Addressing: one key for every asset
Every asset the bridge can serve is named by a single string key, and the key is the cache key,
the hash key, the filename stem and the manifest row id:
```
static/3922 one item graphic
static/3922/h33 the same graphic, hue 33 applied
land/3 one land tile
body/34/a0 creature body 34, action 0, first frame
body/400/a0/f0..f9 human male, action 0, all ten frames
cliloc/enu the whole converted string table (not an image)
tree/Spawns/Trammel.xml a ServUO tree file (§10)
```
Three properties this shape buys:
- **Hue is part of the key, not a transform.** `itemId` and `hue` are already on the wire together
(`BridgeMarket.cs:582`, `BridgeProfile.cs:314`), so a marketplace listing already knows the exact
key for its own picture. Applying hues website-side would mean shipping `Hues.mul` semantics into
Node for no gain.
- **Depth is expressible without being mandatory.** `body/400/a0` and `body/400/a0/f0..f9` are the
same addressing scheme at two depths, which is what lets §11 bulk-import thumbnails and fetch full
animations on demand without a second protocol.
- **Nothing in the key is client-version-specific**, so a client patch changes an asset's *bytes*,
not its name — which is what makes §7's delta work.
### 5.1 There is no direction segment, because only one direction is wanted
Bodies are stored in **five** directions and the client mirrors three of them to reach eight. Only
one is needed here, so **direction is fixed by the extractor and is not part of the key**. Leaving
it in would advertise a choice nobody is going to vary and would five-fold every count in §11 for
nothing.
**Which one depends on whether the body is a player character:**
| Body | Direction | Why |
|---|---|---|
| A player character body | **0** — head-on, facing the viewer | A character is a portrait; it should look at you |
| Everything else | **1** — front three-quarter | The view that actually reads as a creature (see the caveat below) |
Which index is which was determined by **rendering all five** for a human, a wolf and a dragon
rather than from a table, because the answer is not obvious and the small-thumbnail version of the
same test suggested the exact opposite:
| Index | View |
|---|---|
| **0** | **Head-on, facing the viewer** — face, chest and front legs visible |
| 1 | Front three-quarter |
| 2 | Full side profile |
| 3 | Rear three-quarter |
| 4 | Directly away — back of the head, and a quadruped's tail toward the camera |
The caveat the render made obvious is what produced the split: **index 0 is the least legible view
for four-legged and long-bodied creatures.** A wolf seen head-on is a dark blob; the same wolf at
index 1 is unmistakably a wolf, which is also why UOFiddler's own thumbnail list picks that view. A
humanoid has no such problem — it reads fine head-on, and head-on is what a character portrait
wants.
Both indices stay **configuration values** (defaulting to 0 and 1), so changing the catalogue's mind
later is a setting and a re-import, not a protocol change.
### 5.2 "Player character body" is asked of the shard, never hardcoded
`Server.Race.AllRaces` gives every registered race, and each carries `MaleBody`, `FemaleBody`,
`MaleGhostBody` and `FemaleGhostBody`. The plugin enumerates those four ids per race and that set —
nothing else — takes index 0. On stock ServUO 57.4 that is twelve ids:
| Race | Male | Female | Male ghost | Female ghost |
|---|---|---|---|---|
| Human | 400 | 401 | 402 | 403 |
| Elf | 605 | 606 | 607 | 608 |
| Gargoyle | 666 | 667 | **695** | **694** |
This is the §8 argument again in miniature: only code inside ServUO can answer it, and asking is
the only thing that works on a shard with a custom race. Two details make the case that a
hardcoded list would have been wrong — `RaceDefinitions.cs` passes the gargoyle's ghost bodies in
the **opposite order** to the other two races (695 male, 694 female), and a shard that calls
`RegisterRace` adds ids no table of ours would contain.
**Half of that set does not decode with ServUO's vendored library.** Measured:
| Decodes | Does not |
|---|---|
| Human male/female (400, 401) | **Human ghosts (402, 403)** |
| Elf male/female (605, 606) | **Every gargoyle body (666, 667, 694, 695)** |
| Elf ghosts (607, 608) | |
Six of twelve, including a whole playable race — and §4 accepts that rather than reimplementing
`Animations` to reach `AnimationFrame*.uop`. Those bodies render without an image, exactly as every
creature does today.
Two things follow for the build. The catalogue must **not** treat a missing player body as an
error — it is the expected answer for half the set, and a status screen that flags six failures on
every import will teach an operator to ignore it. And `shard_spawn_creatures.art` staying NULL has
to remain a first-class state everywhere it is consumed, which it already is.
---
## 6. The manifest, and what the two buttons actually do
Two stages, which is where **Import** and **Update** come from.
**Stage 1 — the source gate.** The shard reports a manifest of the client files themselves:
size, mtime and content hash of `Cliloc.enu`, `anim*.idx`/`anim*.mul`, `art.mul`/`artidx.mul`,
`Body.def`, `Bodyconv.def`, `Hues.mul`. Unchanged since the last import, and nothing else happens.
This is the same hash gate the spawn atlas and the cliloc table already use, and for the same
reason: the normal case is a restart that changed nothing, and it must cost nothing.
`anim.mul` is 195 MB and `art.mul` is 148 MB, so the gate is **(size, mtime) first, content hash
only when those differ** — a full hash of 343 MB on every status poll would make the admin panel
feel broken.
**Stage 2 — the asset manifest.** For the working set (§11), the shard streams
`[{ key, sha256, bytes }]` — no pixels. The website diffs that against what it holds and requests
**only the keys whose hash changed**.
- **Update** = stage 1, then stage 2, then fetch the diff.
- **Import** = the same path with the diff skipped and every key fetched.
- **A key that has vanished** from the manifest is staged for review, never applied silently —
the same rule, and the same reasoning, as a vanished cliloc source or a disappearing atlas
facet. An unmounted volume and a deliberate client downgrade look identical from here.
Clilocs are the exception and stay a **whole-table replace** whenever the file hash changes:
the measured cost is 663 ms for 67,496 rows, so per-entry deltas would be complexity bought for
nothing.
---
## 7. The parser version applies here too
`spawnAtlasSource.js` carries `PARSER_VERSION` (currently 5) and the cliloc source carries its own,
both counted as drift so that a corrected parse reaches an install whose files never change. The
asset pipeline inherits the rule and needs it more, not less: a fixed hue application or a
corrected frame offset changes the bytes we derive from files that are byte-identical.
**`EXTRACTOR_VERSION` lives in the plugin**, because the plugin is what derives the bytes, and it
is folded into stage 1's gate. Bumping it makes every asset drift, which is correct.
---
## 8. Body ids: the part only the shard can do
The atlas knows creatures by **slug**, derived from type names in `Spawns/*.xml`. The client knows
them by **body id**. Nothing in the ServUO tree declares the mapping as data — today an operator
bridges it by grepping `Scripts/Mobiles/Normal/<Name>.cs` for `Body =`, which appears variously as
a decimal, as hex (`0xD1`), as `Utility.RandomList(35, 36)`, and as an `m_IDs[]` table.
Inside ServUO the problem does not exist. `BridgeWorld.cs:350` already does exactly the required
thing for a different feature:
```csharp
var type = ScriptCompiler.FindTypeByName(name, true);
var creature = Activator.CreateInstance(type) as BaseCreature;
```
Construct, read `creature.Body.BodyID`, `Delete()`. Authoritative, no source parsing, and correct
for custom creatures a grep would never find.
**This pass must run on the Core thread** — it constructs and deletes mobiles, which is world
mutation — while the decode in §4 must run **off** it. That split is the one genuinely new
threading shape in this protocol, and it is why slug→body resolution is its own request kind with
its own (small) batch size rather than a step inside asset extraction.
Constructing arbitrary creature types has side effects: constructors pack items, set skills, start
timers. The mitigations are per-type `try`/`catch`, immediate `Delete()`, small batches, and the
fact that the whole pass is admin-triggered rather than something that runs at boot.
---
## 9. The cliloc decompressor is ours now
Every modern client ships `Cliloc.*` in the Mythic compressed container — this machine's
`Cliloc.enu` is 4,989,921 bytes beginning `E8 79 67 8E`, high byte `0x8E`. ServUO's bundled
`Ultima.StringList` implements only the plain layout and throws on it (§1.1), which is also why
the shard's own `VendorSearch.GetItemName` is already inert.
**UOFiddler is released under the Beerware licence**, so porting its decompressor into our
GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C# — the **only** decoder
Protocol 8 writes rather than calls (§4) — and from that point:
- No operator installs UOFiddler.
- No operator runs `dotnet build` on a converter.
- No operator copies a 5 MB file to a server.
- `website/server/tools/cliloc-export/` is retired, and `UOFIDDLER.md` is deleted rather than
rewritten.
**What survives untouched is the `custom/` overlay mechanism.** Shard-added items carry cliloc ids
no client table has, and ServUO has no server-side notion of a custom cliloc — that is a real gap
in the *game*, not an artefact of the manual pipeline, and `CLILOCS.md`'s reasoning for it stands.
The base table now arrives over the bridge; overlays still come from a directory the site reads.
Measured on the live shard: 16,434 cliloc ids referenced by the script tree, 37 absent from stock.
---
## 10. The atlas stops needing a shared filesystem
Today `SPAWN_ATLAS.md` requires the **website** to read the ServUO tree — "same host, a bind mount,
or a shared volume". That is the one place the platform's own rule (only the sidecar bridges the
shard) is broken, and it is broken by the component that faces the internet.
The same transport closes it. `spawnAtlasSource.js` already labels every file it reads with a
portable key:
| Label | Count (stock 57.4) |
|---|---|
| `Data/Regions.xml` | 1 |
| `Data/Locations/*.xml` | 6 |
| `Spawns/*.xml` | 13, ~10.5 MB |
| `Config/ChampionSpawns.xml` | 1 |
| `Data/Decoration/**` | tree |
So the shard serves `tree/<label>` → bytes over the same batched request/reply path, and
`spawnAtlasSource.js` gains a second backend behind its existing interface: **filesystem** (today,
kept for same-host installs and for development) or **sidecar** (new, and the default once
configured).
**The parsers do not move.** `spawnAtlasParse.js` is pure, fs-free and CI-covered without a ServUO
tree, and every quirk it handles — the two respawn delay units, `:OBJ=` splitting, facet-name
reconciliation, the XmlSpawner directive stripping — stays exactly where it is. The shard sends
bytes; the website still decides what they mean. That is the same division as §2, and it keeps the
sidecar a forwarder here too.
`SERVUO_PATH` and the `spawn_atlas_servuo_path` setting remain, and select the filesystem backend.
---
## 11. What is bulk and what is on demand
The scope approved is creature art, item art, player models "and everything", against a future
project. §1.1's measurements make the sizing question concrete:
| Kind | Addressable | Bulk? |
|---|---|---|
| Item statics | **~49,150** | No — on demand, cached, keyed by `itemId` (+ hue) |
| Land tiles | **16,384** | No — on demand |
| Creature/player bodies, first frame | **1,144** | **Yes** — this is the catalogue |
| One body, every action, one direction | **210 frames** (body 400); 96210 measured across six bodies | No — on demand, per body |
| All bodies, every action, one direction | **~173,000 frames**, ~170 MB | No — but no longer unthinkable |
| The same at five directions | ~865,000 frames | Not built (§5.1) |
| Cliloc table | 123,490 entries → 67,496 rows | **Yes** — whole-table replace |
| ServUO tree files (§10) | ~21 files, ~10.6 MB | **Yes** |
**The working set is one thumbnail per body, plus the atlas's own creatures.** 1,144 sprites at
roughly a kilobyte each is under 2 MB — trivial to import, trivial to re-hash, and it is the set
that makes a bestiary, a marketplace listing and a character sheet render.
Everything deeper is the *same protocol at a deeper key* (§5), fetched on demand and cached. That
is what serves the future project without exporting 3.5 GB of someone else's copyrighted client
into a database: a viewer that wants body 400's full walk cycle asks for `body/400/a2/f0..f9` and
gets it, once, and it is cached from then on.
Because §5.1 dropped four of the five directions, a **complete** one-direction animation set for
every body is now ~173,000 frames rather than ~865,000 — around 170 MB. That is still not the
default and still not something to import before anything asks for it, but it has moved from
"never" to "a thing an operator could reasonably choose", and phase 5 should leave room for a
bulk-fill-everything switch rather than assuming on-demand is the only mode.
**Hued variants are on demand, always.** `static/3922/h33` is generated when something on the wire
actually carries hue 33. The cross product of 49,150 statics and ~3,000 hues is not a set anyone
enumerates.
**Gump art is out of scope for Protocol 8, and that is now a safety rule rather than a priority
call** — §4.1. It is the only decoder that reaches the `hasExtra: true` branch, and that branch
corrupts the process on the second id. Adding gump art later means fixing that path first,
deliberately; it is additive under the same key scheme (`gump/<id>`), which is the point of §5.
---
## 12. Where it lands on the website
Images are written by module-uo into the upload directory. `ctx.uploads`
(`{ upload, UPLOAD_DIR, MIME_EXT }`) is **already** exposed to modules and
[`MODULE_API.md:626`](../website/MODULE_API.md) already names its consumer as "atlas art import",
so no `MODULE_API_VERSION` bump is needed to store them.
- `shard_spawn_creatures.art` stops being NULL-by-default and starts being filled by the import.
- A new asset table carries `key`, `sha256`, `bytes`, `width`, `height`, `imported_at` — the
manifest side of §6, and what makes an Update a diff rather than a re-download.
- **The operator-supplied `spawnAtlas.art.json` map stays supported** and continues to win over an
imported asset. An operator who has drawn their own creature portraits must not have them
overwritten by a sprite rip on the next Update.
Licensing is unchanged and the reasoning is unchanged: these are the operator's own client files,
extracted on their own host, for their own shard. Nothing is committed, nothing ships in a repo,
and nothing is redistributed. What changes is only that the extraction stopped requiring a GUI on a
desktop.
---
## 13. Visibility
New surfaces over shard data are admin-toggleable with an operator-set audience, and this is no
exception. Asset serving is gated like every other shard read: a `requireFeature` gate, an
audience, and 404-not-403 when the feature is off, so a disabled feature does not advertise itself.
The default is the least surprising one: assets are as public as the page that uses them. A
bestiary that is already anonymous does not become staff-only because its pictures arrived over a
new pipe.
---
## 14. Routes and commands added
**Loopback (shard ↔ sidecar), all request/reply:**
| Command | Reply | Purpose |
|---|---|---|
| `assets.sources` | `assets.sources.ok` | Stage 1: client file manifest + `EXTRACTOR_VERSION` |
| `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes}]`, paged |
| `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged |
| `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) |
| `cliloc.table` | `cliloc.table.ok` | The converted table, paged |
| `tree.manifest` / `tree.fetch` | `.ok` | §10, the ServUO tree files |
**Sidecar REST** mirrors those one for one under `/assets/*`, `/cliloc`, `/tree/*`, carrying
`X-UOLink-Version: 8` and forwarding verbatim.
**Website admin** (`Admin → Shard`, admin-only): status, **Import**, **Update**, approve/reject for
a vanished key, and the existing path settings. Every action to the admin activity log, as
`shard.assets.*`.
---
## 15. Cross-repo obligations
`PROTOCOL_VERSION` goes **7 → 8** in `link/sidecar/src/main.rs`, and in the **same PR**
`servuo-plugins/overlay.toml` — the installer refuses to pair a sidecar and an overlay that
disagree, so a split bump means the next bundle silently fails to compose.
| Repo | Work |
|---|---|
| `servuo-plugins/` | Extraction over ServUO's own `Ultima` (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` |
| `link/` | Six command families forwarded, the REST surface, **the inbound line cap (§3.3)**, `PROTOCOL_VERSION` |
| `module-uo/` | Client calls, asset store, the atlas source backend (§10), cliloc ingest, admin surface |
| `website/` | None expected — `ctx.uploads` already suffices (§12) |
| `docs/` | This file; rewrite `CLILOCS.md` §Converting and `SPAWN_ATLAS.md` §Artwork + §Configuring; **delete `UOFIDDLER.md`** |
| `installer/` | None expected; the bundle pairing already enforces §15 |
| `android-app/` | Consumes images by URL; no parity gate expected until a screen shows one |
| `integration-kit/` | A chapter note only — this is UO-specific and teaches nothing about the module contract |
---
## 16. Phases
| # | Scope | Repos |
|---|---|---|
| 0 | Spike: the vendored decoders driven **from inside a running ServUO**, over a deliberately patched client — statics, land, bodies, and the Mythic cliloc against UOFiddler's output. What it is looking for is a fault on a path we call (§4.2) | servuo-plugins |
| 1 | The transport: `assets.sources`, flow control, the sidecar line cap, `EXTRACTOR_VERSION`, protocol bump | servuo-plugins, link |
| 2 | Clilocs end to end; retire the converter and `UOFIDDLER.md` §Part 1 | all |
| 3 | Body resolution (§8) + the 1,144-body catalogue; `shard_spawn_creatures.art` filled | servuo-plugins, module-uo |
| 4 | Item statics and land on demand, hued keys, the cache | servuo-plugins, module-uo |
| 5 | Deep animation keys (`body/<id>/a<n>/f<n>`) for the future project, plus the bulk-fill switch | servuo-plugins, module-uo |
| 6 | The atlas over the sidecar (§10); shared-filesystem requirement retired | module-uo |
| 7 | Admin surface, Import/Update, approve/reject, activity log | module-uo |
| 8 | Docs pass across five repos; live walk on the real rig | docs |
Phase 0 exists because §4 chose to call code that can take the shard down if it is wrong, and the
honest way to hold that choice is to try to break it on purpose — in the real host process, against
a client that has been patched — *before* building seven phases on top of it. The probes behind
§1.1 were run from PowerShell against a stock client; neither of those is the environment this will
actually run in.
---
## 17. Decisions still open
1. **§4: settled 2026-09-10 — call ServUO's vendored `Ultima`.** No decoders reimplemented. The
three accepted costs are in §4.2; the crash is confined to the `hasExtra: true` branch that only
`Gumps` reaches, and nothing here calls `Gumps`.
2. **§17.1's consequence: Linux/Mono shards.** §4.2 item 2 leaves `System.Drawing` in the decode
path, so a non-Windows shard needs **libgdiplus** installed or art extraction fails there. Three
ways to answer it, and this is the one real question left: document it as a prerequisite in
`SHARD_PREREQS.md` and let it fail loudly; have the installer's `doctor` detect and report it;
or fall back to no-art on that platform with a status the panel explains. Phase 0 should
establish which failure it actually is before we pick.
3. **§5.1/§5.2: direction — settled 2026-09-10.** Player character bodies use index 0, everything
else index 1, direction is not in the key, and the player-body set is enumerated from
`Race.AllRaces` rather than hardcoded. Nothing outstanding; recorded here because it changes
every count in §11.
4. **§13: the default audience** for asset serving — inheriting the using page's audience is
proposed; the operator sets the policy either way.