Merge pull request 'docs(link): phase 1 built the transport, and re-measured the catalogue' (#236) from docs/asset-bridge-p1 into main
Reviewed-on: #236
This commit is contained in:
@@ -57,9 +57,9 @@ The wire protocol is versioned so a mismatch is caught immediately instead of fa
|
|||||||
|
|
||||||
The current version is **6**. It is not released yet — it lives on `edge` and ships with the event system's cutover; the last released pairing is protocol **5**, sidecar **v2.1.0** + overlay **v1.1.0**, resolved as bundle **2026.09.01**, never as "latest of each".
|
The current version is **6**. It is not released yet — it lives on `edge` and ships with the event system's cutover; the last released pairing is protocol **5**, sidecar **v2.1.0** + overlay **v1.1.0**, resolved as bundle **2026.09.01**, never as "latest of each".
|
||||||
|
|
||||||
- Every response carries an **`X-UOLink-Version: 6`** header.
|
- Every response carries an **`X-UOLink-Version: 8`** header.
|
||||||
- `GET /health` and the WebSocket `ws.hello` frame include `"protocol": 6`.
|
- `GET /health` and the WebSocket `ws.hello` frame include `"protocol": 6`.
|
||||||
- **Optionally**, send `X-UOLink-Version: 6` on your requests. If it disagrees with the sidecar, the request is rejected **409 Conflict**:
|
- **Optionally**, send `X-UOLink-Version: 8` on your requests. If it disagrees with the sidecar, the request is rejected **409 Conflict**:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{ "error": "protocol version mismatch", "sidecar_protocol": 6, "client_protocol": "5" }
|
{ "error": "protocol version mismatch", "sidecar_protocol": 6, "client_protocol": "5" }
|
||||||
@@ -1277,7 +1277,7 @@ so a retry loop cannot quietly swallow a mismatched deployment.
|
|||||||
A typical character page:
|
A typical character page:
|
||||||
|
|
||||||
```js
|
```js
|
||||||
const H = { "Authorization": `Bearer ${TOKEN}`, "X-UOLink-Version": "6" };
|
const H = { "Authorization": `Bearer ${TOKEN}`, "X-UOLink-Version": "8" };
|
||||||
|
|
||||||
// 1. render the roster
|
// 1. render the roster
|
||||||
const roster = await fetch(`${BASE}/roster/${account}`, { headers: H }).then(r => r.json());
|
const roster = await fetch(`${BASE}/roster/${account}`, { headers: H }).then(r => r.json());
|
||||||
|
|||||||
@@ -69,3 +69,44 @@ World: Loading...
|
|||||||
## Unrelated, still open
|
## Unrelated, still open
|
||||||
|
|
||||||
`DllNotFoundException: zlibwapi64` crashed this shard once (`Crash 6-5-2026-22-38-3.log`) while sending a packed gump. `zlibwapi64.dll` is present in the repo root, so this is a working-directory / native-load-path problem. It will bite the bridge if the bridge ever triggers a gump send. Resolve before load testing.
|
`DllNotFoundException: zlibwapi64` crashed this shard once (`Crash 6-5-2026-22-38-3.log`) while sending a packed gump. `zlibwapi64.dll` is present in the repo root, so this is a working-directory / native-load-path problem. It will bite the bridge if the bridge ever triggers a gump send. Resolve before load testing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Host prerequisite: `libgdiplus` on Linux (Protocol 8)
|
||||||
|
|
||||||
|
Everything above is a repair to one shard's *scripts*. This one is different in kind: it is a
|
||||||
|
requirement on the **host**, it applies to every shard, and only to Linux ones.
|
||||||
|
|
||||||
|
The Asset Bridge ([`v8.md`](v8.md) §4.4) has the shard read art out of the operator's own UO client
|
||||||
|
files. ServUO targets `net48`, so on Linux it runs under Mono, and Mono's `System.Drawing` is a thin
|
||||||
|
layer over **libgdiplus** — which sits in the **decode** path and not merely the encode:
|
||||||
|
`Ultima.Frame` writes ARGB1555 through a `LockBits` pointer. Without that library a Linux shard
|
||||||
|
cannot read a sprite at all.
|
||||||
|
|
||||||
|
**Windows shard hosts need nothing.** `System.Drawing` ships with .NET Framework.
|
||||||
|
|
||||||
|
| Host | Get it with |
|
||||||
|
|---|---|
|
||||||
|
| Debian / Ubuntu | `sudo apt-get install libgdiplus` — in Debian since bullseye (6.0.4) and bookworm/trixie (6.1), and in Ubuntu universe |
|
||||||
|
| Fedora / RHEL | `sudo dnf install libgdiplus` (EPEL or the Mono repository) |
|
||||||
|
| Docker | `RUN apt-get update && apt-get install -y libgdiplus` in the shard image |
|
||||||
|
| Alpine, or a distro with no package | Build from source. This is the awkward case, and it is worth avoiding by choosing a Debian-based image |
|
||||||
|
|
||||||
|
Upstream is <https://github.com/mono/libgdiplus>. **That repository was archived in March 2025** and
|
||||||
|
is read-only; distributions still package and patch it, so installing it is a normal supported thing
|
||||||
|
to do today, but nobody is maintaining it upstream. It is the strongest long-term argument for
|
||||||
|
eventually moving extraction off `System.Drawing`.
|
||||||
|
|
||||||
|
**Its absence is not an error and never a crash.** The shard reports a named status on the source
|
||||||
|
gate — the first call any import makes — so an operator meets this while setting the shard up rather
|
||||||
|
than as an empty bestiary weeks later:
|
||||||
|
|
||||||
|
```
|
||||||
|
imaging: { ok: false, code: "NO_IMAGING",
|
||||||
|
reason: "This shard host cannot render images — Mono's System.Drawing needs
|
||||||
|
libgdiplus. Install it (apt-get install libgdiplus) and re-run the
|
||||||
|
import. Cliloc and atlas import are unaffected." }
|
||||||
|
```
|
||||||
|
|
||||||
|
Clilocs and the ServUO tree files are genuinely unaffected: neither touches a pixel. The installer's
|
||||||
|
`doctor` checks for this alongside its other host checks.
|
||||||
|
|||||||
252
link/v8.md
252
link/v8.md
@@ -51,7 +51,7 @@ Against this machine's ServUO 57.4 tree (`C:\Users\colby\Desktop\ServUO`) and cl
|
|||||||
| `Art.GetStatic(0…16383)` | 16,384 decoded, 0 errors |
|
| `Art.GetStatic(0…16383)` | 16,384 decoded, 0 errors |
|
||||||
| `Art.GetStatic(16384…65535)` | 32,766 decoded, 1 empty, 16,385 clean out-of-range errors |
|
| `Art.GetStatic(16384…65535)` | 32,766 decoded, 1 empty, 16,385 clean out-of-range errors |
|
||||||
| `Art.GetLand(0…16383)` | 16,384 decoded, 0 errors |
|
| `Art.GetLand(0…16383)` | 16,384 decoded, 0 errors |
|
||||||
| `Animations.GetAnimation(0…2047, 0, 1)` | **1,144** bodies with a decodable first frame, 904 empty, 0 errors |
|
| `Animations.GetAnimation(0…2047, 0, 1)` | **1,144** bodies with a decodable first frame, 904 empty, 0 errors — of which **357 are wrong pictures**; see §4.8 |
|
||||||
| `Hues.GetHue(33)` | loads |
|
| `Hues.GetHue(33)` | loads |
|
||||||
| `Bitmap.Save(…, Png)` | 852-byte PNG from one creature frame |
|
| `Bitmap.Save(…, Png)` | 852-byte PNG from one creature frame |
|
||||||
| `Gumps.GetGump(2)` | **hard crash** — `AccessViolationException`, process exit `0xC0000005` |
|
| `Gumps.GetGump(2)` | **hard crash** — `AccessViolationException`, process exit `0xC0000005` |
|
||||||
@@ -61,6 +61,12 @@ Two of those rows are load-bearing and are dealt with in §4 and §9. The rest s
|
|||||||
**most of the extraction this protocol needs is already implemented, already compiled, and already
|
**most of the extraction this protocol needs is already implemented, already compiled, and already
|
||||||
referenced by the plugin's own build.**
|
referenced by the plugin's own build.**
|
||||||
|
|
||||||
|
**Every "decoded" count in that table is an upper bound, not a measurement**, and phases 0 and 1
|
||||||
|
spent themselves establishing by how much. `Ultima` reports success for records that do not exist
|
||||||
|
(§4.5, §4.8), so the honest reading of the animation row is 787 bodies with art, 357 that return
|
||||||
|
the previously-decoded body's bitmap, and 904 that return nothing. The static and land rows are
|
||||||
|
overstated the same way. This is not a table to size anything from any more — §11 is.
|
||||||
|
|
||||||
Depth, for §11's sizing: body 400 (human male) has **35 actions × 5 directions = 1,050 frames**.
|
Depth, for §11's sizing: body 400 (human male) has **35 actions × 5 directions = 1,050 frames**.
|
||||||
One body. §5.1 cuts that by exactly 5×.
|
One body. §5.1 cuts that by exactly 5×.
|
||||||
|
|
||||||
@@ -127,6 +133,22 @@ The rule that makes this safe is flow control, not a bigger queue: **the website
|
|||||||
lost reply simply times out and the batch is re-requested, which is safe because reading a client
|
lost reply simply times out and the batch is re-requested, which is safe because reading a client
|
||||||
file is idempotent and has no world side effects.
|
file is idempotent and has no world side effects.
|
||||||
|
|
||||||
|
**Phase 1 made that a rule the shard enforces rather than one the website is trusted to follow.**
|
||||||
|
The asset plane has a **single slot**: a request arriving while one is in flight is answered
|
||||||
|
`bridge.busy` — which the sidecar already maps to 425 — and runs nothing. The bound belongs on the
|
||||||
|
side where the memory actually is; a documented convention would have held right up until the
|
||||||
|
first website bug, and its failure mode is the 2 GB above.
|
||||||
|
|
||||||
|
The alternative considered was serialising in the sidecar, so a second caller waits instead of
|
||||||
|
being refused. It was rejected because a waiter spends the website's own 12 s timeout doing
|
||||||
|
nothing, and because it leaves the shard itself unguarded against anything that is not that one
|
||||||
|
sidecar.
|
||||||
|
|
||||||
|
What this costs, and it is deliberate: **a status poll shares the slot with a batch**, so polling
|
||||||
|
during a long import is answered 425 until the batch lands. That is honest — this plane really
|
||||||
|
does do one thing at a time — and it is why the one genuinely long operation on it, hashing 343 MB
|
||||||
|
of client files, is explicitly *not* a job on this worker (§6).
|
||||||
|
|
||||||
### 3.3 The size ceilings are already fixed, and one of them is missing
|
### 3.3 The size ceilings are already fixed, and one of them is missing
|
||||||
|
|
||||||
| Limit | Value | Where |
|
| Limit | Value | Where |
|
||||||
@@ -134,7 +156,8 @@ file is idempotent and has no world side effects.
|
|||||||
| Sidecar waits for a shard reply | **10 s** | `rpc.rs` `REPLY_TIMEOUT` |
|
| Sidecar waits for a shard reply | **10 s** | `rpc.rs` `REPLY_TIMEOUT` |
|
||||||
| Website waits for the sidecar | **12 s** | `module-uo/server/utils/uoLinkClient.js` `TIMEOUT_MS` |
|
| Website waits for the sidecar | **12 s** | `module-uo/server/utils/uoLinkClient.js` `TIMEOUT_MS` |
|
||||||
| Sidecar → shard line | 1 MiB | `BridgeLink.cs:283` |
|
| Sidecar → shard line | 1 MiB | `BridgeLink.cs:283` |
|
||||||
| **Shard → sidecar line** | **none** | `shard.rs` uses `read_line` unbounded |
|
| **Shard → sidecar line** | **1 MiB** | `shard.rs` `MAX_INBOUND_LINE_BYTES` — **added in phase 1**; it was unbounded |
|
||||||
|
| **Batch budget** | **512 KiB encoded** | `Bridge.AssetBatchBytes` — **added in phase 1** |
|
||||||
|
|
||||||
The first two bound a batch: it must decode, encode, serialise and cross the wire inside ten
|
The first two bound a batch: it must decode, encode, serialise and cross the wire inside ten
|
||||||
seconds. The last is a gap this protocol must close — an unbounded `read_line` facing a component
|
seconds. The last is a gap this protocol must close — an unbounded `read_line` facing a component
|
||||||
@@ -145,6 +168,48 @@ legal batch and rejecting rather than buffering past it.
|
|||||||
Batches are therefore sized by bytes, not by count, with the emitter cutting a batch short when it
|
Batches are therefore sized by bytes, not by count, with the emitter cutting a batch short when it
|
||||||
would exceed the cap. Base64 costs 33%; the budget must be stated in encoded bytes.
|
would exceed the cap. Base64 costs 33%; the budget must be stated in encoded bytes.
|
||||||
|
|
||||||
|
**The two numbers, settled in phase 1: a 512 KiB batch budget under a 1 MiB line cap.** The cap is
|
||||||
|
symmetric with the one the shard has always applied to its own inbound lines, so both directions
|
||||||
|
of this link read the same. The factor of two between them is load-bearing rather than cautious: a
|
||||||
|
page always admits its **first** item even when that item alone exceeds the budget, because the
|
||||||
|
alternative is an oversized item being skipped for the budget on every page forever and its family
|
||||||
|
never making progress. The headroom is what makes that overshoot land on the wire instead of being
|
||||||
|
rejected by the cap.
|
||||||
|
|
||||||
|
An over-long line is **discarded and the connection kept**, which is what `BridgeLink.cs` has
|
||||||
|
always done in the other direction. Tearing the link down would take the live event feed with it
|
||||||
|
over a single malformed frame, and the reply that was lost simply times out and is re-requested —
|
||||||
|
safe, because reading a client file is idempotent.
|
||||||
|
|
||||||
|
### 3.4 One paging envelope, defined once
|
||||||
|
|
||||||
|
Five of the families in §14 page: clilocs (phase 2), the body catalogue (3), statics and land (5),
|
||||||
|
deep animation keys (6) and the ServUO tree files (7). Left to themselves that is five chances to
|
||||||
|
invent five slightly different shapes, and the website would have to learn each one — so phase 1
|
||||||
|
defines the envelope before the first family needs it, and `assets.sources` is its first user even
|
||||||
|
though it has nothing to page.
|
||||||
|
|
||||||
|
```
|
||||||
|
"files": [ … ], // the array, named by the family
|
||||||
|
"more": true, // ask again, with this cursor
|
||||||
|
"cursor": "s:4104", // opaque to everyone but the shard; absent when more is false
|
||||||
|
"cut": "budget" // budget | end | limit — WHY the page stopped
|
||||||
|
```
|
||||||
|
|
||||||
|
`cut` is the field that is easy to leave out and expensive not to have. "This page is short" has
|
||||||
|
three different meanings — the source ran out (`end`), the byte budget was spent (`budget`), or
|
||||||
|
the family stopped at a limit of its own (`limit`) — and **only the first means the import is
|
||||||
|
finished.** A website that had to infer completion from an item count would resume from the wrong
|
||||||
|
place the first time a page happened to land exactly on a boundary.
|
||||||
|
|
||||||
|
The cursor is deliberately **opaque and shard-defined**. The shard is the only side that knows how
|
||||||
|
its own walk is ordered, and a cursor the website could parse is a cursor the website would
|
||||||
|
eventually construct.
|
||||||
|
|
||||||
|
The budget is counted in **UTF-8 bytes**, not characters and not items. Characters would undercount
|
||||||
|
every non-ASCII byte in a cliloc row, and the ceiling this has to live inside — §3.3's line cap —
|
||||||
|
is measured in bytes.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 4. The decoders are ServUO's own — decided, and the crash is narrower than it looked
|
## 4. The decoders are ServUO's own — decided, and the crash is narrower than it looked
|
||||||
@@ -359,6 +424,87 @@ first, deliberately.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
### 4.8 What phase 1 measured: the animation path has the same defect, and it is worse
|
||||||
|
|
||||||
|
§4.5 ended by saying the animation path had no validator and that extending it was phase 1's work
|
||||||
|
rather than a phase 9 tidy-up. Phase 1 built it, ran it, and the reason that sentence was right is
|
||||||
|
larger than the verdata entry that prompted it.
|
||||||
|
|
||||||
|
**357 of the 1,144 "decodable" bodies are wrong pictures, on the stock client.** Their index entry
|
||||||
|
reads `length 0` — no record at all — and `GetAnimation` returns a real bitmap anyway, for the same
|
||||||
|
reason `LoadStatic` does: `m_StreamBuffer` is reused, only ever grown, and filled by a
|
||||||
|
`stream.Read` whose return value is discarded.
|
||||||
|
|
||||||
|
Measured directly, because a count could not tell:
|
||||||
|
|
||||||
|
| Decode body 320 (`lookup 22638982, length 0`) straight after… | What comes back |
|
||||||
|
|---|---|
|
||||||
|
| body 12, the dragon | the dragon's bitmap, 176×167, identical hash |
|
||||||
|
| body 34, the wolf | the wolf's dimensions, 35×34 |
|
||||||
|
| body 400, the human male | the human's bitmap, 27×63, identical hash |
|
||||||
|
|
||||||
|
That is not a near miss or a misaligned fragment. Body 320 has no art, and it renders **whichever
|
||||||
|
creature was decoded before it** — which means the picture a bestiary page got would depend on the
|
||||||
|
order the importer happened to walk the catalogue in.
|
||||||
|
|
||||||
|
So the working set is **787 bodies, not 1,144**:
|
||||||
|
|
||||||
|
| | Bodies 0–2047, direction 1, first frame |
|
||||||
|
|---|---|
|
||||||
|
| Real art | **787** |
|
||||||
|
| Wrong pictures (empty record, library returned a bitmap) | **357** |
|
||||||
|
| Absent, and the library agreed | 903 |
|
||||||
|
| `bodyconv` resolves nowhere, nothing swept (§4.3) | 1 |
|
||||||
|
| Refused by the record walk | **0** |
|
||||||
|
| Threw | 0 |
|
||||||
|
|
||||||
|
**That last-but-one row is the number that matters as much as the first.** The record-level
|
||||||
|
animation checks — palette, frame count, frame table, and every run header walked against both the
|
||||||
|
record's own length and the bitmap it locks — refused **nothing** across every real body on a stock
|
||||||
|
client. §4.5's rule holds: a checker that refuses real art is worse than no checker, and this one
|
||||||
|
does not.
|
||||||
|
|
||||||
|
#### The player bodies: four of twelve, not six
|
||||||
|
|
||||||
|
§5.2's table was built from the library's answer alone, and two of the six bodies it listed as
|
||||||
|
decoding do not have art:
|
||||||
|
|
||||||
|
| Body | Index entry | Library | Actually |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Human male/female (400, 401) | real | 24×64, 24×63 | **art** |
|
||||||
|
| Elf male/female (605, 606) | real | 24×64, 24×63 | **art** |
|
||||||
|
| **Elf ghosts (607, 608)** | `lookup 27221378, length 0` | 24×63 | **the previous body's picture** |
|
||||||
|
| Human ghosts (402, 403) | `lookup -1` | nothing | honestly absent |
|
||||||
|
| Gargoyle (666, 667, 694, 695) | `lookup -1` | nothing | honestly absent |
|
||||||
|
|
||||||
|
Confirmed the same way: body 607 decoded after the dragon is the dragon, after the wolf is the
|
||||||
|
wolf, after the human male is the human male. Its 24×63 was the elf *female*'s dimensions, because
|
||||||
|
606 is what the catalogue walk decoded immediately before it.
|
||||||
|
|
||||||
|
Two consequences, both of which change work elsewhere in this document:
|
||||||
|
|
||||||
|
- **§4.3's UOP decoder covers eight player bodies, not six.** The elf ghosts join the human ghosts
|
||||||
|
and the four gargoyle bodies. Phase 4's scope grows by two ids and its argument does not change.
|
||||||
|
- **The `lookup -1` / `length 0` distinction is the whole difference between an honest absence and
|
||||||
|
a wrong picture**, and it is not visible from outside the index. `artidx.mul` and the legacy
|
||||||
|
`anim*.idx` write `-1` for a record that is not there; an empty UOP slot, and evidently a
|
||||||
|
deliberately blanked legacy entry, is a zeroed struct. `FileIndex.Seek` rejects the first and
|
||||||
|
accepts the second, and the second is 357 creatures and two playable ghosts.
|
||||||
|
|
||||||
|
#### Why this could not have been found any other way
|
||||||
|
|
||||||
|
Phase 0 ran this exact sweep and reported "1,144 decoded, 904 empty, 0 faults", and every one of
|
||||||
|
those numbers is true. The library raised nothing, returned bitmaps of plausible sizes, and agreed
|
||||||
|
with itself. Nothing downstream of the decode — not a count, not an exception, not a hash of the
|
||||||
|
output, not a look at one picture in isolation — distinguishes body 320's dragon from body 12's.
|
||||||
|
The only things that did were **validating the index entry before the call** and **decoding the
|
||||||
|
same id twice after different neighbours**.
|
||||||
|
|
||||||
|
That is the same method note §4.3 ended on, and this is its second confirmed catch. Any time a UO
|
||||||
|
file lookup is addressed by index, a success count is evidence of nothing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## 5. Addressing: one key for every asset
|
## 5. Addressing: one key for every asset
|
||||||
|
|
||||||
Every asset the bridge can serve is named by a single string key, and the key is the cache key,
|
Every asset the bridge can serve is named by a single string key, and the key is the cache key,
|
||||||
@@ -439,21 +585,25 @@ hardcoded list would have been wrong — `RaceDefinitions.cs` passes the gargoyl
|
|||||||
the **opposite order** to the other two races (695 male, 694 female), and a shard that calls
|
the **opposite order** to the other two races (695 male, 694 female), and a shard that calls
|
||||||
`RegisterRace` adds ids no table of ours would contain.
|
`RegisterRace` adds ids no table of ours would contain.
|
||||||
|
|
||||||
**Half of that set does not decode with ServUO's vendored library.** Measured:
|
**Two thirds of that set has no art in ServUO's vendored library.** Measured — and re-measured in
|
||||||
|
phase 1, which moved two rows across (§4.8):
|
||||||
|
|
||||||
| Decodes | Does not |
|
| Has art | Does not |
|
||||||
|---|---|
|
|---|---|
|
||||||
| Human male/female (400, 401) | **Human ghosts (402, 403)** |
|
| Human male/female (400, 401) | **Human ghosts (402, 403)** — `lookup -1` |
|
||||||
| Elf male/female (605, 606) | **Every gargoyle body (666, 667, 694, 695)** |
|
| Elf male/female (605, 606) | **Elf ghosts (607, 608)** — `length 0`, and the library returns the previously-decoded body's bitmap |
|
||||||
| Elf ghosts (607, 608) | |
|
| | **Every gargoyle body (666, 667, 694, 695)** — `lookup -1` |
|
||||||
|
|
||||||
Six of twelve, including a whole playable race. These are UOP-only and **§4.3 adds a decoder for
|
**Four of twelve, not six.** The elf ghosts were counted as decoding for as long as the library's
|
||||||
them** — verified genuinely absent from the legacy files rather than merely mis-addressed, by
|
own answer was the measurement: their index entry has no record, and what came back was whatever
|
||||||
rendering what the other anim files hold at those indices and finding spiders.
|
the catalogue walk had decoded the moment before — the elf female, at her exact dimensions. All
|
||||||
|
eight are UOP-only and **§4.3 adds a decoder for them** — verified genuinely absent from the legacy
|
||||||
|
files rather than merely mis-addressed, by rendering what the other anim files hold at those
|
||||||
|
indices and finding spiders.
|
||||||
|
|
||||||
Until that phase lands, and on any client that lacks them entirely, the catalogue must **not** treat
|
Until that phase lands, and on any client that lacks them entirely, the catalogue must **not** treat
|
||||||
a missing player body as an error. It is the expected answer for half the set, and a status screen
|
a missing player body as an error. It is the expected answer for two thirds of the set, and a
|
||||||
that flags six failures on every import teaches an operator to ignore it.
|
status screen that flags eight failures on every import teaches an operator to ignore it.
|
||||||
`shard_spawn_creatures.art` staying NULL remains a first-class state everywhere it is consumed,
|
`shard_spawn_creatures.art` staying NULL remains a first-class state everywhere it is consumed,
|
||||||
which it already is.
|
which it already is.
|
||||||
|
|
||||||
@@ -473,6 +623,23 @@ reason: the normal case is a restart that changed nothing, and it must cost noth
|
|||||||
only when those differ** — a full hash of 343 MB on every status poll would make the admin panel
|
only when those differ** — a full hash of 343 MB on every status poll would make the admin panel
|
||||||
feel broken.
|
feel broken.
|
||||||
|
|
||||||
|
**Phase 1 found that the rule is not sufficient on its own, because of §3.3's other ceiling.** The
|
||||||
|
first time those hashes are needed there is nothing cached to compare against, so "hash only when
|
||||||
|
(size, mtime) differ" still means hashing 343 MB — inside a 10 s reply timeout it will not fit,
|
||||||
|
and the call would 504 rather than answer. So the shard's hashes are computed **off the request
|
||||||
|
path entirely**:
|
||||||
|
|
||||||
|
- `assets.sources` always answers immediately, with `size` and `mtime` for every file and
|
||||||
|
`sha256: null` for any file whose hash is not cached against exactly that (size, mtime).
|
||||||
|
- A file with a missing hash starts a background pass on its **own** thread — deliberately not a
|
||||||
|
job on §3.2's single-slot worker, which would answer every status poll `bridge.busy` for the
|
||||||
|
minutes the pass takes, at exactly the moment an operator is watching the panel.
|
||||||
|
- The reply carries `hashing` and `complete`, so the website knows to poll again rather than to
|
||||||
|
treat a null hash as a changed file.
|
||||||
|
|
||||||
|
The gate is unchanged; what changed is that "the normal case must cost nothing" now also means
|
||||||
|
"and the abnormal case must not time out".
|
||||||
|
|
||||||
**Stage 2 — the asset manifest.** For the working set (§11), the shard streams
|
**Stage 2 — the asset manifest.** For the working set (§11), the shard streams
|
||||||
`[{ key, sha256, bytes }]` — no pixels. The website diffs that against what it holds and requests
|
`[{ key, sha256, bytes }]` — no pixels. The website diffs that against what it holds and requests
|
||||||
**only the keys whose hash changed**.
|
**only the keys whose hash changed**.
|
||||||
@@ -596,24 +763,31 @@ project. §1.1's measurements make the sizing question concrete:
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Item statics | **~49,150** | No — on demand, cached, keyed by `itemId` (+ hue) |
|
| Item statics | **~49,150** | No — on demand, cached, keyed by `itemId` (+ hue) |
|
||||||
| Land tiles | **16,384** | No — on demand |
|
| Land tiles | **16,384** | No — on demand |
|
||||||
| Creature/player bodies, first frame | **1,144** | **Yes** — this is the catalogue |
|
| Creature/player bodies, first frame | **787** (§4.8 — *not* the 1,144 the library reports) | **Yes** — this is the catalogue |
|
||||||
| One body, every action, one direction | **210 frames** (body 400); 96–210 measured across six bodies | No — on demand, per body |
|
| One body, every action, one direction | **210 frames** (body 400); 96–210 measured across six bodies | No — on demand, per body |
|
||||||
| All bodies, every action, one direction | **~173,000 frames**, ~170 MB | No — but no longer unthinkable |
|
| All bodies, every action, one direction | **~119,000 frames**, ~117 MB | No — but no longer unthinkable |
|
||||||
| The same at five directions | ~865,000 frames | Not built (§5.1) |
|
| The same at five directions | ~865,000 frames | Not built (§5.1) |
|
||||||
| Cliloc table | 123,490 entries → 67,496 rows | **Yes** — whole-table replace |
|
| Cliloc table | 123,490 entries → 67,496 rows | **Yes** — whole-table replace |
|
||||||
| ServUO tree files (§10) | ~21 files, ~10.6 MB | **Yes** |
|
| ServUO tree files (§10) | ~21 files, ~10.6 MB | **Yes** |
|
||||||
|
|
||||||
**The working set is one thumbnail per body, plus the atlas's own creatures.** 1,144 sprites at
|
**The working set is one thumbnail per body, plus the atlas's own creatures.** 787 sprites at
|
||||||
roughly a kilobyte each is under 2 MB — trivial to import, trivial to re-hash, and it is the set
|
roughly a kilobyte each is under 1 MB — trivial to import, trivial to re-hash, and it is the set
|
||||||
that makes a bestiary, a marketplace listing and a character sheet render.
|
that makes a bestiary, a marketplace listing and a character sheet render.
|
||||||
|
|
||||||
|
Phase 1 took that count down from 1,144, and the 357 it removed are the important part: those are
|
||||||
|
ids with **no art** that the library returns a picture for (§4.8). Importing them would have
|
||||||
|
written 357 duplicate creature portraits into the site, each one showing whichever body the walk
|
||||||
|
happened to decode before it. The count went down; what the catalogue is worth went up.
|
||||||
|
|
||||||
Everything deeper is the *same protocol at a deeper key* (§5), fetched on demand and cached. That
|
Everything deeper is the *same protocol at a deeper key* (§5), fetched on demand and cached. That
|
||||||
is what serves the future project without exporting 3.5 GB of someone else's copyrighted client
|
is what serves the future project without exporting 3.5 GB of someone else's copyrighted client
|
||||||
into a database: a viewer that wants body 400's full walk cycle asks for `body/400/a2/f0..f9` and
|
into a database: a viewer that wants body 400's full walk cycle asks for `body/400/a2/f0..f9` and
|
||||||
gets it, once, and it is cached from then on.
|
gets it, once, and it is cached from then on.
|
||||||
|
|
||||||
Because §5.1 dropped four of the five directions, a **complete** one-direction animation set for
|
Because §5.1 dropped four of the five directions, a **complete** one-direction animation set for
|
||||||
every body is now ~173,000 frames rather than ~865,000 — around 170 MB. That is still not the
|
every body is now ~119,000 frames rather than ~865,000 — around 117 MB (~173,000 before §4.8 cut
|
||||||
|
the body count to the ones that have art; both figures carry the same measured ~151-frame average
|
||||||
|
per body). That is still not the
|
||||||
default and still not something to import before anything asks for it, but it has moved from
|
default and still not something to import before anything asks for it, but it has moved from
|
||||||
"never" to "a thing an operator could reasonably choose", and phase 6 should leave room for a
|
"never" to "a thing an operator could reasonably choose", and phase 6 should leave room for a
|
||||||
bulk-fill-everything switch rather than assuming on-demand is the only mode.
|
bulk-fill-everything switch rather than assuming on-demand is the only mode.
|
||||||
@@ -666,17 +840,23 @@ new pipe.
|
|||||||
|
|
||||||
**Loopback (shard ↔ sidecar), all request/reply:**
|
**Loopback (shard ↔ sidecar), all request/reply:**
|
||||||
|
|
||||||
| Command | Reply | Purpose |
|
| Command | Reply | Purpose | Built |
|
||||||
|---|---|---|
|
|---|---|---|---|
|
||||||
| `assets.sources` | `assets.sources.ok` | Stage 1: client file manifest + `EXTRACTOR_VERSION` |
|
| `assets.sources` | `assets.sources.ok` | Stage 1: client file manifest + `EXTRACTOR_VERSION` | **phase 1** |
|
||||||
| `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes}]`, paged |
|
| `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes}]`, paged | phase 3 |
|
||||||
| `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged |
|
| `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged | phase 3 |
|
||||||
| `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) |
|
| `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) | phase 3 |
|
||||||
| `cliloc.table` | `cliloc.table.ok` | The converted table, paged |
|
| `cliloc.table` | `cliloc.table.ok` | The converted table, paged | phase 2 |
|
||||||
| `tree.manifest` / `tree.fetch` | `.ok` | §10, the ServUO tree files |
|
| `tree.manifest` / `tree.fetch` | `.ok` | §10, the ServUO tree files | phase 7 |
|
||||||
|
|
||||||
|
Every one of them is refused outright when `Bridge.AssetsEnabled` is off, and every one of them
|
||||||
|
requires a `reqId` — a request without one is refused rather than answered, because an uncorrelated
|
||||||
|
reply is by definition an event, and §3.1 is the reason none of this may ever be one.
|
||||||
|
|
||||||
**Sidecar REST** mirrors those one for one under `/assets/*`, `/cliloc`, `/tree/*`, carrying
|
**Sidecar REST** mirrors those one for one under `/assets/*`, `/cliloc`, `/tree/*`, carrying
|
||||||
`X-UOLink-Version: 8` and forwarding verbatim.
|
`X-UOLink-Version: 8` and forwarding verbatim. `GET /assets/sources` exists as of phase 1; its
|
||||||
|
responder maps `bridge.busy` to **425** (flow control, and the ordinary answer mid-import rather
|
||||||
|
than a rare one) and a disabled plane to **403** (an operator's refusal, not a malformed request).
|
||||||
|
|
||||||
**Website admin** (`Admin → Shard`, admin-only): status, **Import**, **Update**, approve/reject for
|
**Website admin** (`Admin → Shard`, admin-only): status, **Import**, **Update**, approve/reject for
|
||||||
a vanished key, and the existing path settings. Every action to the admin activity log, as
|
a vanished key, and the existing path settings. Every action to the admin activity log, as
|
||||||
@@ -708,10 +888,10 @@ disagree, so a split bump means the next bundle silently fails to compose.
|
|||||||
| # | Scope | Repos |
|
| # | Scope | Repos |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| 0 | **DONE 2026-09-10.** Spike: the vendored decoders driven from inside a running ServUO, over a client broken 21 ways. §4 stands; the finding was 22,102 wrong pictures on a *stock* client, and the validator that answers them (§4.5-§4.7) | servuo-plugins |
|
| 0 | **DONE 2026-09-10.** Spike: the vendored decoders driven from inside a running ServUO, over a client broken 21 ways. §4 stands; the finding was 22,102 wrong pictures on a *stock* client, and the validator that answers them (§4.5-§4.7) | servuo-plugins |
|
||||||
| 1 | The transport: `assets.sources`, flow control, the sidecar line cap, `EXTRACTOR_VERSION`, protocol bump. **Plus §4.5's validator promoted into the overlay, extended to animations** — phase 0 left that half unbuilt and proved it is needed | servuo-plugins, link |
|
| 1 | **DONE 2026-09-10.** The transport: `assets.sources`, the single-slot gate (§3.2), the paging envelope (§3.4), the sidecar line cap (§3.3), `EXTRACTOR_VERSION`, `NO_IMAGING`, protocol 7→8. Plus §4.5's validator promoted into the overlay and extended to animations — which found **357 wrong pictures in the body catalogue on a stock client** and cut it from 1,144 to 787 (§4.8) | servuo-plugins, link |
|
||||||
| 2 | Clilocs end to end; retire the converter and `UOFIDDLER.md` §Part 1 | all |
|
| 2 | Clilocs end to end; retire the converter and `UOFIDDLER.md` §Part 1 | all |
|
||||||
| 3 | Body resolution (§8) + the 1,144-body catalogue; `shard_spawn_creatures.art` filled | servuo-plugins, module-uo |
|
| 3 | Body resolution (§8) + the **787**-body catalogue (§4.8); `shard_spawn_creatures.art` filled | servuo-plugins, module-uo |
|
||||||
| 4 | **The UOP animation decoder (§4.3)** — the six player bodies the legacy path cannot reach, and the never-sweep-file-types rule enforced in code | servuo-plugins |
|
| 4 | **The UOP animation decoder (§4.3)** — the **eight** player bodies the legacy path cannot reach (§4.8 moved the elf ghosts across), the never-sweep rule already enforced in code by phase 1's `ResolveAnimation` | servuo-plugins |
|
||||||
| 5 | Item statics and land on demand, hued keys, the cache | servuo-plugins, module-uo |
|
| 5 | Item statics and land on demand, hued keys, the cache | servuo-plugins, module-uo |
|
||||||
| 6 | Deep animation keys (`body/<id>/a<n>/f<n>`) for the future project, plus the bulk-fill switch | servuo-plugins, module-uo |
|
| 6 | Deep animation keys (`body/<id>/a<n>/f<n>`) for the future project, plus the bulk-fill switch | servuo-plugins, module-uo |
|
||||||
| 7 | The atlas over the sidecar (§10); shared-filesystem requirement retired | module-uo |
|
| 7 | The atlas over the sidecar (§10); shared-filesystem requirement retired | module-uo |
|
||||||
@@ -725,7 +905,7 @@ a client that has been patched — *before* building eight phases on top of it.
|
|||||||
actually run in.
|
actually run in.
|
||||||
|
|
||||||
Phase 4 sits **after** the catalogue rather than inside it on purpose. The catalogue is useful with
|
Phase 4 sits **after** the catalogue rather than inside it on purpose. The catalogue is useful with
|
||||||
1,138 of 1,144 bodies, the UOP reader is the one piece of genuinely new format work in this
|
783 of its 787 bodies, the UOP reader is the one piece of genuinely new format work in this
|
||||||
protocol, and putting it on the critical path would hold up every website-side phase behind it. Its
|
protocol, and putting it on the critical path would hold up every website-side phase behind it. Its
|
||||||
acceptance test is not "it decodes" — it is that a gargoyle looks like a gargoyle, checked by eye,
|
acceptance test is not "it decodes" — it is that a gargoyle looks like a gargoyle, checked by eye,
|
||||||
because §4.3's whole point is that this failure mode produces confident, wrong pictures.
|
because §4.3's whole point is that this failure mode produces confident, wrong pictures.
|
||||||
@@ -766,3 +946,15 @@ in the document.
|
|||||||
crash a child process would contain, it is a **wrong picture** that no containment strategy
|
crash a child process would contain, it is a **wrong picture** that no containment strategy
|
||||||
would have caught, on **22,102 ids of a stock client**. See §4.5 for the checks and the
|
would have caught, on **22,102 ids of a stock client**. See §4.5 for the checks and the
|
||||||
false-refusal measurement that says the boundary is in the right place.
|
false-refusal measurement that says the boundary is in the right place.
|
||||||
|
6. **§3.2/§3.3/§3.4: the transport's three numbers and one shape — settled 2026-09-10, phase 1.**
|
||||||
|
A **512 KiB** batch budget under a **1 MiB** inbound line cap, with the factor of two load-bearing
|
||||||
|
(a page always admits its first item, so it may overshoot by one). Flow control is enforced
|
||||||
|
**on the shard**, as a single slot answering `bridge.busy`, rather than serialised in the
|
||||||
|
sidecar or left to the website as a convention — the bound belongs where the memory is. And one
|
||||||
|
**paging envelope** (`more` / `cursor` / `cut`) is defined now, with `assets.sources` as its
|
||||||
|
first user, rather than left for whichever family pages first to invent.
|
||||||
|
7. **§4.8: the catalogue is 787 bodies, not 1,144 — measured 2026-09-10, phase 1.** Recorded here
|
||||||
|
because it changes §11's sizing, phase 3's scope and phase 4's, and because of *how* it was
|
||||||
|
found: the animation path has §4.5's shared-buffer defect too, and 357 ids with no art were
|
||||||
|
returning the previously-decoded body's bitmap. The elf ghosts moved from "decodes" to "no art"
|
||||||
|
in §5.2 for the same reason, taking phase 4's set from six player bodies to eight.
|
||||||
|
|||||||
Reference in New Issue
Block a user