docs(link): the hue belongs where the files are, and the cache poisons it (Phase 5)
§11.1 is new and carries what phase 5 measured: 49,152 addressable static ids (not the 81,884 `artidx.mul` declares -- `FileIndex` sizes its table from its length ARGUMENT), 39,189 with art, 4,244 land tiles, 9,963 + 12,140 empty index slots, and the whole set at 81 MB decoding in 34 s. That last number reopens the bulk question and the answer is still no: 108 MB of base64 through a 512 KB single-slot channel to store 43,433 pictures a shard displays a few hundred of. Two traps, both §4.5's failure mode -- a confident, plausible, wrong picture: - `Art.GetStatic` hands back the SAME cached Bitmap and `Hue.ApplyTo` repaints in place, so hueing edits the library's own copy: the plain key comes back hued from then on, and the next hue stacks. `Files.CacheData` off process-wide fixes it and also stops a game server retaining 74 MB of Bitmap. Copying instead does not solve the retention, and `new Bitmap(src)` throws on ARGB1555 anyway. - `PartialHue` (13,259 of 65,536 ids) decides whether a hue repaints every pixel or only the grey ones, from a file only the shard has. Item 597 is a wooden screen with painted flowers; one mode reddens the flowers, the other the whole screen, and both decode. Hence land takes no hue segment and `h0` is not a key. Plus the namespace trap that compiled: unqualified `TileData` binds to ServUO's own `Server.TileData`, because the enclosing namespace beats `using Ultima;`. §14 records what the wire gained -- the `static` and `land` families, `families` on `assets.sources`, and `assets.fetch` becoming shared plumbing whose family is DERIVED from the keys (§5 made the key the address; a request naming its family too would have two places to be wrong and one of them silent). Additive, so the protocol stays 8 and EXTRACTOR_VERSION stays 2. §15 records that `link` needed nothing in phases 4 or 5: it forwards verbatim in both directions. §17.10 is the four org-lead decisions. §12 and modules/uo/SCHEMA.md carry the website side: `uploads/items/`, per-row `catalog` staleness, and why a key with no art writes no row at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
147
link/v8.md
147
link/v8.md
@@ -1017,8 +1017,8 @@ project. §1.1's measurements make the sizing question concrete:
|
|||||||
|
|
||||||
| Kind | Addressable | Bulk? |
|
| Kind | Addressable | Bulk? |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Item statics | **~49,150** | No — on demand, cached, keyed by `itemId` (+ hue) |
|
| Item statics | **49,152** addressable, **39,189** with art (§11.1, phase 5) | No — on demand, cached, keyed by `itemId` (+ hue) |
|
||||||
| Land tiles | **16,384** | No — on demand |
|
| Land tiles | **16,384** addressable, **4,244** with art | No — on demand |
|
||||||
| Creature/player bodies, first frame | **1,022** — 787 legacy (§4.8, *not* the 1,144 the library reports) + 235 UOP (§4.9) | **Yes** — this is the catalogue |
|
| Creature/player bodies, first frame | **1,022** — 787 legacy (§4.8, *not* the 1,144 the library reports) + 235 UOP (§4.9) | **Yes** — this is the catalogue |
|
||||||
| One body, every action, one direction | **210 frames** (body 400); 96–210 measured across six bodies | No — on demand, per body |
|
| One body, every action, one direction | **210 frames** (body 400); 96–210 measured across six bodies | No — on demand, per body |
|
||||||
| All bodies, every action, one direction | **~119,000 frames**, ~117 MB | No — but no longer unthinkable |
|
| All bodies, every action, one direction | **~119,000 frames**, ~117 MB | No — but no longer unthinkable |
|
||||||
@@ -1054,9 +1054,94 @@ default and still not something to import before anything asks for it, but it ha
|
|||||||
bulk-fill-everything switch rather than assuming on-demand is the only mode.
|
bulk-fill-everything switch rather than assuming on-demand is the only mode.
|
||||||
|
|
||||||
**Hued variants are on demand, always.** `static/3922/h33` is generated when something on the wire
|
**Hued variants are on demand, always.** `static/3922/h33` is generated when something on the wire
|
||||||
actually carries hue 33. The cross product of 49,150 statics and ~3,000 hues is not a set anyone
|
actually carries hue 33. The cross product of 49,152 statics and 3,000 hues is not a set anyone
|
||||||
enumerates.
|
enumerates.
|
||||||
|
|
||||||
|
### 11.1 What phase 5 measured, and the two traps it found
|
||||||
|
|
||||||
|
The sizing above was an estimate taken off `art.mul`'s length. Measured through the reader itself,
|
||||||
|
against this machine's stock client:
|
||||||
|
|
||||||
|
| | Measured |
|
||||||
|
|---|---|
|
||||||
|
| Static ids addressable | **49,152** |
|
||||||
|
| ...with real art | **39,189** |
|
||||||
|
| ...empty index slots (§4.5's shape) | **9,963** |
|
||||||
|
| Land tiles addressable / with art | 16,384 / **4,244** |
|
||||||
|
| Whole static + land set, as PNG | 43,433 files, **81 MB**; mean 1.9 KB, max 30.9 KB (id 18213) |
|
||||||
|
| Time to decode and encode all of it | **34 s** |
|
||||||
|
| Hue slots in `hues.mul` | 3,000 (2,062 named) |
|
||||||
|
| Item ids flagged `PartialHue` | **13,259** of 65,536 |
|
||||||
|
|
||||||
|
Two of those need saying out loud.
|
||||||
|
|
||||||
|
**49,152, not the 81,884 entries `artidx.mul` declares.** `FileIndex` sizes its index table from the
|
||||||
|
**length argument it is constructed with** (`0x10000`), not from the idx file, so the addressable
|
||||||
|
static range is `0x10000 - 0x4000`. A ceiling read off the file instead would invent 16,348 ids and
|
||||||
|
answer every one of them out of an array nobody bounded. (The first probe of this phase made exactly
|
||||||
|
that mistake and reported 65,500 — PowerShell returns `$null` for an out-of-range array index rather
|
||||||
|
than throwing, so the over-run counted silently as "empty slots".)
|
||||||
|
|
||||||
|
**81 MB is small enough to reopen the bulk question, and the answer is still no.** Not on size — on
|
||||||
|
what the transfer buys. Base64 puts it at 108 MB through a 512 KB single-slot channel, roughly 210
|
||||||
|
round trips, to store 43,433 pictures of which a live shard displays a few hundred. On-demand stays
|
||||||
|
right; phase 6's bulk-fill switch is where an operator who wants the lot says so.
|
||||||
|
|
||||||
|
#### The library's cache poisons a hued sprite
|
||||||
|
|
||||||
|
`Art.GetStatic` and `Art.GetLand` memoise into a static `Bitmap[0xFFFF]` and hand back **the same
|
||||||
|
instance** on every call; `Hue.ApplyTo` repaints a bitmap **in place**. So the obvious implementation
|
||||||
|
— ask the library, apply the hue, encode — edits the library's own copy. Measured before the fix:
|
||||||
|
hue item 3922 once, and every later request for the **plain** 3922 comes back hued, with a second hue
|
||||||
|
stacking on the first.
|
||||||
|
|
||||||
|
This is §4.5's failure mode exactly — a confident, plausible, correctly-sized wrong picture that
|
||||||
|
every success count agrees with — reached through a door §4.5 never looked at, because phase 0 was
|
||||||
|
auditing *records* and this is the library's *cache*. Nothing downstream can see it: the key is
|
||||||
|
right, the dimensions are right, the hash is stable.
|
||||||
|
|
||||||
|
The fix is `Files.CacheData = false` for the life of the process, set once when the asset plane
|
||||||
|
initialises, and it pays twice: the same array is never trimmed, so decoding this client's 39,189
|
||||||
|
statics would otherwise leave **74 MB of `Bitmap` in a static field of a game server** to serve
|
||||||
|
pictures nobody asks for twice. `Animations` does not consult the flag at all, so the body catalogue
|
||||||
|
is untouched, and each reader keeps its own cache of *encoded PNG bytes* instead — a tenth of the
|
||||||
|
size, already hashed, released when it goes idle.
|
||||||
|
|
||||||
|
The obvious alternative, copying each bitmap before hueing, was rejected for the retention alone —
|
||||||
|
but also because `new Bitmap(src)` **throws** on the `Format16bppArgb1555` these decoders produce.
|
||||||
|
The copy has to name the source pixel format explicitly, which is a subtlety on the wrong side of a
|
||||||
|
correctness boundary. The invariant is instead re-checked before every hue: a hue is refused if the
|
||||||
|
cache is somehow on, because an invariant nothing verifies is a comment.
|
||||||
|
|
||||||
|
#### `PartialHue` decides the picture, and only the shard can read it
|
||||||
|
|
||||||
|
A hue is not a tint. It is a 32-entry colour ramp out of `hues.mul` indexed by each pixel's own red
|
||||||
|
channel — and whether it replaces **every** pixel or only the grey ones is a per-item-id flag in
|
||||||
|
`tiledata.mul`. On this client **13,259 of 65,536 item ids carry it**.
|
||||||
|
|
||||||
|
Item 597 is a wooden screen with painted flowers. Hued 33 the right way the flowers turn red; the
|
||||||
|
wrong way the whole screen turns red. Both decode, both are 44×112, both report success. This is why
|
||||||
|
§5 put hue in the key rather than leaving it to the website: shipping `Hues.mul` semantics and a
|
||||||
|
65,536-row flag table into Node, to answer a question the shard can answer for free, is the trade
|
||||||
|
§2.1 already refused.
|
||||||
|
|
||||||
|
Two consequences fall out of it. **Land takes no hue segment** — the mode is an *item* flag and land
|
||||||
|
has no equivalent, so `land/3/h33` is refused rather than guessed; nothing on the wire carries one
|
||||||
|
today, and if something ever does it will arrive with a reason to choose. And **`h0` is not a key**:
|
||||||
|
hue 0 on the wire means "not hued", so the plain key already names that picture, and accepting both
|
||||||
|
would store one PNG twice under two names and diff them separately forever.
|
||||||
|
|
||||||
|
#### The namespace trap that compiled
|
||||||
|
|
||||||
|
The first cut of the reader wrote `TileData.ItemTable` and `TileFlag.PartialHue` unqualified. ServUO
|
||||||
|
declares its **own** `Server.TileData`, `Server.ItemData` and `Server.TileFlag` — with a
|
||||||
|
`PartialHue` member — in `Server/TileData.cs`, and the reader lives in `Server.Custom.Bridge`, where
|
||||||
|
the enclosing namespace beats `using Ultima;`. It compiled. At runtime it read a file resolved
|
||||||
|
through `Core.DataDirectories` rather than through `Ultima.Files`, which is §4.6's rule broken in a
|
||||||
|
new place: deciding a picture with a file other than the one the pixels came out of. The live rig
|
||||||
|
caught it as a `TypeInitializationException` refusing every hued key, from a class the code never
|
||||||
|
meant to name.
|
||||||
|
|
||||||
**Gump art is out of scope for Protocol 8, and that is now a safety rule rather than a priority
|
**Gump art is out of scope for Protocol 8, and that is now a safety rule rather than a priority
|
||||||
call** — §4.1. It is the only decoder that reaches the `hasExtra: true` branch, and that branch
|
call** — §4.1. It is the only decoder that reaches the `hasExtra: true` branch, and that branch
|
||||||
corrupts the process on the second id. Adding gump art later means fixing that path first,
|
corrupts the process on the second id. Adding gump art later means fixing that path first,
|
||||||
@@ -1077,6 +1162,12 @@ so no `MODULE_API_VERSION` bump is needed to store them.
|
|||||||
- **The operator-supplied `spawnAtlas.art.json` map stays supported** and continues to win over an
|
- **The operator-supplied `spawnAtlas.art.json` map stays supported** and continues to win over an
|
||||||
imported asset. An operator who has drawn their own creature portraits must not have them
|
imported asset. An operator who has drawn their own creature portraits must not have them
|
||||||
overwritten by a sprite rip on the next Update.
|
overwritten by a sprite rip on the next Update.
|
||||||
|
- **Item and land pictures land in `uploads/items/`, beside the creature portraits and not among
|
||||||
|
them** (phase 5). Same content-addressed naming (`uo-static-3922-h33-<sha8>.png`), same "the API
|
||||||
|
returns a filename, the client builds the URL" contract, and the same rule that a missing picture
|
||||||
|
is a first-class state rather than an error. Separate directories because they have different
|
||||||
|
lifetimes: the catalogue is imported as a set and re-imported as a set, while these arrive one at
|
||||||
|
a time because something asked for them.
|
||||||
|
|
||||||
Licensing is unchanged and the reasoning is unchanged: these are the operator's own client files,
|
Licensing is unchanged and the reasoning is unchanged: these are the operator's own client files,
|
||||||
extracted on their own host, for their own shard. Nothing is committed, nothing ships in a repo,
|
extracted on their own host, for their own shard. Nothing is committed, nothing ships in a repo,
|
||||||
@@ -1145,7 +1236,7 @@ new pipe.
|
|||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| `assets.sources` | `assets.sources.ok` | Stage 1: client file manifest + `EXTRACTOR_VERSION` | **phase 1** |
|
| `assets.sources` | `assets.sources.ok` | Stage 1: client file manifest + `EXTRACTOR_VERSION` | **phase 1** |
|
||||||
| `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes, width, height, body, direction, source}]`, paged | **phase 3** (`source` phase 4) |
|
| `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes, width, height, body, direction, source}]`, paged | **phase 3** (`source` phase 4) |
|
||||||
| `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged; base64 PNG per row | **phase 3** (`source` phase 4) |
|
| `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged; base64 PNG per row | **phase 3** (`source` phase 4; `static`/`land` families phase 5) |
|
||||||
| `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) | **phase 3** |
|
| `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) | **phase 3** |
|
||||||
| `cliloc.table` | `cliloc.table.ok` | The decompressed table, paged (`?lang=`, `?cursor=`) | **phase 2** |
|
| `cliloc.table` | `cliloc.table.ok` | The decompressed table, paged (`?lang=`, `?cursor=`) | **phase 2** |
|
||||||
| `tree.manifest` / `tree.fetch` | `.ok` | §10, the ServUO tree files | phase 7 |
|
| `tree.manifest` / `tree.fetch` | `.ok` | §10, the ServUO tree files | phase 7 |
|
||||||
@@ -1167,6 +1258,27 @@ side effect — a few hundred asset keys do not belong in a query string. They a
|
|||||||
this link that take one. `422` gains a second meaning on this plane alongside "cannot decode": the
|
this link that take one. `422` gains a second meaning on this plane alongside "cannot decode": the
|
||||||
mid-import guard, a `catalog` that no longer describes the files on disk.
|
mid-import guard, a `catalog` that no longer describes the files on disk.
|
||||||
|
|
||||||
|
**Phase 5 added two families, one field, and no command.** `assets.fetch` grew `static` and `land`
|
||||||
|
(§5, §11.1) and `assets.sources` grew **`families`** — which key families this overlay serves. Both
|
||||||
|
are additive, so **the protocol stays 8**, and `EXTRACTOR_VERSION` stays **2**: no existing key's
|
||||||
|
bytes change, and a new key is not a re-derivation of an old one.
|
||||||
|
|
||||||
|
The command itself became shared plumbing. Phase 3 gave `assets.fetch` to the body catalogue
|
||||||
|
outright, which was right with one family and wrong with three: the command is the *transport* and
|
||||||
|
the family is a property of the key. So the correlation id, the operator's consent, the key-count
|
||||||
|
ceiling and the family decision now happen once, and a reader only ever sees keys it owns.
|
||||||
|
|
||||||
|
**The family is derived from the keys and is not a request field.** §5 made the key the address of an
|
||||||
|
asset; a request that also named its family would have two places to be wrong and one of them
|
||||||
|
silent. A batch must be of **one** family — mixing them is refused (400) rather than split — because
|
||||||
|
the reply carries a single `catalog` id, and two families have two fingerprints. A reply claiming one
|
||||||
|
of them would be lying about the other.
|
||||||
|
|
||||||
|
`families` matters more than it looks. Without it, a website talking to a phase-3 or phase-4 overlay
|
||||||
|
discovers the gap as a refusal *per key, per pass, forever*, with no picture ever appearing and a
|
||||||
|
warning in the log every few minutes. With it, that is one reported state carrying a sentence naming
|
||||||
|
the fix.
|
||||||
|
|
||||||
**Phase 4 added one field and no command.** `source` on a manifest or fetch row is `legacy` or
|
**Phase 4 added one field and no command.** `source` on a manifest or fetch row is `legacy` or
|
||||||
`uop` — which reader produced the bytes (§4.9). It is additive, so **the protocol stays 8**: a
|
`uop` — which reader produced the bytes (§4.9). It is additive, so **the protocol stays 8**: a
|
||||||
consumer that does not read it is unaffected, and one that does can say which half of the extractor
|
consumer that does not read it is unaffected, and one that does can say which half of the extractor
|
||||||
@@ -1194,7 +1306,7 @@ disagree, so a split bump means the next bundle silently fails to compose.
|
|||||||
| Repo | Work |
|
| Repo | Work |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `servuo-plugins/` | Extraction over ServUO's own `Ultima` (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` |
|
| `servuo-plugins/` | Extraction over ServUO's own `Ultima` (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` |
|
||||||
| `link/` | Six command families forwarded, the REST surface, **the inbound line cap (§3.3)**, `PROTOCOL_VERSION` |
|
| `link/` | Six command families forwarded, the REST surface, **the inbound line cap (§3.3)**, `PROTOCOL_VERSION`. **Nothing in phases 4 or 5** — `assets_call` forwards a request body verbatim and `respond_assets` returns the reply verbatim, so a new key family and a new reply field both pass through untouched |
|
||||||
| `module-uo/` | Client calls, asset store, the atlas source backend (§10), cliloc ingest, admin surface |
|
| `module-uo/` | Client calls, asset store, the atlas source backend (§10), cliloc ingest, admin surface |
|
||||||
| `website/` | Almost none — `ctx.uploads` already suffices (§12). Phase 2 deleted `server/tools/cliloc-export/`, the converter this protocol retires |
|
| `website/` | Almost none — `ctx.uploads` already suffices (§12). Phase 2 deleted `server/tools/cliloc-export/`, the converter this protocol retires |
|
||||||
| `docs/` | This file; rewrite `CLILOCS.md` §Converting and `SPAWN_ATLAS.md` §Artwork + §Configuring; **delete `UOFIDDLER.md`**; add the libgdiplus prerequisite to `SHARD_PREREQS.md` (§4.4) |
|
| `docs/` | This file; rewrite `CLILOCS.md` §Converting and `SPAWN_ATLAS.md` §Artwork + §Configuring; **delete `UOFIDDLER.md`**; add the libgdiplus prerequisite to `SHARD_PREREQS.md` (§4.4) |
|
||||||
@@ -1213,7 +1325,7 @@ disagree, so a split bump means the next bundle silently fails to compose.
|
|||||||
| 2 | **DONE 2026-09-10.** Clilocs end to end (§9.1, §9.2): the Mythic decompressor ported into the overlay, `cliloc.table` + `GET /cliloc`, the paging walk and the source switch on the website, module-uo's protocol pin 7→8. `cliloc-export/` deleted and `UOFIDDLER.md` §Part 1 with it. **67,496 rows, 290 ms, ~11 pages** — the same count UOFiddler's own DLL produced from this client | all |
|
| 2 | **DONE 2026-09-10.** Clilocs end to end (§9.1, §9.2): the Mythic decompressor ported into the overlay, `cliloc.table` + `GET /cliloc`, the paging walk and the source switch on the website, module-uo's protocol pin 7→8. `cliloc-export/` deleted and `UOFIDDLER.md` §Part 1 with it. **67,496 rows, 290 ms, ~11 pages** — the same count UOFiddler's own DLL produced from this client | all |
|
||||||
| 3 | **DONE 2026-09-10.** Body resolution (§8) + the **787**-body catalogue (§4.8), `assets.manifest` / `assets.fetch` / `assets.bodies` and their REST mirrors, `shard_spawn_creatures.art` filled and rendered (§8.1, §12.1). **787 rows in one 734 ms page; 455 types resolved at ~190 ms per 100 on the Core thread; zero mobiles leaked.** `UOFIDDLER.md` deleted, two phases early | servuo-plugins, **link**, module-uo |
|
| 3 | **DONE 2026-09-10.** Body resolution (§8) + the **787**-body catalogue (§4.8), `assets.manifest` / `assets.fetch` / `assets.bodies` and their REST mirrors, `shard_spawn_creatures.art` filled and rendered (§8.1, §12.1). **787 rows in one 734 ms page; 455 types resolved at ~190 ms per 100 on the Core thread; zero mobiles leaked.** `UOFIDDLER.md` deleted, two phases early | servuo-plugins, **link**, module-uo |
|
||||||
| 4 | **DONE 2026-09-11.** The UOP animation decoder (§4.3, §4.9): `BridgeUop` + a PNG encoder that never touches `System.Drawing`, wired in beneath the legacy reader. **Two of the eight player bodies turned out to exist** (gargoyles 666/667); the other six are in no client file, and ghost ids left the player-body set (§5.2, §17.9). The same fallback added **233 other bodies**: the catalogue is **1,022 rows, 1,409 ms cold**, and all six player bodies have art for the first time. `EXTRACTOR_VERSION` 1 → 2 | servuo-plugins |
|
| 4 | **DONE 2026-09-11.** The UOP animation decoder (§4.3, §4.9): `BridgeUop` + a PNG encoder that never touches `System.Drawing`, wired in beneath the legacy reader. **Two of the eight player bodies turned out to exist** (gargoyles 666/667); the other six are in no client file, and ghost ids left the player-body set (§5.2, §17.9). The same fallback added **233 other bodies**: the catalogue is **1,022 rows, 1,409 ms cold**, and all six player bodies have art for the first time. `EXTRACTOR_VERSION` 1 → 2 | servuo-plugins |
|
||||||
| 5 | Item statics and land on demand, hued keys, the cache | servuo-plugins, module-uo |
|
| 5 | **DONE 2026-09-11.** Item statics and land on demand (§11.1): the `static` and `land` families, hue applied on the shard from `tiledata.mul`, the byte-bounded art cache, `assets.fetch` made family-aware, `families` on `assets.sources`. Website side: the warm pass, per-row `catalog` staleness, and pictures on the marketplace and the character sheet. **39,189 statics and 4,244 land tiles served; the only refusals are the 9,963 + 12,140 empty index slots §4.5 predicted.** Two traps found — the library's bitmap cache poisons a hued sprite, and `PartialHue` decides the picture from a file only the shard has. Protocol stays 8; `EXTRACTOR_VERSION` stays 2 | servuo-plugins, module-uo |
|
||||||
| 6 | Deep animation keys (`body/<id>/a<n>/f<n>`) for the future project, plus the bulk-fill switch | servuo-plugins, module-uo |
|
| 6 | Deep animation keys (`body/<id>/a<n>/f<n>`) for the future project, plus the bulk-fill switch | servuo-plugins, module-uo |
|
||||||
| 7 | The atlas over the sidecar (§10); shared-filesystem requirement retired | module-uo |
|
| 7 | The atlas over the sidecar (§10); shared-filesystem requirement retired | module-uo |
|
||||||
| 8 | Admin surface, Import/Update, approve/reject, activity log | module-uo |
|
| 8 | Admin surface, Import/Update, approve/reject, activity log | module-uo |
|
||||||
@@ -1298,6 +1410,29 @@ in the document.
|
|||||||
returning the previously-decoded body's bitmap. The elf ghosts moved from "decodes" to "no art"
|
returning the previously-decoded body's bitmap. The elf ghosts moved from "decodes" to "no art"
|
||||||
in §5.2 for the same reason, taking phase 4's set from six player bodies to eight.
|
in §5.2 for the same reason, taking phase 4's set from six player bodies to eight.
|
||||||
|
|
||||||
|
10. **§11.1: phase 5's four, settled 2026-09-11.** Put to the org lead after the client was measured
|
||||||
|
and before the reader was written, because the first measurement changed what the risk was:
|
||||||
|
|
||||||
|
- **Ingest warms; the route only serves.** A page renders the pictures already on disk and
|
||||||
|
leaves out the ones that are not; fetching happens behind it, on a timer, from the keys the
|
||||||
|
site's own rows name. Chosen over fetching on first request, on one number: the asset plane
|
||||||
|
serves **one request at a time** (§3.2), so a URL that fetched would let any anonymous visitor
|
||||||
|
walk 49,152 ids × 3,000 hues through that single slot and park an operator's own import behind
|
||||||
|
it. Warming from the site's own data has no such surface — the ceiling is the number of
|
||||||
|
distinct (item, hue) pairs the shard has already told the site about.
|
||||||
|
- **Staleness is a per-row catalogue id, not a manifest.** A client patch changes the shard's
|
||||||
|
`catalog` fingerprint and a restart does not, so "is this out of date?" is a column
|
||||||
|
comparison. The alternative — a `static` manifest family enumerating 39,189 rows with hashes —
|
||||||
|
would cost a 34-second scan of the whole art file per Update to answer a question about maybe
|
||||||
|
three hundred pictures, and would re-fetch art nobody looks at any more. Lazy costs nothing
|
||||||
|
for the ones nobody wants.
|
||||||
|
- **The pictures appear on the marketplace and the character sheet**, the two places the data
|
||||||
|
already existed and rendered as `id 1234, hue 33`. That gives the phase an acceptance test
|
||||||
|
checkable by eye, which §11.1 says is the only kind that catches this failure mode.
|
||||||
|
- **`Files.CacheData` goes off process-wide** at asset init, rather than copying each bitmap
|
||||||
|
before hueing. See §11.1: the copy does not solve the 74 MB retention, and the ordinary copy
|
||||||
|
constructor throws on ARGB1555 anyway.
|
||||||
|
|
||||||
9. **§4.3/§5.2: phase 4's four, settled 2026-09-11.** Put to the org lead after the packages were
|
9. **§4.3/§5.2: phase 4's four, settled 2026-09-11.** Put to the org lead after the packages were
|
||||||
opened and before the reader was written, because the first measurement changed what the phase
|
opened and before the reader was written, because the first measurement changed what the phase
|
||||||
was worth:
|
was worth:
|
||||||
|
|||||||
@@ -56,7 +56,7 @@ feed) are described where their wire frames are, in
|
|||||||
| GET | `/shard/ruleset` | the shard's own published ruleset (Protocol 3.0 `world.ruleset`): expansion, which optional systems are on, skill/stat caps, account and house limits, champion scroll rules, the save/restart schedule. Served from `shard_ruleset`, so it renders while the shard is down; live via `world.ruleset` on `/shard/stream`. Behind `requireFeature('ruleset')`. **`null`** means the shard has never published one — a real answer, distinct from a published ruleset. `caps.skill` / `caps.totalSkill` are in **tenths** (1000 = 100.0). |
|
| GET | `/shard/ruleset` | the shard's own published ruleset (Protocol 3.0 `world.ruleset`): expansion, which optional systems are on, skill/stat caps, account and house limits, champion scroll rules, the save/restart schedule. Served from `shard_ruleset`, so it renders while the shard is down; live via `world.ruleset` on `/shard/stream`. Behind `requireFeature('ruleset')`. **`null`** means the shard has never published one — a real answer, distinct from a published ruleset. `caps.skill` / `caps.totalSkill` are in **tenths** (1000 = 100.0). |
|
||||||
| GET | `/shard/points` | every points/loyalty leaderboard the shard publishes (Protocol 3.0 `points.board`) — Queen's Loyalty, Void Pool, the nine city loyalties, Clean Up Britannia, … Served from `shard_points_boards`, so it renders while the shard is down; live via `points.board` on `/shard/stream`. Behind `requireFeature('leaderboards')`, ordered by display name. **`maxPoints: 0` means uncapped** (the common case), and `nameString` is usually `null` with `nameNumber` holding a cliloc — resolve client-side or humanise the `system` key. |
|
| GET | `/shard/points` | every points/loyalty leaderboard the shard publishes (Protocol 3.0 `points.board`) — Queen's Loyalty, Void Pool, the nine city loyalties, Clean Up Britannia, … Served from `shard_points_boards`, so it renders while the shard is down; live via `points.board` on `/shard/stream`. Behind `requireFeature('leaderboards')`, ordered by display name. **`maxPoints: 0` means uncapped** (the common case), and `nameString` is usually `null` with `nameNumber` holding a cliloc — resolve client-side or humanise the `system` key. |
|
||||||
| GET | `/shard/points/:system` | one board by the shard's `PointsType` name (e.g. `QueensLoyalty`); `:system` must match `/^[A-Za-z][A-Za-z0-9_]{0,47}$/` or **400** before any query runs. **404** = the shard has never published that system, which is distinct from a published board nobody has scored in yet (**200** with an empty `top`). |
|
| GET | `/shard/points/:system` | one board by the shard's `PointsType` name (e.g. `QueensLoyalty`); `:system` must match `/^[A-Za-z][A-Za-z0-9_]{0,47}$/` or **400** before any query runs. **404** = the shard has never published that system, which is distinct from a published board nobody has scored in yet (**200** with an empty `top`). |
|
||||||
| GET | `/shard/market?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=` | search the player-vendor marketplace (Protocol 3.0 `vendor.listing`). Returns **listings**, not vendors — "who sells X and for how much" is the question, and a vendor-shaped result would make every caller flatten the shops back out. Served from `shard_vendors` + `shard_vendor_items`, so it renders while the shard is down. Behind `requireFeature('market')` **and rate-limited** — the first genuinely expensive public read on the site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*` table, reachable with no session). `sort ∈ {price_asc, price_desc, recent}`. `q` matches the resolved display name **or** the item's literal name, with `%`/`_` escaped: they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let `?q=%` match every listing on the shard. Every response repeats `staleAt` (the oldest vendor row) because the shard sweeps round-robin — a banner that ages with the results it labels, not one fetched once. |
|
| GET | `/shard/market?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=` | search the player-vendor marketplace (Protocol 3.0 `vendor.listing`). Returns **listings**, not vendors — "who sells X and for how much" is the question, and a vendor-shaped result would make every caller flatten the shops back out. Served from `shard_vendors` + `shard_vendor_items`, so it renders while the shard is down. Behind `requireFeature('market')` **and rate-limited** — the first genuinely expensive public read on the site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*` table, reachable with no session). `sort ∈ {price_asc, price_desc, recent}`. `q` matches the resolved display name **or** the item's literal name, with `%`/`_` escaped: they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let `?q=%` match every listing on the shard. Every response repeats `staleAt` (the oldest vendor row) because the shard sweeps round-robin — a banner that ages with the results it labels, not one fetched once. As of Protocol 8 phase 5 each listing also carries **`art`**: the FILENAME of the item's picture under `uploads/items/`, already hued, or `null` where this site does not hold one. `null` is ordinary — the picture is fetched behind the page and never by it, so a newly listed item shows text first and gains its icon a pass later, and some item ids have no art in any client. The same field appears on a character sheet's equipment entries. |
|
||||||
| GET | `/shard/market/meta` | index size, staleness (`staleAt`/`freshAt`) and which facets and regions actually hold vendors, so a client builds its filters without running a search it will discard. |
|
| GET | `/shard/market/meta` | index size, staleness (`staleAt`/`freshAt`) and which facets and regions actually hold vendors, so a client builds its filters without running a search it will discard. |
|
||||||
| GET | `/shard/market/vendors/:serial` | one shop and its listings; `:serial` must match `/^0x[0-9A-Fa-f]{1,16}$/` or **400** before any query runs. **404** = a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer, and distinguishing them would leak that a hidden vendor exists. `truncated` (with `total` exceeding `count`) means the shop holds more than the shard publishes per frame. |
|
| GET | `/shard/market/vendors/:serial` | one shop and its listings; `:serial` must match `/^0x[0-9A-Fa-f]{1,16}$/` or **400** before any query runs. **404** = a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer, and distinguishing them would leak that a hidden vendor exists. `truncated` (with `total` exceeding `count`) means the shop holds more than the shard publishes per frame. |
|
||||||
| GET | `/shard/features` | the shard features **this caller** may reach plus the audience rung they resolved to (§4 below), so a client hides nav it can't follow. Reports only what the caller can see — the list itself never discloses a gated feature. Consumed by the SPA header and (pending) the Android nav. |
|
| GET | `/shard/features` | the shard features **this caller** may reach plus the audience rung they resolved to (§4 below), so a client hides nav it can't follow. Reports only what the caller can see — the list itself never discloses a gated feature. Consumed by the SPA header and (pending) the Android nav. |
|
||||||
@@ -83,6 +83,7 @@ account-linking routes and the sidecar config under `/admin/uo-link` are in the
|
|||||||
| POST | `/shard/clilocs/import` | reload after a client patch or an overlay edit; `{force}` ignores the hash gate, `{approve}` accepts a **vanished** source (refused by default — see the table notes above). **A missing path — or the likely mistake of pointing at the client's own COMPRESSED `Cliloc.enu` — answers 200 with `status:"unavailable"` and a `code`, not 500.** `COMPRESSED` is called out by name: a 500 would say only "something broke", and the operator needs to be told which file to convert. |
|
| POST | `/shard/clilocs/import` | reload after a client patch or an overlay edit; `{force}` ignores the hash gate, `{approve}` accepts a **vanished** source (refused by default — see the table notes above). **A missing path — or the likely mistake of pointing at the client's own COMPRESSED `Cliloc.enu` — answers 200 with `status:"unavailable"` and a `code`, not 500.** `COMPRESSED` is called out by name: a 500 would say only "something broke", and the operator needs to be told which file to convert. |
|
||||||
| GET | `/shard/assets` | client-asset import status (`adminOnly`, Protocol 8): the imported body catalogue, how many sprites are on disk, how many atlas creatures resolved to a body id, and the shard's own client files beside them. `drift:true` means the client was patched. `shard.imaging.ok:false` is the named `NO_IMAGING` state — a Linux shard host with no `libgdiplus` cannot decode a sprite at all, and the reason names the package. No public counterpart: the pictures are served as ordinary files under `/uploads`. |
|
| GET | `/shard/assets` | client-asset import status (`adminOnly`, Protocol 8): the imported body catalogue, how many sprites are on disk, how many atlas creatures resolved to a body id, and the shard's own client files beside them. `drift:true` means the client was patched. `shard.imaging.ok:false` is the named `NO_IMAGING` state — a Linux shard host with no `libgdiplus` cannot decode a sprite at all, and the reason names the package. No public counterpart: the pictures are served as ordinary files under `/uploads`. |
|
||||||
| POST | `/shard/assets/import` | import creature artwork from the shard's UO client; `{force}` ignores the hash gate, `{approve}` accepts a catalogue that no longer offers assets this site holds (refused by default — an unmounted client volume and a deliberate downgrade are indistinguishable, and the wrong guess deletes artwork). **This is the ONLY thing that imports** — boot deliberately never calls the shard. An operator's `spawnAtlas.art.json` always wins over an imported sprite. A body this client has no art for is **not** a failure: about half the addressable body range is in that state on a stock client. |
|
| POST | `/shard/assets/import` | import creature artwork from the shard's UO client; `{force}` ignores the hash gate, `{approve}` accepts a catalogue that no longer offers assets this site holds (refused by default — an unmounted client volume and a deliberate downgrade are indistinguishable, and the wrong guess deletes artwork). **This is the ONLY thing that imports** — boot deliberately never calls the shard. An operator's `spawnAtlas.art.json` always wins over an imported sprite. A body this client has no art for is **not** a failure: about half the addressable body range is in that state on a stock client. |
|
||||||
|
| POST | `/shard/assets/warm` | fetch item and land pictures this site is missing, now, instead of waiting for the warm timer (`adminOnly`, Protocol 8 phase 5). The pass works out which item pictures the site's own rows name — every distinct (ItemID, hue) on a player vendor, plus anything a character sheet has shown since the last pass — and fetches the ones it does not hold, **hued on the shard**, into `uploads/items/`. There is deliberately **no manifest and no bulk import**: the client addresses 49,152 item graphics times 3,000 hues, so the working set is what the site displays. `{force}` re-fetches pictures already held (how an operator recovers a wiped uploads volume); `{limit}` bounds one pass, default 400, because the shard serves one asset request at a time. A plugin overlay older than phase 5 answers `status:"unavailable"`, `code:"UNSUPPORTED"` with a sentence naming the fix. |
|
||||||
| PUT | `/shard/clilocs/path` | point the site at a different cliloc base file or directory (persisted as `cliloc_client_path`, which wins over `UO_CLIENT_PATH`). Overlays are read from `custom/` beside it either way. Blank clears it. Deliberately **does not import**, same reasoning as the atlas path. |
|
| PUT | `/shard/clilocs/path` | point the site at a different cliloc base file or directory (persisted as `cliloc_client_path`, which wins over `UO_CLIENT_PATH`). Overlays are read from `custom/` beside it either way. Blank clears it. Deliberately **does not import**, same reasoning as the atlas path. |
|
||||||
|
|
||||||
## 4. Shard visibility — the audience boundary (Protocol 3.0)
|
## 4. Shard visibility — the audience boundary (Protocol 3.0)
|
||||||
|
|||||||
@@ -202,7 +202,7 @@ Creature artwork read from the shard's own UO client ([`link/v8.md`](../../link/
|
|||||||
|
|
||||||
| Table | Shape |
|
| Table | Shape |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `shard_assets` | `asset_key` VARCHAR PK (§5's key, e.g. `body/34/a0`), `family`, `sha256`, `bytes`, `width`, `height`, `body`, `direction`, `file`, `imported_at` |
|
| `shard_assets` | `asset_key` VARCHAR PK (§5's key, e.g. `body/34/a0`, `static/3922/h33`, `land/3`), `family`, `sha256`, `bytes`, `width`, `height`, `body`, `direction`, `file`, `catalog`, `imported_at` |
|
||||||
| `shard_creature_bodies` | `slug` PK, `type_name` (the ServUO class name asked), `body` nullable, `status`, `resolved_at` |
|
| `shard_creature_bodies` | `slug` PK, `type_name` (the ServUO class name asked), `body` nullable, `status`, `resolved_at` |
|
||||||
| `shard_asset_meta` | Singleton (`id = 1`), `payload` JSON (catalogue id, extractor version, source fingerprint, counts), `imported_at` |
|
| `shard_asset_meta` | Singleton (`id = 1`), `payload` JSON (catalogue id, extractor version, source fingerprint, counts), `imported_at` |
|
||||||
|
|
||||||
@@ -235,6 +235,34 @@ Four details that are load-bearing rather than incidental:
|
|||||||
winning spelling of the spawn type token rather than inventing a display label — which is why the
|
winning spelling of the spawn type token rather than inventing a display label — which is why the
|
||||||
body pass needs no extra column to ask its question.
|
body pass needs no extra column to ask its question.
|
||||||
|
|
||||||
|
### Item and land art: the same table, a different shape of use (phase 5)
|
||||||
|
|
||||||
|
The creature catalogue is a **set**: one manifest walk covers every key, so one stored fingerprint in
|
||||||
|
`shard_asset_meta` describes all of them and an Update is a hash diff. Item art has no set — the
|
||||||
|
client addresses 49,152 item graphics times 3,000 hues — so those rows arrive one at a time, because
|
||||||
|
something on this site named the key.
|
||||||
|
|
||||||
|
Three consequences in this schema:
|
||||||
|
|
||||||
|
- **`catalog` is per row**, and it is what phase 5 added. It records the shard's art catalogue id (a
|
||||||
|
hash of `artLegacyMUL.uop`/`art.mul`, `hues.mul`, `tiledata.mul`, `verdata.mul` and its extractor
|
||||||
|
version), so staleness is a column comparison rather than a manifest diff. A client patch changes
|
||||||
|
it; a restart does not. NULL means "written before the column existed", which is stale by the same
|
||||||
|
test and costs one re-fetch. The body import fills it too, so one column answers the question
|
||||||
|
everywhere.
|
||||||
|
- **`shard_asset_meta` stays the body catalogue's alone.** A warm pass writing there would tell the
|
||||||
|
body import that a client it never looked at is unchanged, and the creature catalogue would quietly
|
||||||
|
stop updating.
|
||||||
|
- **`family` is now load-bearing**, not decoration: `body`, `static` and `land` rows share the table
|
||||||
|
and have different lifetimes. The admin status counts them separately for the same reason — there
|
||||||
|
is no "how many are there" to compare `static` against, so the only honest number is how many the
|
||||||
|
site has been asked for and holds.
|
||||||
|
|
||||||
|
Pictures land in `server/uploads/items/` (gitignored like `uploads/atlas/`), content-addressed the
|
||||||
|
same way — `uo-static-3922-h33-<sha8>.png`. A key the shard has no art for writes **no row at all**:
|
||||||
|
an empty row would make it "held" and it would never be asked again, including after the operator
|
||||||
|
patches in the graphic that was missing.
|
||||||
|
|
||||||
## shard_clilocs / shard_cliloc_meta — UO's localization table (Protocol 3.0)
|
## shard_clilocs / shard_cliloc_meta — UO's localization table (Protocol 3.0)
|
||||||
|
|
||||||
Items on the wire carry a `LabelNumber`, not a name. The bridge has always sent it —
|
Items on the wire carry a `LabelNumber`, not a name. The bridge has always sent it —
|
||||||
|
|||||||
Reference in New Issue
Block a user