Compare commits
1 Commits
f6ea5c0484
...
docs/event
| Author | SHA1 | Date | |
|---|---|---|---|
| c26d06a782 |
@@ -79,8 +79,6 @@ android-app/
|
||||
│ │ │ │ │ │ └── PushTickle.kt
|
||||
│ │ │ │ │ ├── result/
|
||||
│ │ │ │ │ │ └── ApiResult.kt
|
||||
│ │ │ │ │ ├── time/
|
||||
│ │ │ │ │ │ └── Instants.kt
|
||||
│ │ │ │ │ ├── web/
|
||||
│ │ │ │ │ │ ├── WebHandoff.kt
|
||||
│ │ │ │ │ │ └── WebsiteUrls.kt
|
||||
@@ -92,7 +90,6 @@ android-app/
|
||||
│ │ │ │ │ │ │ ├── AdminDto.kt
|
||||
│ │ │ │ │ │ │ ├── AuthDto.kt
|
||||
│ │ │ │ │ │ │ ├── ContactDto.kt
|
||||
│ │ │ │ │ │ │ ├── EventsDto.kt
|
||||
│ │ │ │ │ │ │ ├── NotificationsDto.kt
|
||||
│ │ │ │ │ │ │ ├── PageDto.kt
|
||||
│ │ │ │ │ │ │ ├── PlayerShardDto.kt
|
||||
@@ -105,7 +102,6 @@ android-app/
|
||||
│ │ │ │ │ │ ├── AdminApi.kt
|
||||
│ │ │ │ │ │ ├── AuthApi.kt
|
||||
│ │ │ │ │ │ ├── AuthRefreshApi.kt
|
||||
│ │ │ │ │ │ ├── EventsApi.kt
|
||||
│ │ │ │ │ │ ├── MeApi.kt
|
||||
│ │ │ │ │ │ ├── NotificationsApi.kt
|
||||
│ │ │ │ │ │ ├── PlayerShardApi.kt
|
||||
@@ -121,13 +117,11 @@ android-app/
|
||||
│ │ │ │ │ ├── ConnectionRepository.kt
|
||||
│ │ │ │ │ ├── ContactRepository.kt
|
||||
│ │ │ │ │ ├── ContentRepository.kt
|
||||
│ │ │ │ │ ├── EventsRepository.kt
|
||||
│ │ │ │ │ ├── NotificationsRepository.kt
|
||||
│ │ │ │ │ ├── PlayerShardRepository.kt
|
||||
│ │ │ │ │ ├── SettingsRepository.kt
|
||||
│ │ │ │ │ ├── ShardFeaturesRepository.kt
|
||||
│ │ │ │ │ ├── ShardRepository.kt
|
||||
│ │ │ │ │ ├── SiteCapabilitiesRepository.kt
|
||||
│ │ │ │ │ └── WikiRepository.kt
|
||||
│ │ │ │ ├── di/
|
||||
│ │ │ │ │ ├── AppModule.kt
|
||||
@@ -163,16 +157,6 @@ android-app/
|
||||
│ │ │ │ │ ├── contact/
|
||||
│ │ │ │ │ │ ├── ContactScreen.kt
|
||||
│ │ │ │ │ │ └── ContactViewModel.kt
|
||||
│ │ │ │ │ ├── events/
|
||||
│ │ │ │ │ │ ├── EventScreen.kt
|
||||
│ │ │ │ │ │ ├── EventSeriesScreen.kt
|
||||
│ │ │ │ │ │ ├── EventSeriesViewModel.kt
|
||||
│ │ │ │ │ │ ├── EventsScreen.kt
|
||||
│ │ │ │ │ │ ├── EventsViewModel.kt
|
||||
│ │ │ │ │ │ ├── EventTimes.kt
|
||||
│ │ │ │ │ │ ├── EventViewModel.kt
|
||||
│ │ │ │ │ │ ├── MyEventsScreen.kt
|
||||
│ │ │ │ │ │ └── MyEventsViewModel.kt
|
||||
│ │ │ │ │ ├── home/
|
||||
│ │ │ │ │ │ ├── HomeScreen.kt
|
||||
│ │ │ │ │ │ └── HomeViewModel.kt
|
||||
@@ -351,7 +335,6 @@ android-app/
|
||||
│ │ │ │ │ └── WikiDtoTest.kt
|
||||
│ │ │ │ └── fake/
|
||||
│ │ │ │ ├── FakeAdminApi.kt
|
||||
│ │ │ │ ├── FakeEventsApi.kt
|
||||
│ │ │ │ ├── FakeNotificationsApi.kt
|
||||
│ │ │ │ ├── FakePlayerShardApi.kt
|
||||
│ │ │ │ ├── FakePublicApi.kt
|
||||
@@ -362,8 +345,7 @@ android-app/
|
||||
│ │ │ └── repository/
|
||||
│ │ │ ├── AccountTrustedDevicesTest.kt
|
||||
│ │ │ ├── ConnectionVersionGuardTest.kt
|
||||
│ │ │ ├── ShardFeaturesRepositoryTest.kt
|
||||
│ │ │ └── SiteCapabilitiesRepositoryTest.kt
|
||||
│ │ │ └── ShardFeaturesRepositoryTest.kt
|
||||
│ │ ├── ui/
|
||||
│ │ │ ├── admin/
|
||||
│ │ │ │ ├── AdminContentViewModelTest.kt
|
||||
@@ -374,12 +356,8 @@ android-app/
|
||||
│ │ │ │ └── BrandAssetsTest.kt
|
||||
│ │ │ ├── contact/
|
||||
│ │ │ │ └── ContactViewModelTest.kt
|
||||
│ │ │ ├── events/
|
||||
│ │ │ │ ├── EventsViewModelsTest.kt
|
||||
│ │ │ │ └── EventTimesTest.kt
|
||||
│ │ │ ├── navigation/
|
||||
│ │ │ │ ├── MenuAccessTest.kt
|
||||
│ │ │ │ ├── MenuCapabilityGatingTest.kt
|
||||
│ │ │ │ ├── MenuFeatureGatingTest.kt
|
||||
│ │ │ │ ├── NavOverridesTest.kt
|
||||
│ │ │ │ ├── NavPathsTest.kt
|
||||
|
||||
31
link/v6.md
31
link/v6.md
@@ -80,31 +80,10 @@ A repeat of a key still in flight is answered **`bridge.busy`**: nothing runs, a
|
||||
told to come back. It is deliberately not spelled `bridge.busy.error` — nothing is wrong, the work
|
||||
is happening.
|
||||
|
||||
**2. A key that has begun is never released — except on a refusal.** Not when the handler throws.
|
||||
Releasing it would let a retry re-run a command that may have applied half of itself, which is the
|
||||
exact failure this file exists to prevent. A handler that throws stores a `bridge.error` reply
|
||||
instead, so the retry gets a definite answer and the step fails once rather than looping.
|
||||
|
||||
**A REFUSAL is the third case**, added by the Phase 16 acceptance walk and amending protocol 7 in
|
||||
place. A handler that ran to completion and answered `*.error` did not do anything — every refusal
|
||||
on this plane is a guard: a missing `runId`, an unknown item, a cap, a rate limit, a write that
|
||||
failed and left the value alone. Remembering it froze the answer for ever, so a refusal that
|
||||
*waiting fixes* could never be retried past. `uo.world.save` is the case that found it: the shard
|
||||
saves at most every 300 seconds, the module documents that as "the one refusal on this plane that
|
||||
waiting fixes", and six attempts over four minutes all replayed one frozen sentence — "the last save
|
||||
was 227 seconds ago" — because the number was the first reply's, not the clock's. A step's key is
|
||||
one value for the life of the step, so the operator's retry control could not escape it either.
|
||||
|
||||
So a refusal releases the key: nothing happened, and the caller may ask again. The refusal is still
|
||||
**emitted** to the caller, which is what ends that attempt; it is simply not remembered as the key's
|
||||
answer. A refusal is recognised by its `kind` ending in `.error`, matched on the suffix so a handler
|
||||
family added later is covered without extending a list. `bridge.error` is excluded deliberately —
|
||||
that is the reply the shard writes when a handler THREW, which is the case whose key must be kept.
|
||||
|
||||
**This puts a rule on handlers, and it is the rule the release rests on: do not answer `*.error`
|
||||
after changing the world.** Report a partial change in an `ok` reply, as `item.grant` does with
|
||||
`granted`/`missed` and `world.despawn` with `removed`/`gone`/`refused`. The shard cannot verify
|
||||
"nothing happened"; it takes the `.error` kind as the claim.
|
||||
**2. A key that has begun is never released.** Not even when the handler throws. Releasing it would
|
||||
let a retry re-run a command that may have applied half of itself, which is the exact failure this
|
||||
file exists to prevent. A handler that throws stores a `bridge.error` reply instead, so the retry
|
||||
gets a definite answer and the step fails once rather than looping.
|
||||
|
||||
**3. A replay is stamped with the REPEAT's correlation id.** The sidecar's `reqId` is a fresh
|
||||
per-process counter, so a retry is waiting on an id the first attempt never used. Replaying the
|
||||
@@ -123,7 +102,7 @@ evicted key's repeat *would* be applied a second time — so an eviction that dr
|
||||
its TTL prints a console warning naming the count. If the promise is ever actually breached, an
|
||||
operator reads it here rather than discovering a doubled spawn in the world.
|
||||
|
||||
`[bridge status` reports `idem(keys= seen= replayed= busy= evicted= uncorrelated= refused=)`.
|
||||
`[bridge status` reports `idem(keys= seen= replayed= busy= evicted= uncorrelated=)`.
|
||||
|
||||
#### 2.1.1 How the reply is captured
|
||||
|
||||
|
||||
768
link/v8.md
768
link/v8.md
@@ -1,768 +0,0 @@
|
||||
# Asset Bridge (Protocol 8) — client assets without UOFiddler
|
||||
|
||||
**Call it the Asset Bridge.** That is the name for this work everywhere — commits, PR titles,
|
||||
branches (`feat/asset-bridge-p<n>`), and conversation. The protocol number is **8** and this file is
|
||||
`docs/link/v8.md`.
|
||||
|
||||
**Status:** Design of record, **phase 0 complete**. Approved in principle 2026-09-09
|
||||
(architecture, asset scope, built-in cliloc decoder, atlas cleanup); refined 2026-09-10 (one
|
||||
direction not five, ServUO's own decoders, the UOP reader for player bodies, the libgdiplus
|
||||
requirement). The phase 0 spike ran on 2026-09-10 and its findings are §4.5-§4.7 -- §4's decision
|
||||
stands, and the response to a malformed record is now measured rather than proposed. **Nothing in
|
||||
§17 is open.**
|
||||
**Supersedes the manual half of:** [`../website/UOFIDDLER.md`](../website/UOFIDDLER.md),
|
||||
[`../website/CLILOCS.md`](../website/CLILOCS.md) §Converting,
|
||||
[`../website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md) §Artwork and §Configuring the tree.
|
||||
|
||||
Two features on this platform read data that only exists inside a UO client, and today both reach
|
||||
the site by hand: the operator installs UOFiddler, converts `Cliloc.enu` on their own desktop,
|
||||
exports sprites one at a time from a GUI, hand-writes a slug→filename JSON map, and copies the
|
||||
result to the server. A third — the spawn atlas — avoids UOFiddler but pays a different price: the
|
||||
**website** must be able to read the shard's ServUO tree directly, over a bind mount or a shared
|
||||
volume.
|
||||
|
||||
This protocol deletes all three arrangements. The shard already has everything, and the bridge
|
||||
already goes to the website.
|
||||
|
||||
---
|
||||
|
||||
## 1. The premise, which turns out to be free
|
||||
|
||||
**A ServUO shard cannot boot without a UO client installation.** It reads maps, statics, tiledata
|
||||
and multis out of `.mul`/`.uop` files, and `Config/DataPath.cfg` is where an operator declares
|
||||
where those live — *required* on Linux, auto-detected from the registry on Windows. At runtime the
|
||||
resolved directories sit in `Server.Core.DataDirectories`, a public static the plugin can read on
|
||||
any shard, with no new configuration and nothing for an operator to set up.
|
||||
|
||||
So the files the operator has been converting on their desktop are already on the shard host, in a
|
||||
directory the shard already knows the path of, in a process the bridge already runs inside.
|
||||
|
||||
Everything below follows from that.
|
||||
|
||||
### 1.1 What was measured, not assumed
|
||||
|
||||
Against this machine's ServUO 57.4 tree (`C:\Users\colby\Desktop\ServUO`) and client
|
||||
(`D:\Games\Electronic Arts\Ultima Online Classic`, 3.5 GB), loading **ServUO's own
|
||||
`Ultima.dll`** — the assembly `overlay/Scripts/Scripts.csproj:39` already carries a
|
||||
`<ProjectReference>` to:
|
||||
|
||||
| Call | Result |
|
||||
|---|---|
|
||||
| `Art.GetStatic(0…16383)` | 16,384 decoded, 0 errors |
|
||||
| `Art.GetStatic(16384…65535)` | 32,766 decoded, 1 empty, 16,385 clean out-of-range errors |
|
||||
| `Art.GetLand(0…16383)` | 16,384 decoded, 0 errors |
|
||||
| `Animations.GetAnimation(0…2047, 0, 1)` | **1,144** bodies with a decodable first frame, 904 empty, 0 errors |
|
||||
| `Hues.GetHue(33)` | loads |
|
||||
| `Bitmap.Save(…, Png)` | 852-byte PNG from one creature frame |
|
||||
| `Gumps.GetGump(2)` | **hard crash** — `AccessViolationException`, process exit `0xC0000005` |
|
||||
| `new StringList("enu", "Cliloc.enu")` | throws — `Non-negative number required` |
|
||||
|
||||
Two of those rows are load-bearing and are dealt with in §4 and §9. The rest say the same thing:
|
||||
**most of the extraction this protocol needs is already implemented, already compiled, and already
|
||||
referenced by the plugin's own build.**
|
||||
|
||||
Depth, for §11's sizing: body 400 (human male) has **35 actions × 5 directions = 1,050 frames**.
|
||||
One body. §5.1 cuts that by exactly 5×.
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture: the shard extracts, the sidecar forwards, the website decides
|
||||
|
||||
```
|
||||
UO client files (operator's own, on the shard host)
|
||||
│ read by the plugin, off the Core thread
|
||||
▼
|
||||
ServUO shard (servuo-plugins/) ← decodes; resolves body ids; hashes
|
||||
│ loopback JSON, request/reply, one batch outstanding at a time
|
||||
▼
|
||||
uo-link sidecar (link/) ← forwards bytes; decides nothing
|
||||
│ REST, bearer-token auth, X-UOLink-Version: 8
|
||||
▼
|
||||
website (module-uo/) ← stores, names, gates, serves
|
||||
```
|
||||
|
||||
This is deliberately the *only* arrangement that keeps
|
||||
[the bridge's standing rules](PLAN.md) intact:
|
||||
|
||||
- **The sidecar stays a dumb forwarder.** It moves opaque assets and decides nothing about them —
|
||||
no audience, no projection, no capability advertisement. Putting the decoders in Rust would have
|
||||
meant the sidecar deciding what an asset *is*, on top of re-deriving in Rust what is already
|
||||
compiled next door in C#.
|
||||
- **Access control stays on the website**, which has the auth machinery and the admin forms.
|
||||
- **The shard is still never network-reachable.** Nothing here opens a port; the plugin answers
|
||||
requests on the connection it already dialled out on.
|
||||
|
||||
### 2.1 Why not the sidecar, and why not the operator's desktop
|
||||
|
||||
A Rust extractor in the sidecar would need ports of: the Mythic cliloc decompressor, `FileIndex`
|
||||
(including UOP), the ARGB1555 run-length frame decoder, `Body.def`/`Bodyconv.def` translation,
|
||||
`Hues.mul`, and a PNG encoder — weeks of work to re-derive what §1.1 shows already runs. It also
|
||||
cannot do §8: resolving a creature slug to a body id requires being inside ServUO.
|
||||
|
||||
Automating on the operator's desktop (shipping the converter with the installer) removes UOFiddler
|
||||
but keeps a manual step and still cannot do §8. It was considered and rejected.
|
||||
|
||||
---
|
||||
|
||||
## 3. The transport, and the three traps in it
|
||||
|
||||
### 3.1 Assets go over the request/reply path, never the event path
|
||||
|
||||
`link/sidecar/src/app.rs:122` persists **every** non-`pong` event into the SQLite store *and*
|
||||
broadcasts it to every WebSocket subscriber. An asset stream on that path would grow the sidecar's
|
||||
store without bound and fan megabytes out to every connected client, forever.
|
||||
|
||||
`rpc.rs`'s `try_route` consumes a correlated reply and `continue`s **before** either of those
|
||||
happens. So an asset batch is a reply, not an event. This is not a new mechanism — it is the one
|
||||
`char.request`, `account.roster` and `vendor.snapshot` already use.
|
||||
|
||||
### 3.2 One batch outstanding, always
|
||||
|
||||
`BridgeLink.Emit()` enqueues onto a **bounded drop-oldest** queue (`Bridge.QueueCap`, default
|
||||
10,000). It counts **lines, not bytes** — a design that is correct for live events and dangerous
|
||||
for bulk transfer, because 10,000 queued 200 KB replies is 2 GB of shard memory.
|
||||
|
||||
The rule that makes this safe is flow control, not a bigger queue: **the website requests batch
|
||||
*n+1* only after batch *n* has arrived.** Queue depth stays at approximately one. A dropped or
|
||||
lost reply simply times out and the batch is re-requested, which is safe because reading a client
|
||||
file is idempotent and has no world side effects.
|
||||
|
||||
### 3.3 The size ceilings are already fixed, and one of them is missing
|
||||
|
||||
| Limit | Value | Where |
|
||||
|---|---|---|
|
||||
| Sidecar waits for a shard reply | **10 s** | `rpc.rs` `REPLY_TIMEOUT` |
|
||||
| Website waits for the sidecar | **12 s** | `module-uo/server/utils/uoLinkClient.js` `TIMEOUT_MS` |
|
||||
| Sidecar → shard line | 1 MiB | `BridgeLink.cs:283` |
|
||||
| **Shard → sidecar line** | **none** | `shard.rs` uses `read_line` unbounded |
|
||||
|
||||
The first two bound a batch: it must decode, encode, serialise and cross the wire inside ten
|
||||
seconds. The last is a gap this protocol must close — an unbounded `read_line` facing a component
|
||||
that is now deliberately sending large lines is a memory-exhaustion shape we would be inventing
|
||||
ourselves. **Protocol 8 adds an explicit inbound line cap to the sidecar**, set above the largest
|
||||
legal batch and rejecting rather than buffering past it.
|
||||
|
||||
Batches are therefore sized by bytes, not by count, with the emitter cutting a batch short when it
|
||||
would exceed the cap. Base64 costs 33%; the budget must be stated in encoded bytes.
|
||||
|
||||
---
|
||||
|
||||
## 4. The decoders are ServUO's own — decided, and the crash is narrower than it looked
|
||||
|
||||
**We call ServUO's vendored `Ultima` (decided 2026-09-10).** No decoders are reimplemented.
|
||||
`overlay/Scripts/Scripts.csproj:39` already references the project, so the art half of this protocol
|
||||
costs plumbing rather than pixel code, and only §9's cliloc decompressor is written from scratch.
|
||||
|
||||
The reason that is safe, rather than merely cheap, is a distinction §1.1 did not draw at first.
|
||||
|
||||
### 4.1 The crash lives on one code path, and nothing we call uses it
|
||||
|
||||
`Gumps.GetGump(2)` does not fail — it **corrupts the process**: `AccessViolationException`, exit
|
||||
`0xC0000005`. That is a corrupted-state exception, uncatchable by an ordinary `try/catch` on .NET
|
||||
Framework 4.8, so in-process on a live shard it is a crash with players on it. That much is
|
||||
alarming, and on its own it looked like an argument against using this library at all.
|
||||
|
||||
It is not, because of how the three decoders construct their `FileIndex`:
|
||||
|
||||
| Decoder | UOP file | `hasExtra` | Probed |
|
||||
|---|---|---|---|
|
||||
| `Art` | `artLegacyMUL.uop` | **false** | 49,150 statics + 16,384 land tiles, **0 faults** |
|
||||
| `Animations` | *none — legacy `anim*.mul` only* | — | 1,144 bodies, **0 faults** |
|
||||
| `Gumps` | `gumpartLegacyMUL.uop` | **true** | **faults on the second id** |
|
||||
|
||||
`FileIndex.cs`'s own comment says the extra-field handling exists *for* `gumpartlegacy.uop` — it is
|
||||
the one UOP layout carrying an extra field, and `hasExtra: true` is the branch written to cope with
|
||||
it. **Gumps is the only caller that sets it.** So the fault is not a general fragility in this
|
||||
library's `unsafe` code; it is a bug on a branch that exactly one decoder reaches, and that decoder
|
||||
is already out of scope (§11).
|
||||
|
||||
The rule this turns into is a safety rule, not a preference: **nothing in this protocol calls
|
||||
`Ultima.Gumps`.** Adding gump art later means fixing or replacing that path first, deliberately,
|
||||
not discovering it in production.
|
||||
|
||||
### 4.2 What the decision accepts
|
||||
|
||||
Three costs come with it, all known and none of them blocking:
|
||||
|
||||
1. **Six of the twelve stock player-character bodies have no art on this path** — both human ghosts
|
||||
and every gargoyle body (§5.2). `Animations` never reads `AnimationFrame*.uop`. **This one is
|
||||
not merely accepted: §4.3 adds a decoder for it**, because it is the player character and the
|
||||
scope says player models.
|
||||
2. **`System.Drawing` is a hard dependency, in the decode and not just the encode.** `Frame`
|
||||
writes ARGB1555 straight through a `LockBits` pointer, so a Linux shard needs **libgdiplus** to
|
||||
read a sprite at all. That is a stated prerequisite — §4.4.
|
||||
3. **We inherit whatever `Ultima` a given ServUO vendors**, which can change under a shard upgrade.
|
||||
`EXTRACTOR_VERSION` (§7) is the mitigation: it already counts as drift, so a shard whose library
|
||||
changed re-derives on the next import.
|
||||
|
||||
The residual risk that remains is a patched or custom client tripping an out-of-bounds read on a
|
||||
path we *do* call. §16's phase 0 is where that gets exercised rather than assumed.
|
||||
|
||||
### 4.3 One decoder we do write: UOP animation, for the player bodies
|
||||
|
||||
The six missing bodies are the player character, and the scope is player models, so they get a
|
||||
decoder rather than a caveat. It is deliberately the **narrowest possible** addition: a reader for
|
||||
`AnimationFrame*.uop`, used *only* for bodies the legacy path cannot resolve. Everything the
|
||||
vendored code can already decode keeps going through the vendored code.
|
||||
|
||||
This client ships `AnimationFrame1/2/3/4/6.uop` (107, 118, 253, 115 and 24 MB) plus
|
||||
`AnimationSequence.uop`. ServUO's `FileIndex` already contains a UOP reader — but `Animations`
|
||||
never constructs one, and the UOP animation *payload* is its own format rather than a repackaged
|
||||
mul record, so wiring the existing `FileIndex` in is not sufficient. Two license-compatible
|
||||
references exist to work from: ClassicUO's animation loader (GPL-3, and we are GPL-3-or-later) and
|
||||
UOFiddler 4.22 (Beerware, already established in §9).
|
||||
|
||||
**The trap this must not fall into, and it is why the phase exists at all.** `Bodyconv.def` maps
|
||||
gargoyle 666 to `anim5`, and `BodyConverter.Convert` faithfully returns fileType 5 — where the
|
||||
client has nothing. Asking the *other* anim files for index 666 does not fail. It returns 175
|
||||
decodable action/direction combinations of **a giant spider**, because something unrelated occupies
|
||||
that index in `anim2.mul`; fileTypes 3 and 4 return misaligned colour fragments. Rendered and
|
||||
confirmed.
|
||||
|
||||
So the extractor takes `BodyConverter.Convert`'s answer and, if that yields nothing, reports
|
||||
nothing. **It must never sweep file types looking for a hit** — that does not find missing art, it
|
||||
silently puts a spider on the gargoyle page, and no error is raised anywhere. A "0 rows" outcome is
|
||||
correct behaviour; a plausible wrong picture is the failure this protocol most needs to avoid,
|
||||
because nothing downstream can detect it.
|
||||
|
||||
### 4.4 Requirement: a Linux shard host needs libgdiplus
|
||||
|
||||
**Stated prerequisite, not a soft recommendation.** ServUO targets `net48`, so on Linux it runs under
|
||||
Mono, and Mono's `System.Drawing` is a thin layer over **libgdiplus**. §4.2 put `System.Drawing` in
|
||||
the decode path, so without that library a Linux shard cannot extract art at all — the cliloc table
|
||||
(§9) and the atlas files (§10) are unaffected, since neither touches pixels.
|
||||
|
||||
**Windows shard hosts need nothing.** `System.Drawing` ships with .NET Framework. This is a
|
||||
Linux-only prerequisite and most shards will never read this section.
|
||||
|
||||
| Host | Get it with |
|
||||
|---|---|
|
||||
| Debian / Ubuntu | `sudo apt-get install libgdiplus` — in Debian since bullseye (6.0.4) and bookworm/trixie (6.1), and in Ubuntu universe |
|
||||
| Fedora / RHEL | `sudo dnf install libgdiplus` (EPEL or the Mono repository) |
|
||||
| Docker | `RUN apt-get update && apt-get install -y libgdiplus` in the shard image |
|
||||
| Alpine, or a distro with no package | Build from source — see the repository below. This is the awkward case and is worth avoiding by choosing a Debian-based image |
|
||||
|
||||
Upstream is <https://github.com/mono/libgdiplus>, with the Mono project's own page at
|
||||
<https://www.mono-project.com/docs/gui/libgdiplus/>.
|
||||
|
||||
**One thing to know before depending on it: that repository was archived in March 2025 and is
|
||||
read-only.** Distributions still package and patch it, so `apt-get install libgdiplus` is a normal,
|
||||
supported thing to do today — but upstream is not maintained. It is the strongest long-term argument
|
||||
for eventually moving extraction off `System.Drawing`, and phase 4's UOP reader (§4.3) is written
|
||||
without it precisely so that door stays open. It does not change the decision now.
|
||||
|
||||
**How its absence must present.** Never a stack trace and never a 500. Missing libgdiplus is a
|
||||
named, actionable outcome in the same family as the cliloc reader's `COMPRESSED`:
|
||||
|
||||
```
|
||||
status: unavailable
|
||||
code: NO_IMAGING
|
||||
reason: This shard host cannot render images — Mono's System.Drawing needs libgdiplus.
|
||||
Install it (apt-get install libgdiplus) and re-run the import. Cliloc and
|
||||
atlas import are unaffected.
|
||||
```
|
||||
|
||||
The installer's `doctor` checks for it and reports it alongside its other host checks, so an
|
||||
operator learns about this while setting the shard up rather than from an empty bestiary weeks
|
||||
later.
|
||||
|
||||
---
|
||||
|
||||
### 4.5 What phase 0 measured, and the rule it produced
|
||||
|
||||
Phase 0 ran §4's decoders from inside a live ServUO 57.4 against a client broken in 21 catalogued
|
||||
ways ([`servuo-plugins/tools/scaffolding/README.md`](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/src/branch/main/tools/scaffolding/README.md)
|
||||
carries the full results). §4's decision **stands** — nothing faulted on a path this protocol
|
||||
calls, and §9's cliloc reader reproduced UOFiddler's 123,490-entry table byte for byte in 218 ms.
|
||||
|
||||
But the spike was looking for the wrong kind of failure, and found a worse one.
|
||||
|
||||
**A malformed record does not usually throw. It renders the previous asset.** `LoadStatic` and
|
||||
`LoadLand` decode out of `m_StreamBuffer`, which is reused across calls, only ever grown, and
|
||||
filled by a `stream.Read` whose return value is discarded. So a record that is short, absent or
|
||||
out of bounds produces a real bitmap of whatever was decoded last — reported as success by every
|
||||
count in the library and undetectable by anything downstream.
|
||||
|
||||
On the **stock, unmodified** client on this machine that is **22,102 ids**: 9,962 statics and
|
||||
12,140 land tiles whose index entry reads `lookup 0, length 0`. `FileIndex.Seek` rejects
|
||||
`lookup < 0` and `length < 0`, and zero is neither, so it treats an empty slot as a hit. A bulk
|
||||
import that trusted the library would have written 22,102 duplicate images into the site under
|
||||
ids that have no art. §1.1's "32,766 decoded" was counting these.
|
||||
|
||||
**This is specific to the UOP path** (see §4.6), because `artidx.mul` stores `-1` for an absent
|
||||
record where an unmapped UOP slot is a zeroed struct.
|
||||
|
||||
So the rule, and it is the deliverable phase 1 inherits:
|
||||
|
||||
> **Validate before calling.** The extractor judges an index entry — and, for statics, the record
|
||||
> header and row table behind it — *before* handing the id to `Ultima`. A record that fails is
|
||||
> reported as absent, never decoded.
|
||||
|
||||
The checks, all of which phase 0 implemented and measured as `BridgeAssetValidator`:
|
||||
|
||||
| Check | The shape it stops |
|
||||
|---|---|
|
||||
| `lookup >= 0`, `length > 0` | the 22,102 empty slots above |
|
||||
| `lookup + length <= <data file>.Length` | **`Seek` checks that a record *starts* inside the file and never that it *ends* inside it**; a short read then decodes the previous asset |
|
||||
| the same bound against `verdata.mul` for a patched entry | `Verdata.Seek` is bounds-checked **nowhere at all** |
|
||||
| land records are at least 2,024 bytes | `LoadLand` reads exactly that many whatever the length says |
|
||||
| declared width and height within a ceiling | `LoadStatic` allocates `new Bitmap(width, height)` from two bytes in the file — phase 0 got a ~128 MB allocation out of an edit, and the same field can ask for 8 GB |
|
||||
| walk the row table and every run, bounded | `LoadStatic`'s two guards bound the **write** into the bitmap and nothing bounds the **read** out of the record |
|
||||
|
||||
Measured against the patched client, this refused all eight record-level defects, seven of which
|
||||
the library rendered without raising anything. Measured against the stock client it refused
|
||||
**nothing** across 49,151 statics and 16,384 land tiles. That second number is the one that makes
|
||||
the boundary defensible: a checker that refuses real art would be worse than no checker.
|
||||
|
||||
Two more ways an id with no art yields a picture, both of which the extractor must handle itself:
|
||||
`Art.GetStatic(id, false)` **throws** `IndexOutOfRangeException` above the index's own ceiling
|
||||
(16,385 ids in a full sweep), and `Art.GetStatic(id)` with the default `checkmaxid: true` is worse
|
||||
— `GetLegalItemID` maps an out-of-range id to **0** and returns **item 0's picture**. Take the
|
||||
ceiling from the index that was opened, and pass `checkmaxid: false` so an overrun is loud.
|
||||
|
||||
**The animation path has none of this yet**, and phase 0 proved it needs it: the patched client's
|
||||
verdata entry for body 34 points past `verdata.mul`'s end and the wolf still "decoded", counted
|
||||
among the 1,144 successes while rendering something else. `GetAnimation` additionally allocates
|
||||
`new int[frameCount]` straight from a file-supplied int. Extending the validator to animations is
|
||||
phase 1 work, not a phase 9 tidy-up.
|
||||
|
||||
### 4.6 The UOP wins outright, and `art.mul` is never opened
|
||||
|
||||
`FileIndex`'s UOP constructor ends with a bare `MulPath = uopPath`. **When `artLegacyMUL.uop` is
|
||||
present it wins, and `art.mul` / `artidx.mul` are not opened at all.** Every current client ships
|
||||
the UOP, so this is the normal case and not an edge one.
|
||||
|
||||
It matters twice.
|
||||
|
||||
**For us:** an index entry's `lookup` is an offset into whichever file `FileIndex` resolved, so any
|
||||
bound taken against `art.mul` while the index holds UOP offsets is not approximate — it is
|
||||
meaningless. Phase 0's first run refused **34,299 perfectly good statics** for "declaring
|
||||
10533x2085" on exactly that mistake, and every one of those refusals read like a real finding. The
|
||||
extractor must resolve the data file with `FileIndex`'s own precedence, not by name.
|
||||
|
||||
**For operators:** a shard that adds custom art to `art.mul` while the UOP is still in place gets
|
||||
nothing, silently. Worth a line in the admin surface's diagnostics (§14) rather than leaving an
|
||||
operator to discover it as "my art did not import".
|
||||
|
||||
### 4.7 The gump crash, reproduced where it counts
|
||||
|
||||
§4.1 argued from source that the access violation lives on the `hasExtra: true` branch only `Gumps`
|
||||
reaches. Phase 0 called `Ultima.Gumps.GetGump(2)` **once, from inside a running shard**: the ServUO
|
||||
process disappeared. No exception line, no `catch` reached, no shutdown, nothing in the console —
|
||||
the probe's checkpoint file, written before the call, was the entire record of what happened.
|
||||
|
||||
`AccessViolationException` is a corrupted-state exception that .NET Framework 4.8 does not deliver
|
||||
to ordinary handlers, so on a live shard this is a crash with players on it and **there is no
|
||||
in-process defence**. §4.1's "nothing in this protocol calls `Ultima.Gumps`" is therefore an earned
|
||||
safety rule rather than a scoping preference, and adding gump art later means fixing that path
|
||||
first, deliberately.
|
||||
|
||||
---
|
||||
|
||||
## 5. Addressing: one key for every asset
|
||||
|
||||
Every asset the bridge can serve is named by a single string key, and the key is the cache key,
|
||||
the hash key, the filename stem and the manifest row id:
|
||||
|
||||
```
|
||||
static/3922 one item graphic
|
||||
static/3922/h33 the same graphic, hue 33 applied
|
||||
land/3 one land tile
|
||||
body/34/a0 creature body 34, action 0, first frame
|
||||
body/400/a0/f0..f9 human male, action 0, all ten frames
|
||||
cliloc/enu the whole converted string table (not an image)
|
||||
tree/Spawns/Trammel.xml a ServUO tree file (§10)
|
||||
```
|
||||
|
||||
Three properties this shape buys:
|
||||
|
||||
- **Hue is part of the key, not a transform.** `itemId` and `hue` are already on the wire together
|
||||
(`BridgeMarket.cs:582`, `BridgeProfile.cs:314`), so a marketplace listing already knows the exact
|
||||
key for its own picture. Applying hues website-side would mean shipping `Hues.mul` semantics into
|
||||
Node for no gain.
|
||||
- **Depth is expressible without being mandatory.** `body/400/a0` and `body/400/a0/f0..f9` are the
|
||||
same addressing scheme at two depths, which is what lets §11 bulk-import thumbnails and fetch full
|
||||
animations on demand without a second protocol.
|
||||
- **Nothing in the key is client-version-specific**, so a client patch changes an asset's *bytes*,
|
||||
not its name — which is what makes §7's delta work.
|
||||
|
||||
### 5.1 There is no direction segment, because only one direction is wanted
|
||||
|
||||
Bodies are stored in **five** directions and the client mirrors three of them to reach eight. Only
|
||||
one is needed here, so **direction is fixed by the extractor and is not part of the key**. Leaving
|
||||
it in would advertise a choice nobody is going to vary and would five-fold every count in §11 for
|
||||
nothing.
|
||||
|
||||
**Which one depends on whether the body is a player character:**
|
||||
|
||||
| Body | Direction | Why |
|
||||
|---|---|---|
|
||||
| A player character body | **0** — head-on, facing the viewer | A character is a portrait; it should look at you |
|
||||
| Everything else | **1** — front three-quarter | The view that actually reads as a creature (see the caveat below) |
|
||||
|
||||
Which index is which was determined by **rendering all five** for a human, a wolf and a dragon
|
||||
rather than from a table, because the answer is not obvious and the small-thumbnail version of the
|
||||
same test suggested the exact opposite:
|
||||
|
||||
| Index | View |
|
||||
|---|---|
|
||||
| **0** | **Head-on, facing the viewer** — face, chest and front legs visible |
|
||||
| 1 | Front three-quarter |
|
||||
| 2 | Full side profile |
|
||||
| 3 | Rear three-quarter |
|
||||
| 4 | Directly away — back of the head, and a quadruped's tail toward the camera |
|
||||
|
||||
The caveat the render made obvious is what produced the split: **index 0 is the least legible view
|
||||
for four-legged and long-bodied creatures.** A wolf seen head-on is a dark blob; the same wolf at
|
||||
index 1 is unmistakably a wolf, which is also why UOFiddler's own thumbnail list picks that view. A
|
||||
humanoid has no such problem — it reads fine head-on, and head-on is what a character portrait
|
||||
wants.
|
||||
|
||||
Both indices stay **configuration values** (defaulting to 0 and 1), so changing the catalogue's mind
|
||||
later is a setting and a re-import, not a protocol change.
|
||||
|
||||
### 5.2 "Player character body" is asked of the shard, never hardcoded
|
||||
|
||||
`Server.Race.AllRaces` gives every registered race, and each carries `MaleBody`, `FemaleBody`,
|
||||
`MaleGhostBody` and `FemaleGhostBody`. The plugin enumerates those four ids per race and that set —
|
||||
nothing else — takes index 0. On stock ServUO 57.4 that is twelve ids:
|
||||
|
||||
| Race | Male | Female | Male ghost | Female ghost |
|
||||
|---|---|---|---|---|
|
||||
| Human | 400 | 401 | 402 | 403 |
|
||||
| Elf | 605 | 606 | 607 | 608 |
|
||||
| Gargoyle | 666 | 667 | **695** | **694** |
|
||||
|
||||
This is the §8 argument again in miniature: only code inside ServUO can answer it, and asking is
|
||||
the only thing that works on a shard with a custom race. Two details make the case that a
|
||||
hardcoded list would have been wrong — `RaceDefinitions.cs` passes the gargoyle's ghost bodies in
|
||||
the **opposite order** to the other two races (695 male, 694 female), and a shard that calls
|
||||
`RegisterRace` adds ids no table of ours would contain.
|
||||
|
||||
**Half of that set does not decode with ServUO's vendored library.** Measured:
|
||||
|
||||
| Decodes | Does not |
|
||||
|---|---|
|
||||
| Human male/female (400, 401) | **Human ghosts (402, 403)** |
|
||||
| Elf male/female (605, 606) | **Every gargoyle body (666, 667, 694, 695)** |
|
||||
| Elf ghosts (607, 608) | |
|
||||
|
||||
Six of twelve, including a whole playable race. These are UOP-only and **§4.3 adds a decoder for
|
||||
them** — verified genuinely absent from the legacy files rather than merely mis-addressed, by
|
||||
rendering what the other anim files hold at those indices and finding spiders.
|
||||
|
||||
Until that phase lands, and on any client that lacks them entirely, the catalogue must **not** treat
|
||||
a missing player body as an error. It is the expected answer for half the set, and a status screen
|
||||
that flags six failures on every import teaches an operator to ignore it.
|
||||
`shard_spawn_creatures.art` staying NULL remains a first-class state everywhere it is consumed,
|
||||
which it already is.
|
||||
|
||||
---
|
||||
|
||||
## 6. The manifest, and what the two buttons actually do
|
||||
|
||||
Two stages, which is where **Import** and **Update** come from.
|
||||
|
||||
**Stage 1 — the source gate.** The shard reports a manifest of the client files themselves:
|
||||
size, mtime and content hash of `Cliloc.enu`, `anim*.idx`/`anim*.mul`, `art.mul`/`artidx.mul`,
|
||||
`Body.def`, `Bodyconv.def`, `Hues.mul`. Unchanged since the last import, and nothing else happens.
|
||||
This is the same hash gate the spawn atlas and the cliloc table already use, and for the same
|
||||
reason: the normal case is a restart that changed nothing, and it must cost nothing.
|
||||
|
||||
`anim.mul` is 195 MB and `art.mul` is 148 MB, so the gate is **(size, mtime) first, content hash
|
||||
only when those differ** — a full hash of 343 MB on every status poll would make the admin panel
|
||||
feel broken.
|
||||
|
||||
**Stage 2 — the asset manifest.** For the working set (§11), the shard streams
|
||||
`[{ key, sha256, bytes }]` — no pixels. The website diffs that against what it holds and requests
|
||||
**only the keys whose hash changed**.
|
||||
|
||||
- **Update** = stage 1, then stage 2, then fetch the diff.
|
||||
- **Import** = the same path with the diff skipped and every key fetched.
|
||||
- **A key that has vanished** from the manifest is staged for review, never applied silently —
|
||||
the same rule, and the same reasoning, as a vanished cliloc source or a disappearing atlas
|
||||
facet. An unmounted volume and a deliberate client downgrade look identical from here.
|
||||
|
||||
Clilocs are the exception and stay a **whole-table replace** whenever the file hash changes:
|
||||
the measured cost is 663 ms for 67,496 rows, so per-entry deltas would be complexity bought for
|
||||
nothing.
|
||||
|
||||
---
|
||||
|
||||
## 7. The parser version applies here too
|
||||
|
||||
`spawnAtlasSource.js` carries `PARSER_VERSION` (currently 5) and the cliloc source carries its own,
|
||||
both counted as drift so that a corrected parse reaches an install whose files never change. The
|
||||
asset pipeline inherits the rule and needs it more, not less: a fixed hue application or a
|
||||
corrected frame offset changes the bytes we derive from files that are byte-identical.
|
||||
|
||||
**`EXTRACTOR_VERSION` lives in the plugin**, because the plugin is what derives the bytes, and it
|
||||
is folded into stage 1's gate. Bumping it makes every asset drift, which is correct.
|
||||
|
||||
---
|
||||
|
||||
## 8. Body ids: the part only the shard can do
|
||||
|
||||
The atlas knows creatures by **slug**, derived from type names in `Spawns/*.xml`. The client knows
|
||||
them by **body id**. Nothing in the ServUO tree declares the mapping as data — today an operator
|
||||
bridges it by grepping `Scripts/Mobiles/Normal/<Name>.cs` for `Body =`, which appears variously as
|
||||
a decimal, as hex (`0xD1`), as `Utility.RandomList(35, 36)`, and as an `m_IDs[]` table.
|
||||
|
||||
Inside ServUO the problem does not exist. `BridgeWorld.cs:350` already does exactly the required
|
||||
thing for a different feature:
|
||||
|
||||
```csharp
|
||||
var type = ScriptCompiler.FindTypeByName(name, true);
|
||||
var creature = Activator.CreateInstance(type) as BaseCreature;
|
||||
```
|
||||
|
||||
Construct, read `creature.Body.BodyID`, `Delete()`. Authoritative, no source parsing, and correct
|
||||
for custom creatures a grep would never find.
|
||||
|
||||
**This pass must run on the Core thread** — it constructs and deletes mobiles, which is world
|
||||
mutation — while the decode in §4 must run **off** it. That split is the one genuinely new
|
||||
threading shape in this protocol, and it is why slug→body resolution is its own request kind with
|
||||
its own (small) batch size rather than a step inside asset extraction.
|
||||
|
||||
Constructing arbitrary creature types has side effects: constructors pack items, set skills, start
|
||||
timers. The mitigations are per-type `try`/`catch`, immediate `Delete()`, small batches, and the
|
||||
fact that the whole pass is admin-triggered rather than something that runs at boot.
|
||||
|
||||
---
|
||||
|
||||
## 9. The cliloc decompressor is ours now
|
||||
|
||||
Every modern client ships `Cliloc.*` in the Mythic compressed container — this machine's
|
||||
`Cliloc.enu` is 4,989,921 bytes beginning `E8 79 67 8E`, high byte `0x8E`. ServUO's bundled
|
||||
`Ultima.StringList` implements only the plain layout and throws on it (§1.1), which is also why
|
||||
the shard's own `VendorSearch.GetItemName` is already inert.
|
||||
|
||||
**UOFiddler is released under the Beerware licence**, so porting its decompressor into our
|
||||
GPL-3.0-or-later tree is clean. It lands in the overlay as ordinary C# — the **only** decoder
|
||||
Protocol 8 writes rather than calls (§4) — and from that point:
|
||||
|
||||
- No operator installs UOFiddler.
|
||||
- No operator runs `dotnet build` on a converter.
|
||||
- No operator copies a 5 MB file to a server.
|
||||
- `website/server/tools/cliloc-export/` is retired, and `UOFIDDLER.md` is deleted rather than
|
||||
rewritten.
|
||||
|
||||
**What survives untouched is the `custom/` overlay mechanism.** Shard-added items carry cliloc ids
|
||||
no client table has, and ServUO has no server-side notion of a custom cliloc — that is a real gap
|
||||
in the *game*, not an artefact of the manual pipeline, and `CLILOCS.md`'s reasoning for it stands.
|
||||
The base table now arrives over the bridge; overlays still come from a directory the site reads.
|
||||
Measured on the live shard: 16,434 cliloc ids referenced by the script tree, 37 absent from stock.
|
||||
|
||||
---
|
||||
|
||||
## 10. The atlas stops needing a shared filesystem
|
||||
|
||||
Today `SPAWN_ATLAS.md` requires the **website** to read the ServUO tree — "same host, a bind mount,
|
||||
or a shared volume". That is the one place the platform's own rule (only the sidecar bridges the
|
||||
shard) is broken, and it is broken by the component that faces the internet.
|
||||
|
||||
The same transport closes it. `spawnAtlasSource.js` already labels every file it reads with a
|
||||
portable key:
|
||||
|
||||
| Label | Count (stock 57.4) |
|
||||
|---|---|
|
||||
| `Data/Regions.xml` | 1 |
|
||||
| `Data/Locations/*.xml` | 6 |
|
||||
| `Spawns/*.xml` | 13, ~10.5 MB |
|
||||
| `Config/ChampionSpawns.xml` | 1 |
|
||||
| `Data/Decoration/**` | tree |
|
||||
|
||||
So the shard serves `tree/<label>` → bytes over the same batched request/reply path, and
|
||||
`spawnAtlasSource.js` gains a second backend behind its existing interface: **filesystem** (today,
|
||||
kept for same-host installs and for development) or **sidecar** (new, and the default once
|
||||
configured).
|
||||
|
||||
**The parsers do not move.** `spawnAtlasParse.js` is pure, fs-free and CI-covered without a ServUO
|
||||
tree, and every quirk it handles — the two respawn delay units, `:OBJ=` splitting, facet-name
|
||||
reconciliation, the XmlSpawner directive stripping — stays exactly where it is. The shard sends
|
||||
bytes; the website still decides what they mean. That is the same division as §2, and it keeps the
|
||||
sidecar a forwarder here too.
|
||||
|
||||
`SERVUO_PATH` and the `spawn_atlas_servuo_path` setting remain, and select the filesystem backend.
|
||||
|
||||
---
|
||||
|
||||
## 11. What is bulk and what is on demand
|
||||
|
||||
The scope approved is creature art, item art, player models "and everything", against a future
|
||||
project. §1.1's measurements make the sizing question concrete:
|
||||
|
||||
| Kind | Addressable | Bulk? |
|
||||
|---|---|---|
|
||||
| Item statics | **~49,150** | No — on demand, cached, keyed by `itemId` (+ hue) |
|
||||
| Land tiles | **16,384** | No — on demand |
|
||||
| Creature/player bodies, first frame | **1,144** | **Yes** — this is the catalogue |
|
||||
| One body, every action, one direction | **210 frames** (body 400); 96–210 measured across six bodies | No — on demand, per body |
|
||||
| All bodies, every action, one direction | **~173,000 frames**, ~170 MB | No — but no longer unthinkable |
|
||||
| The same at five directions | ~865,000 frames | Not built (§5.1) |
|
||||
| Cliloc table | 123,490 entries → 67,496 rows | **Yes** — whole-table replace |
|
||||
| ServUO tree files (§10) | ~21 files, ~10.6 MB | **Yes** |
|
||||
|
||||
**The working set is one thumbnail per body, plus the atlas's own creatures.** 1,144 sprites at
|
||||
roughly a kilobyte each is under 2 MB — trivial to import, trivial to re-hash, and it is the set
|
||||
that makes a bestiary, a marketplace listing and a character sheet render.
|
||||
|
||||
Everything deeper is the *same protocol at a deeper key* (§5), fetched on demand and cached. That
|
||||
is what serves the future project without exporting 3.5 GB of someone else's copyrighted client
|
||||
into a database: a viewer that wants body 400's full walk cycle asks for `body/400/a2/f0..f9` and
|
||||
gets it, once, and it is cached from then on.
|
||||
|
||||
Because §5.1 dropped four of the five directions, a **complete** one-direction animation set for
|
||||
every body is now ~173,000 frames rather than ~865,000 — around 170 MB. That is still not the
|
||||
default and still not something to import before anything asks for it, but it has moved from
|
||||
"never" to "a thing an operator could reasonably choose", and phase 6 should leave room for a
|
||||
bulk-fill-everything switch rather than assuming on-demand is the only mode.
|
||||
|
||||
**Hued variants are on demand, always.** `static/3922/h33` is generated when something on the wire
|
||||
actually carries hue 33. The cross product of 49,150 statics and ~3,000 hues is not a set anyone
|
||||
enumerates.
|
||||
|
||||
**Gump art is out of scope for Protocol 8, and that is now a safety rule rather than a priority
|
||||
call** — §4.1. It is the only decoder that reaches the `hasExtra: true` branch, and that branch
|
||||
corrupts the process on the second id. Adding gump art later means fixing that path first,
|
||||
deliberately; it is additive under the same key scheme (`gump/<id>`), which is the point of §5.
|
||||
|
||||
---
|
||||
|
||||
## 12. Where it lands on the website
|
||||
|
||||
Images are written by module-uo into the upload directory. `ctx.uploads`
|
||||
(`{ upload, UPLOAD_DIR, MIME_EXT }`) is **already** exposed to modules and
|
||||
[`MODULE_API.md:626`](../website/MODULE_API.md) already names its consumer as "atlas art import",
|
||||
so no `MODULE_API_VERSION` bump is needed to store them.
|
||||
|
||||
- `shard_spawn_creatures.art` stops being NULL-by-default and starts being filled by the import.
|
||||
- A new asset table carries `key`, `sha256`, `bytes`, `width`, `height`, `imported_at` — the
|
||||
manifest side of §6, and what makes an Update a diff rather than a re-download.
|
||||
- **The operator-supplied `spawnAtlas.art.json` map stays supported** and continues to win over an
|
||||
imported asset. An operator who has drawn their own creature portraits must not have them
|
||||
overwritten by a sprite rip on the next Update.
|
||||
|
||||
Licensing is unchanged and the reasoning is unchanged: these are the operator's own client files,
|
||||
extracted on their own host, for their own shard. Nothing is committed, nothing ships in a repo,
|
||||
and nothing is redistributed. What changes is only that the extraction stopped requiring a GUI on a
|
||||
desktop.
|
||||
|
||||
---
|
||||
|
||||
## 13. Visibility
|
||||
|
||||
New surfaces over shard data are admin-toggleable with an operator-set audience, and this is no
|
||||
exception. Asset serving is gated like every other shard read: a `requireFeature` gate, an
|
||||
audience, and 404-not-403 when the feature is off, so a disabled feature does not advertise itself.
|
||||
|
||||
The default is the least surprising one: assets are as public as the page that uses them. A
|
||||
bestiary that is already anonymous does not become staff-only because its pictures arrived over a
|
||||
new pipe.
|
||||
|
||||
---
|
||||
|
||||
## 14. Routes and commands added
|
||||
|
||||
**Loopback (shard ↔ sidecar), all request/reply:**
|
||||
|
||||
| Command | Reply | Purpose |
|
||||
|---|---|---|
|
||||
| `assets.sources` | `assets.sources.ok` | Stage 1: client file manifest + `EXTRACTOR_VERSION` |
|
||||
| `assets.manifest` | `assets.manifest.ok` | Stage 2: `[{key, sha256, bytes}]`, paged |
|
||||
| `assets.fetch` | `assets.fetch.ok` | Content for an explicit key list, paged |
|
||||
| `assets.bodies` | `assets.bodies.ok` | Slug → body id (§8, Core thread) |
|
||||
| `cliloc.table` | `cliloc.table.ok` | The converted table, paged |
|
||||
| `tree.manifest` / `tree.fetch` | `.ok` | §10, the ServUO tree files |
|
||||
|
||||
**Sidecar REST** mirrors those one for one under `/assets/*`, `/cliloc`, `/tree/*`, carrying
|
||||
`X-UOLink-Version: 8` and forwarding verbatim.
|
||||
|
||||
**Website admin** (`Admin → Shard`, admin-only): status, **Import**, **Update**, approve/reject for
|
||||
a vanished key, and the existing path settings. Every action to the admin activity log, as
|
||||
`shard.assets.*`.
|
||||
|
||||
---
|
||||
|
||||
## 15. Cross-repo obligations
|
||||
|
||||
`PROTOCOL_VERSION` goes **7 → 8** in `link/sidecar/src/main.rs`, and in the **same PR**
|
||||
`servuo-plugins/overlay.toml` — the installer refuses to pair a sidecar and an overlay that
|
||||
disagree, so a split bump means the next bundle silently fails to compose.
|
||||
|
||||
| Repo | Work |
|
||||
|---|---|
|
||||
| `servuo-plugins/` | Extraction over ServUO's own `Ultima` (§4), the cliloc decompressor (§9), body resolution (§8), the request handlers, `overlay.toml` |
|
||||
| `link/` | Six command families forwarded, the REST surface, **the inbound line cap (§3.3)**, `PROTOCOL_VERSION` |
|
||||
| `module-uo/` | Client calls, asset store, the atlas source backend (§10), cliloc ingest, admin surface |
|
||||
| `website/` | None expected — `ctx.uploads` already suffices (§12) |
|
||||
| `docs/` | This file; rewrite `CLILOCS.md` §Converting and `SPAWN_ATLAS.md` §Artwork + §Configuring; **delete `UOFIDDLER.md`**; add the libgdiplus prerequisite to `SHARD_PREREQS.md` (§4.4) |
|
||||
| `installer/` | A `doctor` check for libgdiplus on Linux hosts (§4.4). Bundle pairing already enforces §15 |
|
||||
| `android-app/` | Consumes images by URL; no parity gate expected until a screen shows one |
|
||||
| `integration-kit/` | A chapter note only — this is UO-specific and teaches nothing about the module contract |
|
||||
|
||||
---
|
||||
|
||||
## 16. Phases
|
||||
|
||||
| # | Scope | Repos |
|
||||
|---|---|---|
|
||||
| 0 | **DONE 2026-09-10.** Spike: the vendored decoders driven from inside a running ServUO, over a client broken 21 ways. §4 stands; the finding was 22,102 wrong pictures on a *stock* client, and the validator that answers them (§4.5-§4.7) | servuo-plugins |
|
||||
| 1 | The transport: `assets.sources`, flow control, the sidecar line cap, `EXTRACTOR_VERSION`, protocol bump. **Plus §4.5's validator promoted into the overlay, extended to animations** — phase 0 left that half unbuilt and proved it is needed | servuo-plugins, link |
|
||||
| 2 | Clilocs end to end; retire the converter and `UOFIDDLER.md` §Part 1 | all |
|
||||
| 3 | Body resolution (§8) + the 1,144-body catalogue; `shard_spawn_creatures.art` filled | servuo-plugins, module-uo |
|
||||
| 4 | **The UOP animation decoder (§4.3)** — the six player bodies the legacy path cannot reach, and the never-sweep-file-types rule enforced in code | servuo-plugins |
|
||||
| 5 | Item statics and land on demand, hued keys, the cache | servuo-plugins, module-uo |
|
||||
| 6 | Deep animation keys (`body/<id>/a<n>/f<n>`) for the future project, plus the bulk-fill switch | servuo-plugins, module-uo |
|
||||
| 7 | The atlas over the sidecar (§10); shared-filesystem requirement retired | module-uo |
|
||||
| 8 | Admin surface, Import/Update, approve/reject, activity log | module-uo |
|
||||
| 9 | Docs pass across five repos; live walk on the real rig | docs |
|
||||
|
||||
Phase 0 exists because §4 chose to call code that can take the shard down if it is wrong, and the
|
||||
honest way to hold that choice is to try to break it on purpose — in the real host process, against
|
||||
a client that has been patched — *before* building eight phases on top of it. The probes behind
|
||||
§1.1 were run from PowerShell against a stock client; neither of those is the environment this will
|
||||
actually run in.
|
||||
|
||||
Phase 4 sits **after** the catalogue rather than inside it on purpose. The catalogue is useful with
|
||||
1,138 of 1,144 bodies, the UOP reader is the one piece of genuinely new format work in this
|
||||
protocol, and putting it on the critical path would hold up every website-side phase behind it. Its
|
||||
acceptance test is not "it decodes" — it is that a gargoyle looks like a gargoyle, checked by eye,
|
||||
because §4.3's whole point is that this failure mode produces confident, wrong pictures.
|
||||
|
||||
---
|
||||
|
||||
## 17. Decisions
|
||||
|
||||
Every item here is settled. Each is recorded because it changes numbers or obligations elsewhere
|
||||
in the document.
|
||||
|
||||
1. **§4: settled 2026-09-10 — call ServUO's vendored `Ultima`,** with one exception added the same
|
||||
day: **§4.3's UOP animation decoder**, scoped to the player bodies the legacy path cannot reach
|
||||
and to nothing else (phase 4). The crash is confined to the `hasExtra: true` branch that only
|
||||
`Gumps` reaches, and nothing here calls `Gumps`. **Phase 0 confirmed both halves in the real
|
||||
host process** — nothing faulted on a path we call, and one `GetGump(2)` killed the shard
|
||||
outright (§4.7).
|
||||
2. **§4.4: settled 2026-09-10 — libgdiplus is a stated requirement** on Linux shard hosts, with
|
||||
all three answers taken rather than one: it goes in `SHARD_PREREQS.md`, the installer's `doctor`
|
||||
checks it, and its absence degrades to a named `NO_IMAGING` status instead of an error. Windows
|
||||
hosts are unaffected.
|
||||
3. **§5.1/§5.2: direction — settled 2026-09-10.** Player character bodies use index 0, everything
|
||||
else index 1, direction is not in the key, and the player-body set is enumerated from
|
||||
`Race.AllRaces` rather than hardcoded. Recorded here because it changes every count in §11.
|
||||
Phase 0 reproduced the twelve stock ids and the six that do not decode, exactly.
|
||||
4. **§13: the default audience — settled 2026-09-10. An asset inherits the audience of the page
|
||||
that uses it.** A bestiary that is already anonymous keeps anonymous pictures; a staff-only
|
||||
screen's art is staff-only. The operator can still set the policy explicitly, and the
|
||||
`requireFeature` gate with its 404-not-403 behaviour is unchanged. The alternative — one flat
|
||||
audience for all asset serving — was rejected because it necessarily disagrees with some page
|
||||
that uses it, in one direction or the other, and the disagreement is silent either way.
|
||||
5. **§4.5: the response to a malformed record — settled 2026-09-10, and now measured. Validate
|
||||
before calling.** The extractor judges an index entry, and for statics the record behind it,
|
||||
before handing the id to `Ultima`; a record that fails is reported absent and never decoded.
|
||||
Chosen ahead of phase 0 over two alternatives — extracting in a child process (much stronger
|
||||
containment, a much larger change to §2 and phase 1) and reversing §4 to write our own decoders
|
||||
(weeks, per §2.1). Phase 0 then found the shape that settles it: the dangerous failure is not a
|
||||
crash a child process would contain, it is a **wrong picture** that no containment strategy
|
||||
would have caught, on **22,102 ids of a stock client**. See §4.5 for the checks and the
|
||||
false-refusal measurement that says the boundary is in the right place.
|
||||
@@ -1847,7 +1847,7 @@ no URL moved.
|
||||
| `DELETE /admin/events/series/:seriesId` | admin, editor | delete it, detaching its definitions; answers with how many |
|
||||
| `GET /admin/events/calendar` | staff | the calendar for a window: materialised runs and projected occurrences (Phase 4) |
|
||||
| `GET/PUT /admin/events/actions` | admin | which actions are enabled on this deployment, and their per-run caps (Phase 6). `admin` on the read as well as the write; the PUT takes one action at a time |
|
||||
| `GET /public/events` | — | **the calendar** (Phase 14a): upcoming, live and recent, by series. Runs and projections interleaved and each saying which it is, ascending by instant. Instants are UTC and every entry carries the EVENT's own zone; the reader's zone places them. Rehearsals and unlisted events are absent. A run is an INTERVAL, not an instant: an entry is in the window when the run OVERLAPS it, so one that began before the window and has not ended is still "what is on" (Phase 16a — reading the start instant alone made this route serve only the first of its three words, while the event's own page said `live`). Defaults to seven days back through 31 days out — the tail is where "recent" lives — and the window may span at most 92; the anonymous surface is the one with no login in front of it. Projections are forecast from NOW, never into the tail, since a slot the runner has already passed did not happen |
|
||||
| `GET /public/events` | — | **the calendar** (Phase 14a): upcoming, live and recent, by series. Runs and projections interleaved and each saying which it is, ascending by instant. Instants are UTC and every entry carries the EVENT's own zone; the reader's zone places them. Rehearsals and unlisted events are absent. Defaults to now through 31 days out and the window may span at most 92 — the anonymous surface is the one with no login in front of it |
|
||||
| `GET /public/events/:slug` | — | **one event** (Phase 14a): storyline, arc, what is live, what is next, what happened recently, and a results table once one is published. Takes an optional `?run=`, which is what an announcement's link carries, so a mail about last Friday's occurrence does not open next Friday's; a run belonging to some other event is **ignored rather than refused**, because a stale link in a months-old mail should land on the event it was about. A draft, an archived definition and an unlisted one all answer 404 |
|
||||
| `GET /public/events/series/:slug` | — | **the arc** (Phase 14a). A series with no listed events is a 404, not an empty page: the arc is a label on its definitions, so a page for an empty one would publish the fact that an operator has named something they have not announced |
|
||||
| `GET /player/events/history` | auth | **this account's participation** (Phase 14a) — the run, when it was, the score a module reported, and the rank once results were published (null until then, which is a real state rather than an error). Self-scoped on the session with **no id parameter**, deliberately: a route that took one would be a middleware mistake away from publishing who attended what. Keyset-paged on the participation row's id. It obeys the calendar's two exclusions, so attending an unannounced event does not disclose that it exists |
|
||||
|
||||
@@ -15,14 +15,6 @@ UO actions, the integrations, the authoring UI, the public surface — needed no
|
||||
Those fourteen phases reach the game only to *announce*, over verbs the write plane already carries; nothing in them creates or
|
||||
changes a thing in the world.
|
||||
|
||||
**COMPLETE as of 2026-09-10.** All seventeen phases are built and on `main` in every repository they
|
||||
touch. P16 ran as three legs — **16a** the acceptance walk from `edge`, **16b** the six-step cutover
|
||||
and the re-verify against released artefacts, **16c** `runicgateway.com` and `.profile` — and 16c
|
||||
opened by closing the seventh cutover step 16b had left standing (`docs#232`). The platform the
|
||||
workstream leaves behind is sidecar **v2.2.0**, overlay **v1.2.0**, bundle **2026.09.10** on
|
||||
**protocol 7**, `Module-uo` **v1.2.2**, and `MODULE_API_VERSION` **1.10.0**. Each phase's record is
|
||||
in its own section below; `edge` stays standing, unused, in every repository.
|
||||
|
||||
---
|
||||
|
||||
## Before anything: three facts about the ground
|
||||
@@ -1828,280 +1820,12 @@ website, emulator — running a real multi-phase event, including three delibera
|
||||
Then `edge` → `main`, in the order every previous cutover used: the protocol side first, the module,
|
||||
core, docs, then the kit's re-pin and `runicgateway.com`.
|
||||
|
||||
> **Split into 16a (the walk) and 16b (the cutover)** (org lead, 2026-09-09), on the same argument
|
||||
> 12a/12b and 14a/14b were split on. The two sentences above cannot both hold: `link`,
|
||||
> `servuo-plugins` and `Module-uo` all release on push to **`main`**, so no released artefact
|
||||
> carrying events can exist until after the cutover. Engagement Phase 13 met the same wall and
|
||||
> resolved it the other way, cutting over first and walking from `main`. Here the walk goes first
|
||||
> against artefacts built from `edge` exactly the way a release builds them, because every walk in
|
||||
> this workstream has found defects and a defect found on `edge` is a reviewed PR rather than a
|
||||
> hotfix to `main`. **16b re-verifies against the real released bundle** — install, boot, run one
|
||||
> event — so the delivery path is still proved, just second. A third leg, **16c**, carries
|
||||
> `runicgateway.com` and `.profile`.
|
||||
>
|
||||
> **16a WALKED, and it is four repos** — `Module-uo`, `website`, `servuo-plugins`, `docs`. The whole
|
||||
> rig: real ServUO 57.4 (208k items, 42k mobiles) → a `cargo --release` sidecar on protocol 7 → core
|
||||
> with the module installed from a release-shaped bundle → the Android app on an emulator. The
|
||||
> overlay was deployed from a tarball built the way CI builds one, into a tree with `Scripts/Custom/
|
||||
> Bridge` and `Saves/Bridge` deleted first, so it was a first install rather than an upgrade.
|
||||
>
|
||||
> **All three deliberate failures pass.** (1) A mid-run process kill landed mid-TEARDOWN — sharper
|
||||
> than mid-step, since a phase executes in about a second — with the run `completed`, cleanup
|
||||
> `pending`, a lease half-returned and 21 world objects up: teardown resumed on restart and all 15
|
||||
> steps still read `attempts = 1`, so nothing re-executed. (2) The sidecar killed during a phase gate
|
||||
> left the run `degraded` rather than failed, `core.lease` retrying with a reason, and the four world
|
||||
> writes behind it **parked at `attempts = 0`**; the shard reconnected on its own. (3) A cap of 5
|
||||
> against a step asking for 12 answered `refused` — its own status — with `code: "cap"` and *"asks
|
||||
> for 12 of `uo.creatures`; 0 of 5 is already spent this run"*, and the dry run had already refused
|
||||
> it in the author's own words.
|
||||
>
|
||||
> **Six defects, all in code already merged to `edge`, and the suites were green on either side of
|
||||
> every one.** Two were blocking or worse. **The spawn atlas could not import on a stock ServUO
|
||||
> tree** — a case-sensitive JS dedupe against an `..._ai_ci` PRIMARY KEY, four colliding decoration
|
||||
> spellings in ServUO's own files, and the whole transaction lost; with no atlas every option source
|
||||
> answers empty and no world verb can be authored at all. **Teardown of all five world verbs was a
|
||||
> no-op that reported success** — `revertOwned` sent the despawn under the step's key, which is the
|
||||
> key the SPAWN used, so the shard replayed the spawn's reply and `OnDespawn` never ran; the ledger
|
||||
> read `reverted` while the shard held all 21 objects, and the same despawn under a fresh key removed
|
||||
> every one. Then: **the public calendar served neither live nor recent runs** though §I promises all
|
||||
> three, so the site said `live` on one page and showed nothing on the other; **a resource left
|
||||
> `reverting` by a crash was never reclaimed**, and the manual cleanup route answered 200 while doing
|
||||
> nothing, which stranded a lease and blocked the NEXT run of the same event; **a transient refusal
|
||||
> under an idempotency key was permanent**, because the shard's store had no case for a handler that
|
||||
> ran and deliberately did nothing; and **three facts every announcement computes were declared by no
|
||||
> trigger** and silently dropped.
|
||||
>
|
||||
> One reported defect was **withdrawn**: `skip` refusing a `failed` step is not a dead end, because
|
||||
> `resume` carries a run past any settled step — the route's own docs say so and the rig confirmed
|
||||
> it. The runner claims only `pending` steps, so `failed` and `refused` are both settled.
|
||||
>
|
||||
> Every fix is verified against the rig, not only against tests: the atlas imports 309 decor types
|
||||
> and 6,455 points; a full four-phase run's teardown leaves the shard owning **0**; a lease stranded
|
||||
> by a real crash is reclaimed in one sweep and `cleanup_status` reaches `complete`; the same save
|
||||
> key 25 seconds apart answers "15 seconds ago" then "40 seconds ago"; and `/site/events` shows a
|
||||
> live run as **Happening now** beside recent ones, in the browser and in the app. Each new test was
|
||||
> confirmed to FAIL without its fix.
|
||||
>
|
||||
> **`Module-uo`'s `revert` no longer forwards core's key at all** — `MODULE_API.md` now says why that
|
||||
> key identifies a lost dispatch rather than addressing the undo. **Protocol 7 is amended in place**:
|
||||
> a refusal releases its key, with the rule that pays for it written down — *do not answer `*.error`
|
||||
> after changing the world*.
|
||||
|
||||
> **16b CUT OVER (2026-09-09/10) — six steps, and core before the module.** The protocol pair
|
||||
> (`link#40` + `servuo-plugins#26`) is ONE step, not two: `bundle.yml`'s Gate 1 reads the protocol
|
||||
> number out of both released artefacts and refuses a pair that disagrees, so whichever lands first
|
||||
> leaves a compose that cannot run. Then core (`website#199`), the module carrying its own re-pin
|
||||
> (`Module-uo#34`), the app (`Android-app#46`), the kit's re-pin (`Integration-kit#11`), and docs.
|
||||
>
|
||||
> **Core lands before the module**, which departs from the sentence above and matches what the
|
||||
> engagement cutover actually did: `Module-uo`'s `ci/core-ref.json` has to name a website `main` sha
|
||||
> carrying MODULE_API 1.10.0, and that sha does not exist until core has landed. Four decisions, all
|
||||
> as recommended (org lead, 2026-09-09): that order; the app merges with **no `v*` tag**, so no APK
|
||||
> was cut; the re-verify walks the whole delivery path; and **`edge` stays standing** in every repo
|
||||
> rather than being deleted as the module-system cutover deleted its own.
|
||||
>
|
||||
> Releases cut: sidecar **v2.2.0**, overlay **v1.2.0**, bundle **2026.09.10 (protocol 7)**. `website`
|
||||
> never releases. `MODULE_API_VERSION` and `EVENTS.md` are untouched by this leg — the cutover moves
|
||||
> no contract.
|
||||
>
|
||||
> **`Integration-kit#10` had been merged early**, on 2026-09-08, though it was written to be held —
|
||||
> so the kit's `main` was red on `checkCoreApi` for two days. That is what step 5 closes, and it is
|
||||
> the reason the re-pin is a repair rather than only a date.
|
||||
>
|
||||
> #### The outage, and what it did not break
|
||||
>
|
||||
> Gitea was unreachable for about ten minutes in the middle of the window (Cloudflare 1033/530) and
|
||||
> killed **both** release runs. `link`'s built every binary and wrote `SHA256SUMS`, then died pushing
|
||||
> the tag: `fatal: unable to access … The requested URL returned error: 530`. `servuo-plugins`' died
|
||||
> inside `Set up job` after 11m52s with no step ever executing — which is why that job's log route
|
||||
> answers 500 while its predecessor's serves fine: **there is no log blob, and that absence is
|
||||
> evidence.** No tag was pushed either time, so the orphaned-tag failure mode did not occur, and
|
||||
> re-running both by `workflow_dispatch` published them. The first to land left the pair mismatched
|
||||
> and compose run 102 failed exactly as the PRs predicted; the second dispatched it again and 103
|
||||
> composed. `link`'s `rust-gates` reds on three earlier PRs were `curl: (6) Could not resolve host:
|
||||
> sh.rustup.rs` inside the runner — infrastructure, not code, on all four counts.
|
||||
>
|
||||
> **A job's log IS readable on this instance, through the web route rather than the API:**
|
||||
> `/{owner}/{repo}/actions/runs/<n>/jobs/<j>/logs` with an API token, served as `text/plain`; step
|
||||
> statuses come from the UI's own POST endpoint with a `_csrf` cookie. Every earlier phase diagnosed
|
||||
> CI by reproducing jobs locally, on the belief that logs were unreachable. They are not, and reading
|
||||
> one is what turned four red X's into four known causes in about ten minutes.
|
||||
>
|
||||
> #### A seventh defect, red on every events PR since Phase 10
|
||||
>
|
||||
> `website`'s `server-tests` job had been failing since `#192` — eight PRs, every one reporting
|
||||
> `# fail 1`, always **the same single test**, so nothing else was ever hiding behind it. The
|
||||
> workstream merged over it eight times.
|
||||
>
|
||||
> `events/announce.js` asked `Intl.DateTimeFormat('en-GB', { …, hour12: true })`, and **that is not
|
||||
> the same request as a 12-hour clock.** For a locale whose default cycle is h23 — `en-GB` is one —
|
||||
> Node 20 resolves `hour12: true` to **`h11`**, whose hours run 0–11, so midnight renders `0:00 am`;
|
||||
> Node 22 and later resolve it to `h12` and it renders `12:00 am`. **Same ICU (78.2) on both sides**,
|
||||
> so this is V8's ECMA-402 behaviour and not locale data — no amount of matching the runner's locale
|
||||
> would have found it.
|
||||
>
|
||||
> The image ships `node:20-alpine` and CI runs Node 20, while a dev machine is newer. So the mail
|
||||
> every real recipient got said **"0:00 am"** beside a schedule editor saying "12:00 AM" — one
|
||||
> instant, two spellings, the exact contradiction that option was added to prevent — and it rendered
|
||||
> correctly in front of everyone who reviewed it. Fixed to `hourCycle: 'h12'` (`website#200`), which
|
||||
> is the form `recurrence.js` had already adopted for the mirror-image case (`h23` **rather than**
|
||||
> `hour12: false`); `announce.js` was the last `hour12` in either repo.
|
||||
>
|
||||
> **The rule: `hour12` is a request about a locale's preference, `hourCycle` is a request about the
|
||||
> clock. Ask for the clock.** And the test now says so out loud, because it can only fail on Node 20:
|
||||
> a green run on a dev machine is not evidence, and CI is what holds that line.
|
||||
>
|
||||
> #### The re-verify, from artefacts an operator would download
|
||||
>
|
||||
> This is the leg 16a could not do — a locally built bundle cannot go through core's module installer,
|
||||
> which is https-only with a host allowlist.
|
||||
>
|
||||
> | | |
|
||||
> |---|---|
|
||||
> | installer | released `v0.1.1` binary, checksum matched against the release's own `SHA256SUMS` |
|
||||
> | bundle | resolved **2026.09.10, protocol 7**; both component checksums verified by the installer |
|
||||
> | overlay sync | a **first install** into a stock 57.4 tree — `add=30 change=1 unchanged=0` |
|
||||
> | script build | `0 Warning(s) 0 Error(s)` — the released overlay compiles on a stock tree, which no release had ever been asked to prove |
|
||||
> | shard boot | `[Bridge] enabled=True … adminWrite=True … events=True`, then `connected to 127.0.0.1:7788` |
|
||||
> | sidecar | `server.hello` for **208,568 items / 42,871 mobiles**; `x-uolink-version: 7` |
|
||||
> | event plane | `lease.list.ok` (config **and** targeted property leases), `item.catalog.ok` with its bounds, `GET /world/<run>` an empty list rather than a 404 |
|
||||
> | core | released `main` on a throwaway database, `capabilities: ["events"]` on `/public/version` |
|
||||
> | one event | published, run, **`completed` / `health: ok`**, results published |
|
||||
> | the page | `/site/events` reads *"Everything scheduled, live and recently finished"* and lists a run that finished two minutes earlier |
|
||||
>
|
||||
> The last row is 16a's calendar fix holding on `main`: before it, a run that had already started or
|
||||
> finished was absent and the page rendered `entries: []`.
|
||||
>
|
||||
> **A fresh `Bridge.cfg` still ships `EventsEnabled=false` and `AdminWriteEnabled=false`** — the
|
||||
> operator's real first-boot state, and the released config confirms it rather than a working tree's.
|
||||
>
|
||||
> One thing checked and deliberately **not** reported as a defect: a **cancelled** run appears on the
|
||||
> public calendar. It is meant to. The entry carries its own `status`, and the page renders a past
|
||||
> cancelled run as **"Did not happen"** — the honest label, not a silent omission.
|
||||
>
|
||||
> **The module's own install was walked too**, once step 3 cut `Module-uo` **v1.2.1**. The module
|
||||
> arrived the way an operator's would: `POST /admin/modules` naming the release's **manifest** (not
|
||||
> its tarball — core answers a tarball with *"the install manifest is larger than 262144 bytes"*,
|
||||
> which is the size guard doing its job), core fetched the artifact over https from the allowlisted
|
||||
> host, verified its `sha256`, and mounted it on the next boot with 12 event actions and 27 triggers.
|
||||
> Then `PUT /admin/uo-link/config` with the four values the installer printed answered
|
||||
> **`status: connected`, `pluginConnected: true`, `protocol: 7`** — released core, released module,
|
||||
> released sidecar, released overlay, all four talking.
|
||||
>
|
||||
> On that rig the two Phase 16a fixes were confirmed in the shipped artefacts rather than in a working
|
||||
> tree: the atlas **imported off a stock tree** (309 decor types, 6,455 points, 800 creatures, 558
|
||||
> landmarks, 387 regions, 25 champions — the import that used to die at 313), and a world verb ran and
|
||||
> **tore down for real** — three orcs spawned, ledger `reverted` ×3, `cleanup: complete`, and the shard
|
||||
> itself answering `world.owned → owned: [], pruned: 0`. That last check is the one 16a's no-op
|
||||
> teardown hid behind. The enablement gate and the cap behaved as specified on the way past: the dry
|
||||
> run refused the action before it was enabled, then priced it `uo.creatures 3 of 10`.
|
||||
>
|
||||
> **And the leg found two more defects, both in the released bundle and neither visible to any test**
|
||||
> (`Module-uo#35`).
|
||||
>
|
||||
> **The aggregator discarded the `UniqueId`, so no Phase 12b property lease was authorable at all.**
|
||||
> All 6,455 spawn points imported with `unique_id` NULL; `listSpawners` filters
|
||||
> `unique_id IS NOT NULL`, so `uo.options.spawners` — the only source those leases have — was an empty
|
||||
> dropdown with nothing to explain itself. Every part of the path was right except one line: the files
|
||||
> carry `<UniqueId>`, `parsePoints` returns it, the column exists, the insert passes it. `buildAtlas`
|
||||
> rebuilds each point from an explicit field list and the field was not on it. **`PARSER_VERSION = 4`'s
|
||||
> own note says a point keeps its `UniqueId` and names Phase 12b as the reason** — that bump exists to
|
||||
> re-read trees for this field, and the field was dropped one function later. The intent shipped as a
|
||||
> comment. Fixing it needs `PARSER_VERSION` 5 as well, because the tree's hashes have not changed —
|
||||
> only what is kept from them — so nothing would re-read an existing install.
|
||||
>
|
||||
> **A landmark option value named 23 places at once.** 558 landmarks, 320 distinct `facet/name`:
|
||||
> `Trammel/Entrance` is Blighted Grove, Covetous, Deceit, Despise, Destard and 18 more, and
|
||||
> `landmarkPoint` resolves with `.find()`. So 22 of the 23 were unreachable and an author who picked
|
||||
> "Entrance — Destard" got Blighted Grove, with a successful run and no warning. **The group was
|
||||
> already the disambiguator** — shown in the dropdown, left out of the value. Now `facet/group/name`,
|
||||
> distinct across all 558, with the two-part read kept as a fallback because a published version is
|
||||
> immutable and those stored values are the authored record. A three-part value whose group is gone
|
||||
> refuses rather than falling back: it asked for one place.
|
||||
>
|
||||
> Both are the same failure shape as 16a's blocking defect and worth naming as a class: **an option
|
||||
> source that answers empty, or answers with a value that does not identify one thing, disables a
|
||||
> feature silently.** Nothing errors, the form simply cannot express the thing — and a test that
|
||||
> checks the parser, or the query, or the column in isolation passes throughout. The atlas fixture had
|
||||
> no `<UniqueId>` in it at all until this phase, which is why a green suite said nothing for two.
|
||||
|
||||
**Two documents that are cutover-window work by construction.**
|
||||
- **`runicgateway.com`** — `checkFacts` reads `main`, so any claim about events is unverifiable until
|
||||
the cutover lands. Same 12a/12b split the engagement workstream needed. **16b landed it**, so both
|
||||
of these are now unblocked: `main` carries the engine, the module and the app, and the bundle triple
|
||||
the site quotes is sidecar **v2.2.0** / overlay **v1.2.0** / bundle **2026.09.10**.
|
||||
the cutover lands. Same 12a/12b split the engagement workstream needed.
|
||||
- **`.profile`** — the org landing page is updated when the *shape* of the project changes, which a new
|
||||
subsystem is.
|
||||
|
||||
> **16c BUILT (2026-09-09/10) — and it began by closing the cutover's missing seventh step.**
|
||||
> `runicgateway.com#30` and `.profile#6`, both onto `main`; the leg's first act was `docs#232`.
|
||||
>
|
||||
> #### The step 16b left on `edge`
|
||||
>
|
||||
> Six repositories were cut over and every one of them showed **0 commits on `edge` that are not on
|
||||
> `main`**. This one showed **46**. Step 6 (`#231`) landed the record *on* `edge` rather than cutting
|
||||
> `edge` over — an easy thing to miss, because the step's own PR merged green and closed. The
|
||||
> consequence was quiet and total: `docs` `main` opened `EVENTS.md` with *"revision 5. **No code
|
||||
> written.** Read against … `MODULE_API_VERSION` 1.9.0 · sidecar protocol 5"* while six repositories
|
||||
> shipped the engine on protocol 7, and `link/v6.md` and `v7.md` — the specs of record for two
|
||||
> protocol versions — existed on no default branch anywhere.
|
||||
>
|
||||
> **Nothing in this workstream could have caught it.** Every check that guards a contract lives in
|
||||
> the repository that *depends* on the contract, and a documentation repository has no dependants.
|
||||
> What found it was the one check that reads `docs` from outside: `runicgateway.com`'s
|
||||
> `checkReference.mjs` asserts every canonical document it names still exists on `main`, and adding
|
||||
> the current protocol spec failed with `✗ canonical doc link/v7.md`. **The site is the docs
|
||||
> repository's only dependant, and 16c is the only phase that would ever have run that check.**
|
||||
>
|
||||
> The merge was clean, and both `PROJECT_TREE.md` files stayed on `main`'s newer automated syncs —
|
||||
> `edge` never edited them, so git kept `main`'s side. `edge` stays standing, per 16b's decision,
|
||||
> now four generated commits behind.
|
||||
>
|
||||
> #### The site
|
||||
>
|
||||
> The checks were red before the phase started and named their own answers, which is the whole
|
||||
> bargain §12 of that repository's plan struck: nine `checkFacts` values (protocol 5 → **7** in all
|
||||
> three declaration sites, `moduleApi` → **1.10.0**, the bundle triple, `link` **v2.2.0**, `Module-uo`
|
||||
> **v1.2.2** — a third module release, one past the v1.2.1 the cutover cut), and **twenty-seven
|
||||
> `Bridge.cfg` keys** the site listed nowhere. Those became five groups rather than an appendix,
|
||||
> because `EventsEnabled` is a *second consent switch* and belongs beside the ceilings it governs
|
||||
> rather than filed under `AdminWriteEnabled`.
|
||||
>
|
||||
> Two pages, the treatment Teams has: **Scheduled events** under Administration and **Events
|
||||
> architecture**. Two capability entries, so the homepage, `/features/` and `/modules/` stop omitting
|
||||
> the subsystem — and the calendar one is deliberately **not** `needsModule`, because a bare core can
|
||||
> author and run an event and that marker means "present, correct and permanently empty".
|
||||
>
|
||||
> **`/privacy` owed a row and had none.** `event_run_participants` is personal data — scores and
|
||||
> ranks against a module-opaque member key, linked to an account where one is linked, feeding a
|
||||
> participant's own history. The new `deploy-events` row states the retention exactly, including the
|
||||
> asymmetry that matters: the diagnostic log is swept after 90 days **and only on terminal runs**,
|
||||
> while the run, its steps and its participants are never swept, because they are the record of what
|
||||
> was done to a shared world.
|
||||
>
|
||||
> **A naming collision worth fixing while it was cheap.** `reference/event-catalog` is about what a
|
||||
> shard *emits*; with a scheduled-event system shipped, two things in the documentation were called
|
||||
> an event catalog. Retitled **"Shard event catalog"**, with the route left alone so nothing outside
|
||||
> that repository breaks — and the page now opens by saying which of the two it is, since the kinds
|
||||
> it lists are exactly what a phase can wait for.
|
||||
>
|
||||
> **No screenshots**, and stated as a choice: capturing the events surfaces means standing the whole
|
||||
> rig back up for images no check requires, and the engagement workstream's site leg added none
|
||||
> either.
|
||||
>
|
||||
> #### `.profile`
|
||||
>
|
||||
> One bullet, and two stale numbers. The bullet says the posture rather than the feature list — off
|
||||
> by default, caps in the database, cleanup generated from a ledger, and *an event does not edit the
|
||||
> world, it holds a lease the game restores on its own deadline*. The numbers are protocol **5 → 7**
|
||||
> in the four values the installer prints (the block a reader copies into Admin → Shard, where a
|
||||
> wrong number is a pairing failure with no obvious cause) and **module-uo v1.1.0 → v1.2.2**.
|
||||
>
|
||||
> #### What 16c did not need
|
||||
>
|
||||
> No `EVENTS.md` change, no `MODULE_API_VERSION` change, no protocol change, no release. The phase
|
||||
> moves no contract — it makes the ones already moved legible from outside the organisation.
|
||||
|
||||
---
|
||||
|
||||
## What this plan does not do
|
||||
|
||||
@@ -1193,16 +1193,6 @@ rather than implementation and belong here:
|
||||
with the key and an EMPTY list, meaning *"a command went out under this key and core never learned
|
||||
what it did"*. Answering that honestly is what makes an unattended world write recoverable; a
|
||||
module that cannot answer it says so, and the row stays visible to an operator.
|
||||
- **That key IDENTIFIES a dispatch; it is not a key to send on the undo.** It names the command core
|
||||
lost the answer to, so the module can ask the game about it. Forwarding it as the outgoing key of
|
||||
the reverting command is a different thing entirely, and on a game whose at-most-once store keys on
|
||||
the key alone — as the uo-link shard's does — the undo is then recognised as a repeat of the DO and
|
||||
answered with the original reply. `module-uo` made exactly this mistake: teardown of all five world
|
||||
verbs was a no-op that reported success, because every despawn carried the key its spawn had gone
|
||||
out under. Found by the Phase 16 acceptance walk, with the ledger reading `reverted` and the shard
|
||||
still holding every object. A command that undoes needs a key of its own or none at all; a repeated
|
||||
undo is usually harmless by construction ("already gone" is a success), which is what makes *none*
|
||||
the right answer more often than not.
|
||||
- **Core owns cleanup, and it is derived rather than authored.** There is no `on_teardown` on an
|
||||
action and no cleanup phase in a spec: an operator cannot be relied on to write the undo, and an
|
||||
aborted run never reaches the phase they wrote it in. Cleanup is one sweep over the ledger and it
|
||||
|
||||
@@ -169,8 +169,6 @@ website/
|
||||
│ │ ├── lib/
|
||||
│ │ │ ├── adminNav.js
|
||||
│ │ │ ├── engagementRules.js
|
||||
│ │ │ ├── eventAuthoring.js
|
||||
│ │ │ ├── eventCalendar.js
|
||||
│ │ │ ├── format.js
|
||||
│ │ │ ├── heroLayout.js
|
||||
│ │ │ ├── moduleAdmin.js
|
||||
@@ -218,11 +216,6 @@ website/
|
||||
│ │ │ │ │ ├── EngagementSuppressions.jsx
|
||||
│ │ │ │ │ ├── EngagementTemplates.jsx
|
||||
│ │ │ │ │ ├── EngagementTriggers.jsx
|
||||
│ │ │ │ │ ├── EventActions.jsx
|
||||
│ │ │ │ │ ├── EventEditor.jsx
|
||||
│ │ │ │ │ ├── EventRun.jsx
|
||||
│ │ │ │ │ ├── EventsAdmin.jsx
|
||||
│ │ │ │ │ ├── EventsCalendar.jsx
|
||||
│ │ │ │ │ ├── HeroEditor.jsx
|
||||
│ │ │ │ │ ├── InvitesAdmin.jsx
|
||||
│ │ │ │ │ ├── Moderation.jsx
|
||||
@@ -253,7 +246,6 @@ website/
|
||||
│ │ │ │ ├── ForgotPassword.jsx
|
||||
│ │ │ │ ├── PlayerAccount.jsx
|
||||
│ │ │ │ ├── PlayerAppeals.jsx
|
||||
│ │ │ │ ├── PlayerEvents.jsx
|
||||
│ │ │ │ ├── PlayerInbox.jsx
|
||||
│ │ │ │ ├── PlayerLogin.jsx
|
||||
│ │ │ │ ├── PlayerNotifications.jsx
|
||||
@@ -266,9 +258,6 @@ website/
|
||||
│ │ │ ├── public/
|
||||
│ │ │ │ ├── About.jsx
|
||||
│ │ │ │ ├── CmsPage.jsx
|
||||
│ │ │ │ ├── EventPage.jsx
|
||||
│ │ │ │ ├── Events.jsx
|
||||
│ │ │ │ ├── EventSeries.jsx
|
||||
│ │ │ │ ├── FiveOnFriday.jsx
|
||||
│ │ │ │ ├── Maintenance.jsx
|
||||
│ │ │ │ ├── News.jsx
|
||||
@@ -290,8 +279,6 @@ website/
|
||||
│ │ ├── apiClient.test.js
|
||||
│ │ ├── emailTemplates.test.js
|
||||
│ │ ├── engagementRules.test.js
|
||||
│ │ ├── eventAuthoring.test.js
|
||||
│ │ ├── eventCalendar.test.js
|
||||
│ │ ├── featureGate.test.js
|
||||
│ │ ├── format.test.js
|
||||
│ │ ├── heroLayout.test.js
|
||||
@@ -365,7 +352,6 @@ website/
|
||||
│ │ │ └── validateBlocks.js
|
||||
│ │ ├── config/
|
||||
│ │ │ ├── brand.js
|
||||
│ │ │ ├── coreEventActions.js
|
||||
│ │ │ ├── coreStreams.js
|
||||
│ │ │ ├── coreTriggers.js
|
||||
│ │ │ ├── csp.js
|
||||
@@ -407,18 +393,6 @@ website/
|
||||
│ │ │ ├── suppressions.js
|
||||
│ │ │ ├── templates.js
|
||||
│ │ │ └── templateSeeds.js
|
||||
│ │ ├── events/
|
||||
│ │ │ ├── announce.js
|
||||
│ │ │ ├── authorize.js
|
||||
│ │ │ ├── cleanup.js
|
||||
│ │ │ ├── dispatch.js
|
||||
│ │ │ ├── gates.js
|
||||
│ │ │ ├── ledger.js
|
||||
│ │ │ ├── participants.js
|
||||
│ │ │ ├── price.js
|
||||
│ │ │ ├── recurrence.js
|
||||
│ │ │ ├── spec.js
|
||||
│ │ │ └── verify.js
|
||||
│ │ ├── middleware/
|
||||
│ │ │ ├── botScore.js
|
||||
│ │ │ ├── loginProtection.js
|
||||
@@ -468,25 +442,6 @@ website/
|
||||
│ │ │ │ ├── engagementSuppressions.db.js
|
||||
│ │ │ │ ├── engagementTemplates.db.js
|
||||
│ │ │ │ └── engagementTemplates.model.js
|
||||
│ │ │ ├── events/
|
||||
│ │ │ │ ├── eventActionSettings.db.js
|
||||
│ │ │ │ ├── eventCalendar.model.js
|
||||
│ │ │ │ ├── eventDefinitions.db.js
|
||||
│ │ │ │ ├── eventDefinitions.model.js
|
||||
│ │ │ │ ├── eventJson.js
|
||||
│ │ │ │ ├── eventPhaseGates.db.js
|
||||
│ │ │ │ ├── eventPublic.model.js
|
||||
│ │ │ │ ├── eventRunBudget.db.js
|
||||
│ │ │ │ ├── eventRunControls.model.js
|
||||
│ │ │ │ ├── eventRunLog.db.js
|
||||
│ │ │ │ ├── eventRunParticipants.db.js
|
||||
│ │ │ │ ├── eventRunResources.db.js
|
||||
│ │ │ │ ├── eventRuns.db.js
|
||||
│ │ │ │ ├── eventRuns.model.js
|
||||
│ │ │ │ ├── eventRunSteps.db.js
|
||||
│ │ │ │ ├── eventSeries.db.js
|
||||
│ │ │ │ ├── eventSeries.model.js
|
||||
│ │ │ │ └── eventVersions.db.js
|
||||
│ │ │ ├── invites/
|
||||
│ │ │ │ ├── invites.db.js
|
||||
│ │ │ │ └── invites.model.js
|
||||
@@ -604,8 +559,6 @@ website/
|
||||
│ │ │ │ │ ├── emailConfig.controller.js
|
||||
│ │ │ │ │ ├── engagement.controller.js
|
||||
│ │ │ │ │ ├── engagement.router.js
|
||||
│ │ │ │ │ ├── events.controller.js
|
||||
│ │ │ │ │ ├── events.router.js
|
||||
│ │ │ │ │ ├── imageUpload.js
|
||||
│ │ │ │ │ ├── index.js
|
||||
│ │ │ │ │ ├── invites.controller.js
|
||||
@@ -654,8 +607,6 @@ website/
|
||||
│ │ │ │ ├── player/
|
||||
│ │ │ │ │ ├── appeals.controller.js
|
||||
│ │ │ │ │ ├── appeals.router.js
|
||||
│ │ │ │ │ ├── events.controller.js
|
||||
│ │ │ │ │ ├── events.router.js
|
||||
│ │ │ │ │ ├── index.js
|
||||
│ │ │ │ │ ├── teamForum.controller.js
|
||||
│ │ │ │ │ ├── teamForum.router.js
|
||||
@@ -664,8 +615,6 @@ website/
|
||||
│ │ │ │ ├── public/
|
||||
│ │ │ │ │ ├── engagement.controller.js
|
||||
│ │ │ │ │ ├── engagement.router.js
|
||||
│ │ │ │ │ ├── events.controller.js
|
||||
│ │ │ │ │ ├── events.router.js
|
||||
│ │ │ │ │ ├── index.js
|
||||
│ │ │ │ │ ├── modules.controller.js
|
||||
│ │ │ │ │ ├── modules.router.js
|
||||
@@ -697,7 +646,6 @@ website/
|
||||
│ │ │ ├── engagementEmit.js
|
||||
│ │ │ ├── engagementRetentionPrune.js
|
||||
│ │ │ ├── engagementWorker.js
|
||||
│ │ │ ├── eventRunner.js
|
||||
│ │ │ ├── forumHtml.js
|
||||
│ │ │ ├── htmlShell.js
|
||||
│ │ │ ├── logger.js
|
||||
@@ -770,27 +718,6 @@ website/
|
||||
│ │ ├── engagementRetentionSql.test.js
|
||||
│ │ ├── engagementTemplatesAdmin.test.js
|
||||
│ │ ├── engagementTriggers.test.js
|
||||
│ │ ├── eventActionRegistry.test.js
|
||||
│ │ ├── eventAnnounce.test.js
|
||||
│ │ ├── eventAuthorize.test.js
|
||||
│ │ ├── eventCleanup.test.js
|
||||
│ │ ├── eventGates.test.js
|
||||
│ │ ├── eventIntegrations.test.js
|
||||
│ │ ├── eventLedger.test.js
|
||||
│ │ ├── eventModuleContract.test.js
|
||||
│ │ ├── eventParticipants.test.js
|
||||
│ │ ├── eventPrice.test.js
|
||||
│ │ ├── eventPublic.test.js
|
||||
│ │ ├── eventRecurrence.test.js
|
||||
│ │ ├── eventRunControls.test.js
|
||||
│ │ ├── eventRunner.test.js
|
||||
│ │ ├── eventRunnerSql.test.js
|
||||
│ │ ├── eventsAdmin.test.js
|
||||
│ │ ├── eventSchedule.test.js
|
||||
│ │ ├── eventSeries.test.js
|
||||
│ │ ├── eventSpec.test.js
|
||||
│ │ ├── eventsRoles.test.js
|
||||
│ │ ├── eventVerify.test.js
|
||||
│ │ ├── honeypot.test.js
|
||||
│ │ ├── htmlShell.test.js
|
||||
│ │ ├── inviteController.test.js
|
||||
|
||||
Reference in New Issue
Block a user