Compare commits
27 Commits
edge
...
e1cefa5be2
| Author | SHA1 | Date | |
|---|---|---|---|
| e1cefa5be2 | |||
| a4b63d87d2 | |||
| 2590d4cc58 | |||
| 6ef4b06c76 | |||
| afd0772289 | |||
| d07772a3d8 | |||
| 1a7481e9f4 | |||
| 4c6b0c566a | |||
| bbd69a8e2e | |||
| df9b3fd990 | |||
| 1a048ae1be | |||
| f6ea5c0484 | |||
| 018f1af5ff | |||
| f2e074c3f7 | |||
| 041e1f4069 | |||
| d39e4163d1 | |||
| 44039e83d5 | |||
| b62d0b6307 | |||
| 0c3f0d65cc | |||
| 556141341f | |||
| 132620e8b3 | |||
| 81b55ac29b | |||
| 2db5737f22 | |||
| eae8a6f93d | |||
|
|
537c79e5de | ||
| abfdcd4658 | |||
|
|
90f95520a3 |
@@ -37,7 +37,6 @@ sidecar as a service, and hands you the values the website needs.
|
|||||||
| [MODERATION_APPEALS.md](website/MODERATION_APPEALS.md) | Moderation actions, content reports and the appeals flow |
|
| [MODERATION_APPEALS.md](website/MODERATION_APPEALS.md) | Moderation actions, content reports and the appeals flow |
|
||||||
| [SPAWN_ATLAS.md](website/SPAWN_ATLAS.md) | The bestiary / spawn atlas: what the shard contains, parsed from its own ServUO tree |
|
| [SPAWN_ATLAS.md](website/SPAWN_ATLAS.md) | The bestiary / spawn atlas: what the shard contains, parsed from its own ServUO tree |
|
||||||
| [CLILOCS.md](website/CLILOCS.md) | UO's id → name table: converting one from your client so items have names |
|
| [CLILOCS.md](website/CLILOCS.md) | UO's id → name table: converting one from your client so items have names |
|
||||||
| [UOFIDDLER.md](website/UOFIDDLER.md) | **Operator runbook** — step-by-step extraction from your own UO client (cliloc table, creature art) |
|
|
||||||
| [MARKETPLACE.md](website/MARKETPLACE.md) | The player-vendor index: how it is gathered, what it costs, how to tune it |
|
| [MARKETPLACE.md](website/MARKETPLACE.md) | The player-vendor index: how it is gathered, what it costs, how to tune it |
|
||||||
| [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) |
|
| [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) |
|
||||||
| [test-plan.md](website/test-plan.md) | The website's test strategy and harness |
|
| [test-plan.md](website/test-plan.md) | The website's test strategy and harness |
|
||||||
|
|||||||
@@ -79,6 +79,8 @@ android-app/
|
|||||||
│ │ │ │ │ │ └── PushTickle.kt
|
│ │ │ │ │ │ └── PushTickle.kt
|
||||||
│ │ │ │ │ ├── result/
|
│ │ │ │ │ ├── result/
|
||||||
│ │ │ │ │ │ └── ApiResult.kt
|
│ │ │ │ │ │ └── ApiResult.kt
|
||||||
|
│ │ │ │ │ ├── time/
|
||||||
|
│ │ │ │ │ │ └── Instants.kt
|
||||||
│ │ │ │ │ ├── web/
|
│ │ │ │ │ ├── web/
|
||||||
│ │ │ │ │ │ ├── WebHandoff.kt
|
│ │ │ │ │ │ ├── WebHandoff.kt
|
||||||
│ │ │ │ │ │ └── WebsiteUrls.kt
|
│ │ │ │ │ │ └── WebsiteUrls.kt
|
||||||
@@ -90,6 +92,7 @@ android-app/
|
|||||||
│ │ │ │ │ │ │ ├── AdminDto.kt
|
│ │ │ │ │ │ │ ├── AdminDto.kt
|
||||||
│ │ │ │ │ │ │ ├── AuthDto.kt
|
│ │ │ │ │ │ │ ├── AuthDto.kt
|
||||||
│ │ │ │ │ │ │ ├── ContactDto.kt
|
│ │ │ │ │ │ │ ├── ContactDto.kt
|
||||||
|
│ │ │ │ │ │ │ ├── EventsDto.kt
|
||||||
│ │ │ │ │ │ │ ├── NotificationsDto.kt
|
│ │ │ │ │ │ │ ├── NotificationsDto.kt
|
||||||
│ │ │ │ │ │ │ ├── PageDto.kt
|
│ │ │ │ │ │ │ ├── PageDto.kt
|
||||||
│ │ │ │ │ │ │ ├── PlayerShardDto.kt
|
│ │ │ │ │ │ │ ├── PlayerShardDto.kt
|
||||||
@@ -102,6 +105,7 @@ android-app/
|
|||||||
│ │ │ │ │ │ ├── AdminApi.kt
|
│ │ │ │ │ │ ├── AdminApi.kt
|
||||||
│ │ │ │ │ │ ├── AuthApi.kt
|
│ │ │ │ │ │ ├── AuthApi.kt
|
||||||
│ │ │ │ │ │ ├── AuthRefreshApi.kt
|
│ │ │ │ │ │ ├── AuthRefreshApi.kt
|
||||||
|
│ │ │ │ │ │ ├── EventsApi.kt
|
||||||
│ │ │ │ │ │ ├── MeApi.kt
|
│ │ │ │ │ │ ├── MeApi.kt
|
||||||
│ │ │ │ │ │ ├── NotificationsApi.kt
|
│ │ │ │ │ │ ├── NotificationsApi.kt
|
||||||
│ │ │ │ │ │ ├── PlayerShardApi.kt
|
│ │ │ │ │ │ ├── PlayerShardApi.kt
|
||||||
@@ -117,11 +121,13 @@ android-app/
|
|||||||
│ │ │ │ │ ├── ConnectionRepository.kt
|
│ │ │ │ │ ├── ConnectionRepository.kt
|
||||||
│ │ │ │ │ ├── ContactRepository.kt
|
│ │ │ │ │ ├── ContactRepository.kt
|
||||||
│ │ │ │ │ ├── ContentRepository.kt
|
│ │ │ │ │ ├── ContentRepository.kt
|
||||||
|
│ │ │ │ │ ├── EventsRepository.kt
|
||||||
│ │ │ │ │ ├── NotificationsRepository.kt
|
│ │ │ │ │ ├── NotificationsRepository.kt
|
||||||
│ │ │ │ │ ├── PlayerShardRepository.kt
|
│ │ │ │ │ ├── PlayerShardRepository.kt
|
||||||
│ │ │ │ │ ├── SettingsRepository.kt
|
│ │ │ │ │ ├── SettingsRepository.kt
|
||||||
│ │ │ │ │ ├── ShardFeaturesRepository.kt
|
│ │ │ │ │ ├── ShardFeaturesRepository.kt
|
||||||
│ │ │ │ │ ├── ShardRepository.kt
|
│ │ │ │ │ ├── ShardRepository.kt
|
||||||
|
│ │ │ │ │ ├── SiteCapabilitiesRepository.kt
|
||||||
│ │ │ │ │ └── WikiRepository.kt
|
│ │ │ │ │ └── WikiRepository.kt
|
||||||
│ │ │ │ ├── di/
|
│ │ │ │ ├── di/
|
||||||
│ │ │ │ │ ├── AppModule.kt
|
│ │ │ │ │ ├── AppModule.kt
|
||||||
@@ -157,6 +163,16 @@ android-app/
|
|||||||
│ │ │ │ │ ├── contact/
|
│ │ │ │ │ ├── contact/
|
||||||
│ │ │ │ │ │ ├── ContactScreen.kt
|
│ │ │ │ │ │ ├── ContactScreen.kt
|
||||||
│ │ │ │ │ │ └── ContactViewModel.kt
|
│ │ │ │ │ │ └── ContactViewModel.kt
|
||||||
|
│ │ │ │ │ ├── events/
|
||||||
|
│ │ │ │ │ │ ├── EventScreen.kt
|
||||||
|
│ │ │ │ │ │ ├── EventSeriesScreen.kt
|
||||||
|
│ │ │ │ │ │ ├── EventSeriesViewModel.kt
|
||||||
|
│ │ │ │ │ │ ├── EventsScreen.kt
|
||||||
|
│ │ │ │ │ │ ├── EventsViewModel.kt
|
||||||
|
│ │ │ │ │ │ ├── EventTimes.kt
|
||||||
|
│ │ │ │ │ │ ├── EventViewModel.kt
|
||||||
|
│ │ │ │ │ │ ├── MyEventsScreen.kt
|
||||||
|
│ │ │ │ │ │ └── MyEventsViewModel.kt
|
||||||
│ │ │ │ │ ├── home/
|
│ │ │ │ │ ├── home/
|
||||||
│ │ │ │ │ │ ├── HomeScreen.kt
|
│ │ │ │ │ │ ├── HomeScreen.kt
|
||||||
│ │ │ │ │ │ └── HomeViewModel.kt
|
│ │ │ │ │ │ └── HomeViewModel.kt
|
||||||
@@ -335,6 +351,7 @@ android-app/
|
|||||||
│ │ │ │ │ └── WikiDtoTest.kt
|
│ │ │ │ │ └── WikiDtoTest.kt
|
||||||
│ │ │ │ └── fake/
|
│ │ │ │ └── fake/
|
||||||
│ │ │ │ ├── FakeAdminApi.kt
|
│ │ │ │ ├── FakeAdminApi.kt
|
||||||
|
│ │ │ │ ├── FakeEventsApi.kt
|
||||||
│ │ │ │ ├── FakeNotificationsApi.kt
|
│ │ │ │ ├── FakeNotificationsApi.kt
|
||||||
│ │ │ │ ├── FakePlayerShardApi.kt
|
│ │ │ │ ├── FakePlayerShardApi.kt
|
||||||
│ │ │ │ ├── FakePublicApi.kt
|
│ │ │ │ ├── FakePublicApi.kt
|
||||||
@@ -345,7 +362,8 @@ android-app/
|
|||||||
│ │ │ └── repository/
|
│ │ │ └── repository/
|
||||||
│ │ │ ├── AccountTrustedDevicesTest.kt
|
│ │ │ ├── AccountTrustedDevicesTest.kt
|
||||||
│ │ │ ├── ConnectionVersionGuardTest.kt
|
│ │ │ ├── ConnectionVersionGuardTest.kt
|
||||||
│ │ │ └── ShardFeaturesRepositoryTest.kt
|
│ │ │ ├── ShardFeaturesRepositoryTest.kt
|
||||||
|
│ │ │ └── SiteCapabilitiesRepositoryTest.kt
|
||||||
│ │ ├── ui/
|
│ │ ├── ui/
|
||||||
│ │ │ ├── admin/
|
│ │ │ ├── admin/
|
||||||
│ │ │ │ ├── AdminContentViewModelTest.kt
|
│ │ │ │ ├── AdminContentViewModelTest.kt
|
||||||
@@ -356,8 +374,12 @@ android-app/
|
|||||||
│ │ │ │ └── BrandAssetsTest.kt
|
│ │ │ │ └── BrandAssetsTest.kt
|
||||||
│ │ │ ├── contact/
|
│ │ │ ├── contact/
|
||||||
│ │ │ │ └── ContactViewModelTest.kt
|
│ │ │ │ └── ContactViewModelTest.kt
|
||||||
|
│ │ │ ├── events/
|
||||||
|
│ │ │ │ ├── EventsViewModelsTest.kt
|
||||||
|
│ │ │ │ └── EventTimesTest.kt
|
||||||
│ │ │ ├── navigation/
|
│ │ │ ├── navigation/
|
||||||
│ │ │ │ ├── MenuAccessTest.kt
|
│ │ │ │ ├── MenuAccessTest.kt
|
||||||
|
│ │ │ │ ├── MenuCapabilityGatingTest.kt
|
||||||
│ │ │ │ ├── MenuFeatureGatingTest.kt
|
│ │ │ │ ├── MenuFeatureGatingTest.kt
|
||||||
│ │ │ │ ├── NavOverridesTest.kt
|
│ │ │ │ ├── NavOverridesTest.kt
|
||||||
│ │ │ │ ├── NavPathsTest.kt
|
│ │ │ │ ├── NavPathsTest.kt
|
||||||
|
|||||||
@@ -57,9 +57,9 @@ The wire protocol is versioned so a mismatch is caught immediately instead of fa
|
|||||||
|
|
||||||
The current version is **6**. It is not released yet — it lives on `edge` and ships with the event system's cutover; the last released pairing is protocol **5**, sidecar **v2.1.0** + overlay **v1.1.0**, resolved as bundle **2026.09.01**, never as "latest of each".
|
The current version is **6**. It is not released yet — it lives on `edge` and ships with the event system's cutover; the last released pairing is protocol **5**, sidecar **v2.1.0** + overlay **v1.1.0**, resolved as bundle **2026.09.01**, never as "latest of each".
|
||||||
|
|
||||||
- Every response carries an **`X-UOLink-Version: 6`** header.
|
- Every response carries an **`X-UOLink-Version: 8`** header.
|
||||||
- `GET /health` and the WebSocket `ws.hello` frame include `"protocol": 6`.
|
- `GET /health` and the WebSocket `ws.hello` frame include `"protocol": 6`.
|
||||||
- **Optionally**, send `X-UOLink-Version: 6` on your requests. If it disagrees with the sidecar, the request is rejected **409 Conflict**:
|
- **Optionally**, send `X-UOLink-Version: 8` on your requests. If it disagrees with the sidecar, the request is rejected **409 Conflict**:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{ "error": "protocol version mismatch", "sidecar_protocol": 6, "client_protocol": "5" }
|
{ "error": "protocol version mismatch", "sidecar_protocol": 6, "client_protocol": "5" }
|
||||||
@@ -1277,7 +1277,7 @@ so a retry loop cannot quietly swallow a mismatched deployment.
|
|||||||
A typical character page:
|
A typical character page:
|
||||||
|
|
||||||
```js
|
```js
|
||||||
const H = { "Authorization": `Bearer ${TOKEN}`, "X-UOLink-Version": "6" };
|
const H = { "Authorization": `Bearer ${TOKEN}`, "X-UOLink-Version": "8" };
|
||||||
|
|
||||||
// 1. render the roster
|
// 1. render the roster
|
||||||
const roster = await fetch(`${BASE}/roster/${account}`, { headers: H }).then(r => r.json());
|
const roster = await fetch(`${BASE}/roster/${account}`, { headers: H }).then(r => r.json());
|
||||||
|
|||||||
@@ -69,3 +69,44 @@ World: Loading...
|
|||||||
## Unrelated, still open
|
## Unrelated, still open
|
||||||
|
|
||||||
`DllNotFoundException: zlibwapi64` crashed this shard once (`Crash 6-5-2026-22-38-3.log`) while sending a packed gump. `zlibwapi64.dll` is present in the repo root, so this is a working-directory / native-load-path problem. It will bite the bridge if the bridge ever triggers a gump send. Resolve before load testing.
|
`DllNotFoundException: zlibwapi64` crashed this shard once (`Crash 6-5-2026-22-38-3.log`) while sending a packed gump. `zlibwapi64.dll` is present in the repo root, so this is a working-directory / native-load-path problem. It will bite the bridge if the bridge ever triggers a gump send. Resolve before load testing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Host prerequisite: `libgdiplus` on Linux (Protocol 8)
|
||||||
|
|
||||||
|
Everything above is a repair to one shard's *scripts*. This one is different in kind: it is a
|
||||||
|
requirement on the **host**, it applies to every shard, and only to Linux ones.
|
||||||
|
|
||||||
|
The Asset Bridge ([`v8.md`](v8.md) §4.4) has the shard read art out of the operator's own UO client
|
||||||
|
files. ServUO targets `net48`, so on Linux it runs under Mono, and Mono's `System.Drawing` is a thin
|
||||||
|
layer over **libgdiplus** — which sits in the **decode** path and not merely the encode:
|
||||||
|
`Ultima.Frame` writes ARGB1555 through a `LockBits` pointer. Without that library a Linux shard
|
||||||
|
cannot read a sprite at all.
|
||||||
|
|
||||||
|
**Windows shard hosts need nothing.** `System.Drawing` ships with .NET Framework.
|
||||||
|
|
||||||
|
| Host | Get it with |
|
||||||
|
|---|---|
|
||||||
|
| Debian / Ubuntu | `sudo apt-get install libgdiplus` — in Debian since bullseye (6.0.4) and bookworm/trixie (6.1), and in Ubuntu universe |
|
||||||
|
| Fedora / RHEL | `sudo dnf install libgdiplus` (EPEL or the Mono repository) |
|
||||||
|
| Docker | `RUN apt-get update && apt-get install -y libgdiplus` in the shard image |
|
||||||
|
| Alpine, or a distro with no package | Build from source. This is the awkward case, and it is worth avoiding by choosing a Debian-based image |
|
||||||
|
|
||||||
|
Upstream is <https://github.com/mono/libgdiplus>. **That repository was archived in March 2025** and
|
||||||
|
is read-only; distributions still package and patch it, so installing it is a normal supported thing
|
||||||
|
to do today, but nobody is maintaining it upstream. It is the strongest long-term argument for
|
||||||
|
eventually moving extraction off `System.Drawing`.
|
||||||
|
|
||||||
|
**Its absence is not an error and never a crash.** The shard reports a named status on the source
|
||||||
|
gate — the first call any import makes — so an operator meets this while setting the shard up rather
|
||||||
|
than as an empty bestiary weeks later:
|
||||||
|
|
||||||
|
```
|
||||||
|
imaging: { ok: false, code: "NO_IMAGING",
|
||||||
|
reason: "This shard host cannot render images — Mono's System.Drawing needs
|
||||||
|
libgdiplus. Install it (apt-get install libgdiplus) and re-run the
|
||||||
|
import. Cliloc and atlas import are unaffected." }
|
||||||
|
```
|
||||||
|
|
||||||
|
Clilocs and the ServUO tree files are genuinely unaffected: neither touches a pixel. The installer's
|
||||||
|
`doctor` checks for this alongside its other host checks.
|
||||||
|
|||||||
@@ -823,6 +823,11 @@ shapes are the plain binary layout and a `number<TAB|,|;>text` export; the site
|
|||||||
and writes the plain form. A shard that never converts is fully supported — names render as ids,
|
and writes the plain form. A shard that never converts is fully supported — names render as ids,
|
||||||
exactly as before.
|
exactly as before.
|
||||||
|
|
||||||
|
> **Superseded by [`v8.md`](v8.md) §9 (protocol 8, phase 2).** The decompressor is in the overlay
|
||||||
|
> now, the shard reads its own client, and `server/tools/cliloc-export/` has been deleted. The
|
||||||
|
> paragraph above is kept as the record of why the manual step existed; everything else in this
|
||||||
|
> section — the overlay set, the hash gate, the refusals — still describes what runs.
|
||||||
|
|
||||||
**Shards edit items and add new ones**, and those carry ids no stock client table has — so this reads
|
**Shards edit items and add new ones**, and those carry ids no stock client table has — so this reads
|
||||||
a **set** of sources, not one file, hash-gated together and re-read on every boot exactly as §6 reads
|
a **set** of sources, not one file, hash-gated together and re-read on every boot exactly as §6 reads
|
||||||
the ServUO tree: a base (the converted client table) plus every overlay under `custom/`, later
|
the ServUO tree: a base (the converted client table) plus every overlay under `custom/`, later
|
||||||
|
|||||||
1455
link/v8.md
Normal file
1455
link/v8.md
Normal file
File diff suppressed because it is too large
Load Diff
@@ -23,7 +23,7 @@ knowing a module answers now.
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `/api/v1/public/shard` | 19 | Anonymous. **Never site-mode gated** — the shard surface stays readable during maintenance, per feature audience. |
|
| `/api/v1/public/shard` | 19 | Anonymous. **Never site-mode gated** — the shard surface stays readable during maintenance, per feature audience. |
|
||||||
| `/api/v1/public/atlas` | 6 | Anonymous, and unlike `/shard` it **is** site-mode gated: nothing here touches the sidecar, it is parsed shard content. |
|
| `/api/v1/public/atlas` | 6 | Anonymous, and unlike `/shard` it **is** site-mode gated: nothing here touches the sidecar, it is parsed shard content. |
|
||||||
| `/api/v1/admin/shard` | 26 | Behind core's `isLoggedIn + noindex + staffOnly` group gate, then **mixed per route** — see below. |
|
| `/api/v1/admin/shard` | 28 | Behind core's `isLoggedIn + noindex + staffOnly` group gate, then **mixed per route** — see below. |
|
||||||
| `/api/v1/admin/uo-link` | 7 | `adminOnly`. The sidecar connection config, its live status, the admin SSE stream and the town crier. |
|
| `/api/v1/admin/uo-link` | 7 | `adminOnly`. The sidecar connection config, its live status, the admin SSE stream and the town crier. |
|
||||||
| `/api/v1/player/shard` | 8 | `requireAuth`, **any role** — staff are a superset of players — and every handler is self-scoped to `req.user.id`. |
|
| `/api/v1/player/shard` | 8 | `requireAuth`, **any role** — staff are a superset of players — and every handler is self-scoped to `req.user.id`. |
|
||||||
| `/api/v1/admin/users/:id/shard/*` | 6 | `adminOnly`. The module's routes hanging off a **core** resource, through core's `admin.users.detail` extension slot: core owns the user, the module owns what it knows about their game accounts. |
|
| `/api/v1/admin/users/:id/shard/*` | 6 | `adminOnly`. The module's routes hanging off a **core** resource, through core's `admin.users.detail` extension slot: core owns the user, the module owns what it knows about their game accounts. |
|
||||||
@@ -56,7 +56,7 @@ feed) are described where their wire frames are, in
|
|||||||
| GET | `/shard/ruleset` | the shard's own published ruleset (Protocol 3.0 `world.ruleset`): expansion, which optional systems are on, skill/stat caps, account and house limits, champion scroll rules, the save/restart schedule. Served from `shard_ruleset`, so it renders while the shard is down; live via `world.ruleset` on `/shard/stream`. Behind `requireFeature('ruleset')`. **`null`** means the shard has never published one — a real answer, distinct from a published ruleset. `caps.skill` / `caps.totalSkill` are in **tenths** (1000 = 100.0). |
|
| GET | `/shard/ruleset` | the shard's own published ruleset (Protocol 3.0 `world.ruleset`): expansion, which optional systems are on, skill/stat caps, account and house limits, champion scroll rules, the save/restart schedule. Served from `shard_ruleset`, so it renders while the shard is down; live via `world.ruleset` on `/shard/stream`. Behind `requireFeature('ruleset')`. **`null`** means the shard has never published one — a real answer, distinct from a published ruleset. `caps.skill` / `caps.totalSkill` are in **tenths** (1000 = 100.0). |
|
||||||
| GET | `/shard/points` | every points/loyalty leaderboard the shard publishes (Protocol 3.0 `points.board`) — Queen's Loyalty, Void Pool, the nine city loyalties, Clean Up Britannia, … Served from `shard_points_boards`, so it renders while the shard is down; live via `points.board` on `/shard/stream`. Behind `requireFeature('leaderboards')`, ordered by display name. **`maxPoints: 0` means uncapped** (the common case), and `nameString` is usually `null` with `nameNumber` holding a cliloc — resolve client-side or humanise the `system` key. |
|
| GET | `/shard/points` | every points/loyalty leaderboard the shard publishes (Protocol 3.0 `points.board`) — Queen's Loyalty, Void Pool, the nine city loyalties, Clean Up Britannia, … Served from `shard_points_boards`, so it renders while the shard is down; live via `points.board` on `/shard/stream`. Behind `requireFeature('leaderboards')`, ordered by display name. **`maxPoints: 0` means uncapped** (the common case), and `nameString` is usually `null` with `nameNumber` holding a cliloc — resolve client-side or humanise the `system` key. |
|
||||||
| GET | `/shard/points/:system` | one board by the shard's `PointsType` name (e.g. `QueensLoyalty`); `:system` must match `/^[A-Za-z][A-Za-z0-9_]{0,47}$/` or **400** before any query runs. **404** = the shard has never published that system, which is distinct from a published board nobody has scored in yet (**200** with an empty `top`). |
|
| GET | `/shard/points/:system` | one board by the shard's `PointsType` name (e.g. `QueensLoyalty`); `:system` must match `/^[A-Za-z][A-Za-z0-9_]{0,47}$/` or **400** before any query runs. **404** = the shard has never published that system, which is distinct from a published board nobody has scored in yet (**200** with an empty `top`). |
|
||||||
| GET | `/shard/market?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=` | search the player-vendor marketplace (Protocol 3.0 `vendor.listing`). Returns **listings**, not vendors — "who sells X and for how much" is the question, and a vendor-shaped result would make every caller flatten the shops back out. Served from `shard_vendors` + `shard_vendor_items`, so it renders while the shard is down. Behind `requireFeature('market')` **and rate-limited** — the first genuinely expensive public read on the site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*` table, reachable with no session). `sort ∈ {price_asc, price_desc, recent}`. `q` matches the resolved display name **or** the item's literal name, with `%`/`_` escaped: they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let `?q=%` match every listing on the shard. Every response repeats `staleAt` (the oldest vendor row) because the shard sweeps round-robin — a banner that ages with the results it labels, not one fetched once. |
|
| GET | `/shard/market?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=` | search the player-vendor marketplace (Protocol 3.0 `vendor.listing`). Returns **listings**, not vendors — "who sells X and for how much" is the question, and a vendor-shaped result would make every caller flatten the shops back out. Served from `shard_vendors` + `shard_vendor_items`, so it renders while the shard is down. Behind `requireFeature('market')` **and rate-limited** — the first genuinely expensive public read on the site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*` table, reachable with no session). `sort ∈ {price_asc, price_desc, recent}`. `q` matches the resolved display name **or** the item's literal name, with `%`/`_` escaped: they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let `?q=%` match every listing on the shard. Every response repeats `staleAt` (the oldest vendor row) because the shard sweeps round-robin — a banner that ages with the results it labels, not one fetched once. As of Protocol 8 phase 5 each listing also carries **`art`**: the FILENAME of the item's picture under `uploads/items/`, already hued, or `null` where this site does not hold one. `null` is ordinary — the picture is fetched behind the page and never by it, so a newly listed item shows text first and gains its icon a pass later, and some item ids have no art in any client. The same field appears on a character sheet's equipment entries. |
|
||||||
| GET | `/shard/market/meta` | index size, staleness (`staleAt`/`freshAt`) and which facets and regions actually hold vendors, so a client builds its filters without running a search it will discard. |
|
| GET | `/shard/market/meta` | index size, staleness (`staleAt`/`freshAt`) and which facets and regions actually hold vendors, so a client builds its filters without running a search it will discard. |
|
||||||
| GET | `/shard/market/vendors/:serial` | one shop and its listings; `:serial` must match `/^0x[0-9A-Fa-f]{1,16}$/` or **400** before any query runs. **404** = a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer, and distinguishing them would leak that a hidden vendor exists. `truncated` (with `total` exceeding `count`) means the shop holds more than the shard publishes per frame. |
|
| GET | `/shard/market/vendors/:serial` | one shop and its listings; `:serial` must match `/^0x[0-9A-Fa-f]{1,16}$/` or **400** before any query runs. **404** = a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer, and distinguishing them would leak that a hidden vendor exists. `truncated` (with `total` exceeding `count`) means the shop holds more than the shard publishes per frame. |
|
||||||
| GET | `/shard/features` | the shard features **this caller** may reach plus the audience rung they resolved to (§4 below), so a client hides nav it can't follow. Reports only what the caller can see — the list itself never discloses a gated feature. Consumed by the SPA header and (pending) the Android nav. |
|
| GET | `/shard/features` | the shard features **this caller** may reach plus the audience rung they resolved to (§4 below), so a client hides nav it can't follow. Reports only what the caller can see — the list itself never discloses a gated feature. Consumed by the SPA header and (pending) the Android nav. |
|
||||||
@@ -81,6 +81,9 @@ account-linking routes and the sidecar config under `/admin/uo-link` are in the
|
|||||||
| PUT | `/shard/atlas/path` | point the atlas at a different tree (persisted as `spawn_atlas_servuo_path`, which wins over `SERVUO_PATH`). Blank clears it. Deliberately **does not import** — moving the mount and reloading the world are separate decisions — and returns fresh status so the panel can offer the import next. |
|
| PUT | `/shard/atlas/path` | point the atlas at a different tree (persisted as `spawn_atlas_servuo_path`, which wins over `SERVUO_PATH`). Blank clears it. Deliberately **does not import** — moving the mount and reloading the world are separate decisions — and returns fresh status so the panel can offer the import next. |
|
||||||
| GET | `/shard/clilocs` | cliloc-table status (`adminOnly`): every source found now (base first, then `custom/` overlays in merge order), what each contributed at the last import, readability, drift across the set, the entry count, and `missingSources`. `configured:false` is a supported state — item names then render as ids. No public counterpart: the table is never served *as* a table. |
|
| GET | `/shard/clilocs` | cliloc-table status (`adminOnly`): every source found now (base first, then `custom/` overlays in merge order), what each contributed at the last import, readability, drift across the set, the entry count, and `missingSources`. `configured:false` is a supported state — item names then render as ids. No public counterpart: the table is never served *as* a table. |
|
||||||
| POST | `/shard/clilocs/import` | reload after a client patch or an overlay edit; `{force}` ignores the hash gate, `{approve}` accepts a **vanished** source (refused by default — see the table notes above). **A missing path — or the likely mistake of pointing at the client's own COMPRESSED `Cliloc.enu` — answers 200 with `status:"unavailable"` and a `code`, not 500.** `COMPRESSED` is called out by name: a 500 would say only "something broke", and the operator needs to be told which file to convert. |
|
| POST | `/shard/clilocs/import` | reload after a client patch or an overlay edit; `{force}` ignores the hash gate, `{approve}` accepts a **vanished** source (refused by default — see the table notes above). **A missing path — or the likely mistake of pointing at the client's own COMPRESSED `Cliloc.enu` — answers 200 with `status:"unavailable"` and a `code`, not 500.** `COMPRESSED` is called out by name: a 500 would say only "something broke", and the operator needs to be told which file to convert. |
|
||||||
|
| GET | `/shard/assets` | client-asset import status (`adminOnly`, Protocol 8): the imported body catalogue, how many sprites are on disk, how many atlas creatures resolved to a body id, and the shard's own client files beside them. `drift:true` means the client was patched. `shard.imaging.ok:false` is the named `NO_IMAGING` state — a Linux shard host with no `libgdiplus` cannot decode a sprite at all, and the reason names the package. No public counterpart: the pictures are served as ordinary files under `/uploads`. |
|
||||||
|
| POST | `/shard/assets/import` | import creature artwork from the shard's UO client; `{force}` ignores the hash gate, `{approve}` accepts a catalogue that no longer offers assets this site holds (refused by default — an unmounted client volume and a deliberate downgrade are indistinguishable, and the wrong guess deletes artwork). **This is the ONLY thing that imports** — boot deliberately never calls the shard. An operator's `spawnAtlas.art.json` always wins over an imported sprite. A body this client has no art for is **not** a failure: about half the addressable body range is in that state on a stock client. |
|
||||||
|
| POST | `/shard/assets/warm` | fetch item and land pictures this site is missing, now, instead of waiting for the warm timer (`adminOnly`, Protocol 8 phase 5). The pass works out which item pictures the site's own rows name — every distinct (ItemID, hue) on a player vendor, plus anything a character sheet has shown since the last pass — and fetches the ones it does not hold, **hued on the shard**, into `uploads/items/`. There is deliberately **no manifest and no bulk import**: the client addresses 49,152 item graphics times 3,000 hues, so the working set is what the site displays. `{force}` re-fetches pictures already held (how an operator recovers a wiped uploads volume); `{limit}` bounds one pass, default 400, because the shard serves one asset request at a time. A plugin overlay older than phase 5 answers `status:"unavailable"`, `code:"UNSUPPORTED"` with a sentence naming the fix. |
|
||||||
| PUT | `/shard/clilocs/path` | point the site at a different cliloc base file or directory (persisted as `cliloc_client_path`, which wins over `UO_CLIENT_PATH`). Overlays are read from `custom/` beside it either way. Blank clears it. Deliberately **does not import**, same reasoning as the atlas path. |
|
| PUT | `/shard/clilocs/path` | point the site at a different cliloc base file or directory (persisted as `cliloc_client_path`, which wins over `UO_CLIENT_PATH`). Overlays are read from `custom/` beside it either way. Blank clears it. Deliberately **does not import**, same reasoning as the atlas path. |
|
||||||
|
|
||||||
## 4. Shard visibility — the audience boundary (Protocol 3.0)
|
## 4. Shard visibility — the audience boundary (Protocol 3.0)
|
||||||
|
|||||||
@@ -21,7 +21,6 @@ these routes *mean* are the ones that already existed and did not move:
|
|||||||
| [`SPAWN_ATLAS.md`](../../website/SPAWN_ATLAS.md) | The bestiary / spawn atlas, parsed from the shard's own ServUO tree |
|
| [`SPAWN_ATLAS.md`](../../website/SPAWN_ATLAS.md) | The bestiary / spawn atlas, parsed from the shard's own ServUO tree |
|
||||||
| [`MARKETPLACE.md`](../../website/MARKETPLACE.md) | The player-vendor index |
|
| [`MARKETPLACE.md`](../../website/MARKETPLACE.md) | The player-vendor index |
|
||||||
| [`CLILOCS.md`](../../website/CLILOCS.md) | UO's id → name table |
|
| [`CLILOCS.md`](../../website/CLILOCS.md) | UO's id → name table |
|
||||||
| [`UOFIDDLER.md`](../../website/UOFIDDLER.md) | Operator runbook for extracting the cliloc table and creature art |
|
|
||||||
| [`../../link/PLAN.md`](../../link/PLAN.md), [`../../link/INTEGRATION.md`](../../link/INTEGRATION.md) | The wire protocol this module speaks to the sidecar |
|
| [`../../link/PLAN.md`](../../link/PLAN.md), [`../../link/INTEGRATION.md`](../../link/INTEGRATION.md) | The wire protocol this module speaks to the sidecar |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -188,10 +188,80 @@ Four column choices worth stating, because each one is a trap:
|
|||||||
Deceit"). The live `champ.update` feed in `shard_champs` is the separate answer to "it is on level 3
|
Deceit"). The live `champ.update` feed in `shard_champs` is the separate answer to "it is on level 3
|
||||||
right now". Both exist; they are not the same data.
|
right now". Both exist; they are not the same data.
|
||||||
|
|
||||||
**`shard_spawn_creatures.art` is always NULL on a fresh import.** The project ships no creature
|
**`shard_spawn_creatures.art` is DERIVED, never written by the atlas import itself** — see
|
||||||
artwork: sprites live in the operator's own client `.mul`/`.uop` files and are theirs, not ours to
|
`shard_assets` below. The project still ships no creature artwork: sprites live in the operator's own
|
||||||
redistribute. An operator supplies art via a gitignored map plus images under the (already
|
client `.mul`/`.uop` files and are theirs, not ours to redistribute. What changed in Protocol 8 is
|
||||||
gitignored) `server/uploads/atlas/`. Text-only is the normal, supported state.
|
who extracts them: the shard does, from its own client, over the bridge. An operator's hand-drawn map
|
||||||
|
plus images under the (already gitignored) `server/uploads/atlas/` still wins over anything imported.
|
||||||
|
Text-only is still the normal, supported state — an install with no shard link never imports one, and
|
||||||
|
even a complete import leaves two thirds of the playable ghost and gargoyle bodies without art.
|
||||||
|
|
||||||
|
## shard_assets / shard_creature_bodies / shard_asset_meta — the Asset Bridge (Protocol 8)
|
||||||
|
|
||||||
|
Creature artwork read from the shard's own UO client ([`link/v8.md`](../../link/v8.md) §6, §8, §12).
|
||||||
|
|
||||||
|
| Table | Shape |
|
||||||
|
|---|---|
|
||||||
|
| `shard_assets` | `asset_key` VARCHAR PK (§5's key, e.g. `body/34/a0`, `static/3922/h33`, `land/3`), `family`, `sha256`, `bytes`, `width`, `height`, `body`, `direction`, `file`, `catalog`, `imported_at` |
|
||||||
|
| `shard_creature_bodies` | `slug` PK, `type_name` (the ServUO class name asked), `body` nullable, `status`, `resolved_at` |
|
||||||
|
| `shard_asset_meta` | Singleton (`id = 1`), `payload` JSON (catalogue id, extractor version, source fingerprint, counts), `imported_at` |
|
||||||
|
|
||||||
|
**These are the one part of this schema that is deliberately NOT import-owned**, and the reason is
|
||||||
|
the atlas tables sitting directly above them. `replaceAtlas` empties and refills
|
||||||
|
`shard_spawn_creatures` on every refresh, and a refresh runs on every boot; an imported filename
|
||||||
|
stored on that row would be destroyed by an ordinary re-parse of the ServUO tree, with the next asset
|
||||||
|
Update finding the client files unchanged, reporting "nothing to do", and never restoring it. So the
|
||||||
|
assets live out here, upserted per key, and `replaceAtlas` re-derives `art` from them on the way
|
||||||
|
past — `{ ...derived, ...operatorMap }`, which is the one place "the operator's map wins" is
|
||||||
|
enforced.
|
||||||
|
|
||||||
|
Four details that are load-bearing rather than incidental:
|
||||||
|
|
||||||
|
- **`file` is a filename under the uploads directory, never a path**, and it is content-addressed
|
||||||
|
(`uo-body-34-a0-<sha8>.png`). A stable name overwritten in place would leave every browser and CDN
|
||||||
|
serving the previous client's sprite from cache, with the row perfectly correct.
|
||||||
|
- **The `art` derivation joins on the catalogue key**, `a.asset_key = CONCAT('body/', b.body, '/a0')`,
|
||||||
|
not `a.body = b.body`. The simpler join is correct today and stops being correct the moment deeper
|
||||||
|
animation keys (`body/400/a2/f0`) arrive, at which point one slug matches dozens of rows.
|
||||||
|
- **`shard_creature_bodies` IS replaced whole**, unlike `shard_assets`: it is derived from the atlas's
|
||||||
|
creature list, so a slug that has left the atlas has no meaning, and the pass that rebuilds it is a
|
||||||
|
shard round trip rather than a file transfer.
|
||||||
|
- **`status` keeps the negative answers** — `unknown` (the spawn files name a type this shard's
|
||||||
|
scripts do not define, which is real drift), `notCreature` (a spawn entry for an item or
|
||||||
|
decoration, a permanent answer), `failed`. Without them the next pass asks again, and each name
|
||||||
|
costs a real constructor on the shard's Core thread.
|
||||||
|
|
||||||
|
`shard_spawn_creatures.name` already holds the ServUO **class name** — the atlas build picks the
|
||||||
|
winning spelling of the spawn type token rather than inventing a display label — which is why the
|
||||||
|
body pass needs no extra column to ask its question.
|
||||||
|
|
||||||
|
### Item and land art: the same table, a different shape of use (phase 5)
|
||||||
|
|
||||||
|
The creature catalogue is a **set**: one manifest walk covers every key, so one stored fingerprint in
|
||||||
|
`shard_asset_meta` describes all of them and an Update is a hash diff. Item art has no set — the
|
||||||
|
client addresses 49,152 item graphics times 3,000 hues — so those rows arrive one at a time, because
|
||||||
|
something on this site named the key.
|
||||||
|
|
||||||
|
Three consequences in this schema:
|
||||||
|
|
||||||
|
- **`catalog` is per row**, and it is what phase 5 added. It records the shard's art catalogue id (a
|
||||||
|
hash of `artLegacyMUL.uop`/`art.mul`, `hues.mul`, `tiledata.mul`, `verdata.mul` and its extractor
|
||||||
|
version), so staleness is a column comparison rather than a manifest diff. A client patch changes
|
||||||
|
it; a restart does not. NULL means "written before the column existed", which is stale by the same
|
||||||
|
test and costs one re-fetch. The body import fills it too, so one column answers the question
|
||||||
|
everywhere.
|
||||||
|
- **`shard_asset_meta` stays the body catalogue's alone.** A warm pass writing there would tell the
|
||||||
|
body import that a client it never looked at is unchanged, and the creature catalogue would quietly
|
||||||
|
stop updating.
|
||||||
|
- **`family` is now load-bearing**, not decoration: `body`, `static` and `land` rows share the table
|
||||||
|
and have different lifetimes. The admin status counts them separately for the same reason — there
|
||||||
|
is no "how many are there" to compare `static` against, so the only honest number is how many the
|
||||||
|
site has been asked for and holds.
|
||||||
|
|
||||||
|
Pictures land in `server/uploads/items/` (gitignored like `uploads/atlas/`), content-addressed the
|
||||||
|
same way — `uo-static-3922-h33-<sha8>.png`. A key the shard has no art for writes **no row at all**:
|
||||||
|
an empty row would make it "held" and it would never be asked again, including after the operator
|
||||||
|
patches in the graphic that was missing.
|
||||||
|
|
||||||
## shard_clilocs / shard_cliloc_meta — UO's localization table (Protocol 3.0)
|
## shard_clilocs / shard_cliloc_meta — UO's localization table (Protocol 3.0)
|
||||||
|
|
||||||
@@ -203,29 +273,49 @@ marketplace listing — but with no table to resolve it against, the character s
|
|||||||
| Table | Shape |
|
| Table | Shape |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `shard_clilocs` | `number` INT PK, `flag`, `text` TEXT |
|
| `shard_clilocs` | `number` INT PK, `flag`, `text` TEXT |
|
||||||
| `shard_cliloc_meta` | Singleton (`id = 1`), `payload` JSON (source file, sha256, count, `parserVersion`), `imported_at` |
|
| `shard_cliloc_meta` | Singleton (`id = 1`), `payload` JSON (`source` — `bridge` or `file` — the base's fingerprint under `base`, the overlay `hashes`, per-source counts, `parserVersion`), `imported_at` |
|
||||||
|
|
||||||
Import-owned and all-or-nothing in one transaction, same contract as the atlas — including **`DELETE`,
|
Import-owned and all-or-nothing in one transaction, same contract as the atlas — including **`DELETE`,
|
||||||
not `TRUNCATE`**, for the same reason.
|
not `TRUNCATE`**, for the same reason.
|
||||||
|
|
||||||
**Sourced from files the operator supplies**, at a path from the `cliloc_client_path` setting falling
|
**The base table comes from the SHARD** on any install with uo-link configured (Protocol 8, phase 2):
|
||||||
back to `UO_CLIENT_PATH`. Nothing client-derived is committed: UO's strings are EA's, exactly as the
|
it reads its own client's compressed `Cliloc.enu` and serves the table paged over the bridge, so
|
||||||
creature sprites are. A shard with nothing configured is fully supported — names render as ids. Full
|
nothing is converted and nothing is copied to the web host. Without a shard link it falls back to a
|
||||||
design and operator guide: [`CLILOCS.md`](../../website/CLILOCS.md).
|
converted file on disk at a path from the `cliloc_client_path` setting, then `UO_CLIENT_PATH` — the
|
||||||
|
pre-protocol-8 pipeline, deprecated rather than removed so an existing install keeps working.
|
||||||
|
|
||||||
|
**Overlays are always the filesystem's**, either way: ServUO has no server-side notion of a custom
|
||||||
|
cliloc, so `custom/` is the only place shard-added ids exist and there is nothing on the shard to ask
|
||||||
|
for. That gap is in the *game*, not in this pipeline.
|
||||||
|
|
||||||
|
Nothing client-derived is committed: UO's strings are EA's, exactly as the creature sprites are. A
|
||||||
|
shard with nothing configured is fully supported — names render as ids. Full design and operator
|
||||||
|
guide: [`CLILOCS.md`](../../website/CLILOCS.md).
|
||||||
|
|
||||||
**It reads a SET of sources, not one file**, because shards edit items and add new ones and those
|
**It reads a SET of sources, not one file**, because shards edit items and add new ones and those
|
||||||
carry cliloc ids no stock client table has. A base (the converted client table) plus every overlay
|
carry cliloc ids no stock client table has. A base (from the shard, or a converted file) plus every
|
||||||
under `custom/` are re-read on every boot and hash-gated **together**, exactly as the atlas re-reads
|
overlay under `custom/` are hash-gated **together**, exactly as the atlas re-reads
|
||||||
`Regions.xml` + `Locations/*.xml` + `Spawns/*.xml` + `ChampionSpawns.xml`. Later sources win, so an
|
`Regions.xml` + `Locations/*.xml` + `Spawns/*.xml` + `ChampionSpawns.xml`. Later sources win, so an
|
||||||
overlay both adds ids and overrides stock ones, and adding one custom item never means re-exporting a
|
overlay both adds ids and overrides stock ones, and adding one custom item never means re-exporting a
|
||||||
5 MB client file. Scale, measured on the live shard: its script tree references 16,434 cliloc ids and
|
5 MB client file. Scale, measured on the live shard: its script tree references 16,434 cliloc ids and
|
||||||
only 37 are absent from stock — tens of entries against a 67k base, which is why this is an overlay
|
only 37 are absent from stock — tens of entries against a 67k base, which is why this is an overlay
|
||||||
and not a second table.
|
and not a second table.
|
||||||
|
|
||||||
The conversion step is not avoidable: **every current client ships its cliloc files compressed**
|
**The conversion step used to be unavoidable, and is not any more.** Every current client ships its
|
||||||
(first DWORD's high byte `0x8E`), and ServUO's own bundled `Ultima.StringList` cannot read that
|
cliloc files compressed (first DWORD's high byte `0x8E`) and ServUO's own bundled
|
||||||
either — so the shard cannot supply names on our behalf. The plain layout and a delimited text export
|
`Ultima.StringList` cannot read that either — which is why, for two protocol versions, the operator
|
||||||
are both accepted, sniffed by header rather than extension.
|
had to install UOFiddler, build a converter against its `Ultima.dll` and copy a 5 MB file to the web
|
||||||
|
host. Protocol 8 phase 2 ported the Mythic decompressor into the overlay, so **the shard reads its
|
||||||
|
own client and supplies the names**. The file half survives only as the fallback above, where the
|
||||||
|
plain layout and a delimited text export are both accepted, sniffed by header rather than extension.
|
||||||
|
|
||||||
|
Two consequences for what this table holds. `shard_cliloc_meta.payload` carries the base's
|
||||||
|
fingerprint under `meta.base` (the shard's file size, mtime, hash and `EXTRACTOR_VERSION`) separately
|
||||||
|
from the overlay hashes, because on the bridge the old `clilocs.plain` label is *supposed* to
|
||||||
|
disappear and a single hash map would read that upgrade as a vanished source. And **boot does not
|
||||||
|
import on the bridge path**: a file could be re-hashed locally on every restart, but asking the shard
|
||||||
|
would put a sidecar round trip in the boot sequence for a table that changes only when an operator
|
||||||
|
patches their client. Importing is an admin action.
|
||||||
|
|
||||||
Three decisions worth stating:
|
Three decisions worth stating:
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,8 @@
|
|||||||
# Cliloc table (item and title names)
|
# Cliloc table (item and title names)
|
||||||
|
|
||||||
**Status:** Complete on `edge` — website [#115](https://gitea.whitlocktech.com/RunicGateway/website/pulls/115), docs [#70](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/70).
|
**Status:** Complete. The base table now arrives **over the bridge** — protocol 8, phase 2.
|
||||||
**Design:** [`docs/link/v3.md` §8.6](../link/v3.md) — Protocol 3.0, the dependency Part B/3 was sequenced behind.
|
**Design:** [`docs/link/v3.md` §8.6](../link/v3.md) (the table itself, Protocol 3.0) and
|
||||||
|
[`docs/link/v8.md` §9](../link/v8.md) (the Asset Bridge, which retired the manual conversion).
|
||||||
|
|
||||||
A "cliloc" is UO's localization table: an integer id mapped to a display string.
|
A "cliloc" is UO's localization table: an integer id mapped to a display string.
|
||||||
**Items on the wire carry a `LabelNumber`, not a name.** The bridge has always
|
**Items on the wire carry a `LabelNumber`, not a name.** The bridge has always
|
||||||
@@ -12,86 +13,70 @@ render `id 1023721` where the game renders **"quarter staff"**.
|
|||||||
|
|
||||||
The number was never the missing piece. The table was.
|
The number was never the missing piece. The table was.
|
||||||
|
|
||||||
## Why the operator has to convert the file
|
## Where the table comes from
|
||||||
|
|
||||||
This is the awkward part, and it is not avoidable:
|
**The shard reads its own client.** A ServUO server cannot boot without a UO
|
||||||
|
client — `Config/DataPath.cfg` resolves into `Core.DataDirectories` at run time —
|
||||||
|
so the file this table is made of is already sitting on the shard host. Since
|
||||||
|
protocol 8 the plugin decompresses it and serves it over the bridge, and the site
|
||||||
|
imports it like any other shard read:
|
||||||
|
|
||||||
**Every current UO client ships its cliloc files compressed.** All eight
|
```
|
||||||
`Cliloc.*` files in a modern client (`chs`, `cht`, `deu`, `enu`, `esp`, `fra`,
|
ServUO shard ──`cliloc.table`──▶ uo-link sidecar ──`GET /cliloc`──▶ website
|
||||||
`jpn`, `kor`) begin with a DWORD whose high byte is `0x8E` — the "Mythic"
|
reads Cliloc.enu, forwards, keeps merges overlays,
|
||||||
compressed container. The plain layout this site parses is what those files
|
decompresses, pages nothing replaces the table
|
||||||
looked like *before* that change.
|
|
||||||
|
|
||||||
Decompressing it means an inverse-BWT coder with a frequency header — a few
|
|
||||||
hundred lines of bit-level work whose failure mode is plausible-looking garbage
|
|
||||||
rather than an error. The site has no business carrying that at runtime.
|
|
||||||
|
|
||||||
Two facts make the alternatives worse, not better:
|
|
||||||
|
|
||||||
- **ServUO cannot read it either.** Its bundled `Ultima.StringList` implements
|
|
||||||
only the plain layout, so on a modern client `VendorSearch.StringList` is null
|
|
||||||
and `VendorSearch.GetItemName` returns `item.Name` — usually nothing. The
|
|
||||||
shard cannot supply names on our behalf; the in-game Vendor Search gump has the
|
|
||||||
same gap.
|
|
||||||
- **Nothing client-derived may be committed.** UO's strings are EA's. The repo
|
|
||||||
ships no string table for the same reason it ships no artwork and no map
|
|
||||||
snapshot — see [`SPAWN_ATLAS.md`](SPAWN_ATLAS.md).
|
|
||||||
|
|
||||||
So the conversion happens **once, on the operator's machine, against their own
|
|
||||||
client**, and the site reads the result from a path it is given. A shard that
|
|
||||||
never does this is in a fully supported state: names render as ids, exactly as
|
|
||||||
they did before the table existed.
|
|
||||||
|
|
||||||
## Converting
|
|
||||||
|
|
||||||
> **Step-by-step operator instructions — where to get UOFiddler, where your
|
|
||||||
> client files are, and how to verify the import — are in
|
|
||||||
> [`UOFIDDLER.md`](UOFIDDLER.md).** This section covers the formats and the
|
|
||||||
> reasoning behind them.
|
|
||||||
|
|
||||||
Either format below is accepted; the site sniffs which one it was handed.
|
|
||||||
|
|
||||||
| Format | Fidelity | Notes |
|
|
||||||
|---|---|---|
|
|
||||||
| **Plain binary** (recommended) | Exact | 6-byte header, then `{int32 number, byte flag, uint16 length, UTF-8}` records |
|
|
||||||
| Delimited text | Loses leading/trailing whitespace | `number<TAB\|,\|;>text` per line; a header row, blank lines and `#` comments are ignored |
|
|
||||||
|
|
||||||
The whitespace caveat is real but cosmetic: ~1,300 of the 123,490 entries in a
|
|
||||||
stock `Cliloc.enu` are label prefixes like `"max = "` whose trailing space is
|
|
||||||
meaningful when the client concatenates a value onto them. Nothing on this site
|
|
||||||
concatenates, and every consumer passes through `displayText()`, which trims.
|
|
||||||
|
|
||||||
### Using the bundled tool
|
|
||||||
|
|
||||||
`server/tools/cliloc-export/` is a small .NET console app that drives
|
|
||||||
[UOFiddler](https://github.com/polserver/UOFiddler)'s `Ultima.dll` — the
|
|
||||||
decompressor that already exists and is already maintained — and writes the plain
|
|
||||||
format. It loads that DLL **reflectively** so it compiles against any SDK, and it
|
|
||||||
writes the records by hand because UOFiddler's own `SaveStringList` *re-compresses*
|
|
||||||
on save (its purpose is round-tripping a file back into the client, so its output
|
|
||||||
is byte-identical to its input — a trap worth knowing about).
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd website/server/tools/cliloc-export
|
|
||||||
dotnet build -c Release
|
|
||||||
|
|
||||||
# binary (recommended)
|
|
||||||
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.plain
|
|
||||||
|
|
||||||
# or tab-delimited
|
|
||||||
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv
|
|
||||||
```
|
```
|
||||||
|
|
||||||
A UOFiddler GUI export works too, but **not unmodified**: its Cliloc tab writes
|
**Why that was worth building.** Every current UO client ships its cliloc files
|
||||||
`Number;Text;Flag` — three columns, the flag *last* — and the parser reads
|
compressed: all eight `Cliloc.*` files in a modern client (`chs`, `cht`, `deu`,
|
||||||
`number<separator>text`, so the trailing field is absorbed into the name and
|
`enu`, `esp`, `fra`, `jpn`, `kor`) begin with a DWORD whose high byte is `0x8E` —
|
||||||
every item renders as `quarter staff;0`. Stripping it is one `sed`, given in
|
the "Mythic" container. The plain layout is what those files looked like *before*
|
||||||
[`UOFIDDLER.md`](UOFIDDLER.md) §Route B.
|
that change, and **ServUO's own bundled `Ultima.StringList` cannot read the new
|
||||||
|
one either**, which is why `VendorSearch.GetItemName` is inert on a modern shard
|
||||||
|
and the in-game Vendor Search gump has the same gap.
|
||||||
|
|
||||||
The parser already tolerates `number,flag,text`, with the flag in the *middle*.
|
So until protocol 8 an operator had to install UOFiddler, build a converter
|
||||||
It is not extended to cover the trailing form because a final `;0` is
|
against its `Ultima.dll`, run it over their client and copy a 5 MB file to the web
|
||||||
indistinguishable from a name that genuinely ends that way — a heuristic there
|
host — every time they patched. The decompressor now lives in the overlay
|
||||||
would corrupt real names to save the operator one command.
|
(`overlay/Scripts/Custom/Bridge/BridgeCliloc.cs`, ported from UOFiddler, which is
|
||||||
|
Beerware and therefore clean to bring into a GPL tree), so **none of that is a
|
||||||
|
step any more**.
|
||||||
|
|
||||||
|
Two things are unchanged and remain the point:
|
||||||
|
|
||||||
|
- **Nothing client-derived is committed.** UO's strings are EA's. The repo ships
|
||||||
|
no string table for the same reason it ships no artwork and no map snapshot —
|
||||||
|
see [`SPAWN_ATLAS.md`](SPAWN_ATLAS.md). The extraction happens on the
|
||||||
|
operator's own host, from their own files, for their own shard.
|
||||||
|
- **A shard with no table is fully supported.** Names render as ids, exactly as
|
||||||
|
they did before the table existed.
|
||||||
|
|
||||||
|
### What arrives, and in how many pieces
|
||||||
|
|
||||||
|
The sidecar's reply timeout is 10 s and its inbound line cap is 1 MiB, so the
|
||||||
|
table is **paged**: the shard cuts at a 512 KiB byte budget and hands back a
|
||||||
|
cursor, and the site walks it until a page says `more: false`. Measured on a stock
|
||||||
|
English client: **67,496 rows in about eleven pages**, decoded on the shard in
|
||||||
|
**290 ms**.
|
||||||
|
|
||||||
|
The rows are `{ n, f, t }` — number, flag, text. **Blanks never leave the shard**:
|
||||||
|
roughly 56,000 of a stock table's 123,490 entries are empty strings the client
|
||||||
|
reserves and never uses, the site drops them at import anyway, and sending them
|
||||||
|
would double the transfer for data that is discarded on arrival.
|
||||||
|
|
||||||
|
Only `cut: "end"` means the table finished. A short page can equally mean the byte
|
||||||
|
budget was spent, and importing a table that stopped early is the one failure that
|
||||||
|
is invisible downstream — some items named, some not, which is exactly what *no
|
||||||
|
table* looks like.
|
||||||
|
|
||||||
|
### The file pipeline is deprecated, not removed
|
||||||
|
|
||||||
|
An install with **no uo-link configured** can still be pointed at a converted file
|
||||||
|
and works exactly as it did. That path exists for shards with no bridge and for
|
||||||
|
development without a running ServUO, it accepts the same two formats it always
|
||||||
|
did (plain binary, or `number<TAB|,|;>text` delimited text), and passing an
|
||||||
|
explicit path to a refresh still selects it as a one-off. Nothing new should be
|
||||||
|
built on it.
|
||||||
|
|
||||||
## Shard-added and shard-edited items
|
## Shard-added and shard-edited items
|
||||||
|
|
||||||
@@ -103,12 +88,16 @@ atlas, which reads `Regions.xml` + `Locations/*.xml` + `Spawns/*.xml` +
|
|||||||
|
|
||||||
```
|
```
|
||||||
<cliloc path>/
|
<cliloc path>/
|
||||||
clilocs.plain ← base: the converted client table
|
|
||||||
custom/
|
custom/
|
||||||
01-uomysticmoon.tsv ← overlays: shard additions and overrides
|
01-uomysticmoon.tsv ← overlays: shard additions and overrides
|
||||||
02-events.tsv
|
02-events.tsv
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The base is no longer a file in that directory — it comes from the shard — so on
|
||||||
|
a bridged install the configured path selects **only** where `custom/` is read
|
||||||
|
from. (Without a shard link, a converted base file sitting beside `custom/` is
|
||||||
|
still found, which is the deprecated pipeline above.)
|
||||||
|
|
||||||
Overlays use the same delimited-text format, are read in **sorted order**, and
|
Overlays use the same delimited-text format, are read in **sorted order**, and
|
||||||
**later sources win** — so an overlay both *adds* ids the client never had and
|
**later sources win** — so an overlay both *adds* ids the client never had and
|
||||||
*overrides* stock ones the shard has re-purposed. Any `.tsv`, `.csv`, `.txt`,
|
*overrides* stock ones the shard has re-purposed. Any `.tsv`, `.csv`, `.txt`,
|
||||||
@@ -116,8 +105,9 @@ Overlays use the same delimited-text format, are read in **sorted order**, and
|
|||||||
say) is ignored.
|
say) is ignored.
|
||||||
|
|
||||||
Adding, editing or removing any overlay counts as drift, so a new custom item
|
Adding, editing or removing any overlay counts as drift, so a new custom item
|
||||||
needs only a file edit and a restart — or the admin panel's Import button.
|
needs only a file edit and the admin panel's Import button. **Adding one item
|
||||||
**Adding one item never means re-exporting a 5 MB client file.**
|
never means re-reading the client table** — though on the bridge that is now
|
||||||
|
cheap enough not to matter much.
|
||||||
|
|
||||||
The import result reports what each source contributed, which is how you confirm
|
The import result reports what each source contributed, which is how you confirm
|
||||||
an overlay took effect — `overrode: 0` on a file meant to re-label stock items
|
an overlay took effect — `overrode: 0` on a file meant to re-label stock items
|
||||||
@@ -125,11 +115,14 @@ says it did not:
|
|||||||
|
|
||||||
```json
|
```json
|
||||||
"sources": [
|
"sources": [
|
||||||
{ "label": "clilocs.plain", "kind": "base", "entries": 123490, "added": 123490, "overrode": 0 },
|
{ "label": "cliloc.enu", "kind": "shard", "entries": 67496, "added": 67496, "overrode": 0 },
|
||||||
{ "label": "custom/uomysticmoon.tsv", "kind": "custom", "entries": 2, "added": 1, "overrode": 1 }
|
{ "label": "custom/uomysticmoon.tsv", "kind": "custom", "entries": 2, "added": 1, "overrode": 1 }
|
||||||
]
|
]
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`kind` says where a source came from: `shard` over the bridge, `base` a converted
|
||||||
|
file on disk, `custom` an overlay.
|
||||||
|
|
||||||
**Why a convention rather than discovery.** Everywhere else this pipeline follows
|
**Why a convention rather than discovery.** Everywhere else this pipeline follows
|
||||||
the shard's own files, but **ServUO has no server-side notion of a custom
|
the shard's own files, but **ServUO has no server-side notion of a custom
|
||||||
cliloc** — they live in the patched client a shard distributes to its players,
|
cliloc** — they live in the patched client a shard distributes to its players,
|
||||||
@@ -143,28 +136,29 @@ ids and only **37** are absent from the stock client table — tens of entries
|
|||||||
against a 67k base, which is what makes an overlay the right shape rather than a
|
against a 67k base, which is what makes an overlay the right shape rather than a
|
||||||
second full table.
|
second full table.
|
||||||
|
|
||||||
## Configuring the path
|
## Choosing the source
|
||||||
|
|
||||||
Two ways to point at the sources, the setting winning over the environment:
|
Nothing to configure: **the shard wins whenever uo-link is configured and
|
||||||
|
enabled.** There is no mode setting, because there is no version of this question
|
||||||
|
an operator benefits from answering — a shard that can serve its own client table
|
||||||
|
is strictly better than a file somebody converted by hand months ago.
|
||||||
|
|
||||||
| Source | Notes |
|
The two escape hatches, both deliberate:
|
||||||
|
|
||||||
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `cliloc_client_path` setting | Admin-editable (Admin → Shard); takes effect on the next refresh without a redeploy |
|
| No uo-link configured | The file pipeline, exactly as before |
|
||||||
| `UO_CLIENT_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |
|
| An explicit `path` passed to a refresh | A one-off "import from this file", which the shard never overrules |
|
||||||
|
|
||||||
The value may be **the base file itself or a directory to search**, because both
|
`cliloc_client_path` (setting, admin-editable) and `UO_CLIENT_PATH` (env
|
||||||
are natural answers to "where is it". Overlays are read from a `custom/`
|
deploy-time default) still name a path, and the setting still wins over the
|
||||||
directory beside the base **either way** — pointing at a file does not forfeit
|
environment. What that path *means* narrowed: on a bridged install it is where
|
||||||
them.
|
`custom/` overlays live. Without a link it is also searched for a base file,
|
||||||
|
case-insensitively (the client writes `Cliloc.enu` on Windows; the site usually
|
||||||
A directory is searched case-insensitively (the client writes `Cliloc.enu` on
|
runs on Linux), in order: `clilocs.tsv`, `clilocs.csv`, `clilocs.plain`,
|
||||||
Windows; the site usually runs on Linux) for, in order: `clilocs.tsv`,
|
`cliloc.plain`, `cliloc.plain.enu`, `cliloc.enu.plain`, `clilocs.txt`,
|
||||||
`clilocs.csv`, `clilocs.plain`, `cliloc.plain`, `cliloc.plain.enu`,
|
`cliloc.enu` — and pointing it straight at an unconverted client directory still
|
||||||
`cliloc.enu.plain`, `clilocs.txt`, `cliloc.enu`.
|
answers by name rather than failing obscurely:
|
||||||
|
|
||||||
That ordering puts explicitly-converted names first on purpose. Pointing the
|
|
||||||
setting straight at an unconverted client directory finds `cliloc.enu`, which is
|
|
||||||
compressed — and the site says so by name rather than failing obscurely:
|
|
||||||
|
|
||||||
```
|
```
|
||||||
status: unavailable
|
status: unavailable
|
||||||
@@ -175,16 +169,25 @@ reason: This is a compressed (Mythic-format) cliloc file, which the site cannot
|
|||||||
|
|
||||||
## Refresh contract
|
## Refresh contract
|
||||||
|
|
||||||
Identical in shape to the spawn atlas, and for the same reasons:
|
- **It never blocks startup**, and on the bridge it never *touches* startup: boot
|
||||||
|
imports nothing when the shard is the source. A sidecar round trip in the boot
|
||||||
- **It never blocks startup.** No path, an unreadable file, a wrong-format file,
|
sequence would be spent answering "no" on every restart but the one after a
|
||||||
a database error — all caught and logged. The site comes up either way.
|
client patch — and patching a client is an operator action, so importing is an
|
||||||
- **Hash-gated.** The boot path hashes the file and skips the parse entirely when
|
operator action. **Admin → Shard → Import** is the button. Whatever table is
|
||||||
it matches what is loaded, which is every restart that did not follow a client
|
loaded keeps serving until then.
|
||||||
patch. Measured on a stock table: **14 ms** for the no-op, **663 ms** for a full
|
- **Still hash-gated**, so pressing Import when nothing changed costs one small
|
||||||
parse and replace.
|
call. The gate is the shard's own `assets.sources`: size, mtime and content
|
||||||
- **A `PARSER_VERSION` bump also counts as drift**, so a corrected parse reaches
|
hash of `Cliloc.enu` plus the shard's `EXTRACTOR_VERSION`. A `sha256` of `null`
|
||||||
an install whose client never patches.
|
with `hashing: true` means the shard has not computed it yet (it hashes off the
|
||||||
|
request path, because the art and animation files it also reports are 343 MB)
|
||||||
|
— that means *ask again*, never *changed*, and the comparison falls back to
|
||||||
|
(size, mtime) meanwhile.
|
||||||
|
- **Without a link** the file path is unchanged: boot hashes the local file and
|
||||||
|
skips the parse when it matches. Measured on a stock table: **14 ms** for the
|
||||||
|
no-op, **663 ms** for a full parse and replace.
|
||||||
|
- **A `PARSER_VERSION` bump counts as drift**, and so does an `EXTRACTOR_VERSION`
|
||||||
|
bump on the shard — same argument at the other end of the wire: a corrected
|
||||||
|
reader must reach an install whose client never patches.
|
||||||
|
|
||||||
### Two ways a refresh is refused
|
### Two ways a refresh is refused
|
||||||
|
|
||||||
@@ -201,7 +204,7 @@ and the rows already loaded are untouched. A malformed overlay names the file it
|
|||||||
came from (`custom/broken.tsv: No cliloc entries found…`), because "which of my
|
came from (`custom/broken.tsv: No cliloc entries found…`), because "which of my
|
||||||
six overlay files is broken" is otherwise a guessing game.
|
six overlay files is broken" is otherwise a guessing game.
|
||||||
|
|
||||||
**A source that has VANISHED** is the hazard a single file did not have. It
|
**An OVERLAY that has VANISHED** is the hazard a single file did not have. It
|
||||||
parses perfectly and imports a table quietly missing everything that file
|
parses perfectly and imports a table quietly missing everything that file
|
||||||
contributed — and an unmounted volume looks exactly like a deliberate deletion
|
contributed — and an unmounted volume looks exactly like a deliberate deletion
|
||||||
from here. This is the same ambiguity the atlas stages a facet removal for, so it
|
from here. This is the same ambiguity the atlas stages a facet removal for, so it
|
||||||
@@ -209,11 +212,28 @@ is escalated rather than applied:
|
|||||||
|
|
||||||
```
|
```
|
||||||
status: needsReview
|
status: needsReview
|
||||||
reason: 1 previously-loaded cliloc source(s) are missing;
|
reason: 1 previously-loaded cliloc overlay(s) are missing;
|
||||||
the existing table is unchanged
|
the existing table is unchanged
|
||||||
missingSources: ["custom/uomysticmoon.tsv"]
|
missingSources: ["custom/uomysticmoon.tsv"]
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**The base is deliberately exempt from that question**, and that is an upgrade
|
||||||
|
detail worth stating: an install that used the converted-file pipeline carries its
|
||||||
|
base file's label in the stored fingerprint, and on the bridge that label is
|
||||||
|
*supposed* to disappear. Counting it as a vanished source would make the first
|
||||||
|
import after the upgrade demand an approval for a change the upgrade itself made.
|
||||||
|
|
||||||
|
**A client patched mid-import** is refused outright rather than staged, because
|
||||||
|
there is nothing to decide: every page echoes the source file's size and mtime, and
|
||||||
|
if they move between pages then half of what arrived came from a file that no
|
||||||
|
longer exists and nothing later can tell which half.
|
||||||
|
|
||||||
|
```
|
||||||
|
status: unavailable
|
||||||
|
code: SOURCE_CHANGED
|
||||||
|
reason: The shard's cliloc file changed while it was being read; nothing was imported
|
||||||
|
```
|
||||||
|
|
||||||
`status()` reports `missingSources` too, so the panel can show it before anyone
|
`status()` reports `missingSources` too, so the panel can show it before anyone
|
||||||
clicks Import. An admin accepts it by re-running the import with
|
clicks Import. An admin accepts it by re-running the import with
|
||||||
`{ "approve": true }`.
|
`{ "approve": true }`.
|
||||||
@@ -229,15 +249,20 @@ admin was already going to run.
|
|||||||
|
|
||||||
| | |
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
| Parsed from a stock `Cliloc.enu` | **123,490** entries |
|
| Entries in a stock `Cliloc.enu` | **123,490** |
|
||||||
| Of those, empty strings | **55,994** (ids the client reserves and never uses) |
|
| Of those, empty strings | **55,994** (ids the client reserves and never uses) |
|
||||||
| Stored in `shard_clilocs` | **67,496** |
|
| Stored in `shard_clilocs` | **67,496** |
|
||||||
|
|
||||||
Blank entries are dropped at import. A row resolving to no name is
|
Blank entries are dropped, and since protocol 8 they are dropped **on the shard**,
|
||||||
indistinguishable from no row at all to every caller, and dropping them makes the
|
before they reach the wire. A row resolving to no name is indistinguishable from
|
||||||
binary and text imports converge on **identical** content — the binary format
|
no row at all to every caller, so sending 56,000 of them would double the
|
||||||
carries the blanks explicitly and a text export may or may not, depending on the
|
transfer for data discarded on arrival. The site still filters at import, because
|
||||||
tool. Verified: both formats import to the same 67,496 rows with the same keys.
|
an overlay file can carry one and because the file pipeline still exists.
|
||||||
|
|
||||||
|
That the shard's own decoder lands on **exactly 67,496** is also the strongest
|
||||||
|
check there is that the ported decompressor is correct: the number was measured
|
||||||
|
first through UOFiddler's `Ultima.dll` against this same client, by an entirely
|
||||||
|
different implementation.
|
||||||
|
|
||||||
`text` is `TEXT`, not `VARCHAR`: the long property descriptions reach 12 KB, and
|
`text` is `TEXT`, not `VARCHAR`: the long property descriptions reach 12 KB, and
|
||||||
silently truncating them would be worse than storing them. The index that matters
|
silently truncating them would be worse than storing them. The index that matters
|
||||||
@@ -297,13 +322,24 @@ All admin-only, alongside the atlas under Admin → Shard:
|
|||||||
|
|
||||||
| Route | Purpose |
|
| Route | Purpose |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `GET /api/v1/admin/shard/clilocs` | Sources found, what each contributed at the last import, readability, drift, entry count, `missingSources` |
|
| `GET /api/v1/admin/shard/clilocs` | Which source is in use (`bridge` / `file`), the shard's client-file fingerprint, what each source contributed at the last import, drift, entry count, `missingSources` |
|
||||||
| `POST /api/v1/admin/shard/clilocs/import` | Reload after a client patch or an overlay edit; `{ "force": true }` reimports an unchanged set, `{ "approve": true }` accepts a vanished source |
|
| `POST /api/v1/admin/shard/clilocs/import` | Reload after a client patch or an overlay edit; `{ "force": true }` reimports an unchanged set, `{ "approve": true }` accepts a vanished overlay |
|
||||||
| `PUT /api/v1/admin/shard/clilocs/path` | Set the path; blank disables resolution |
|
| `PUT /api/v1/admin/shard/clilocs/path` | Set the overlay path (and, with no shard link, the base file's); blank clears it |
|
||||||
|
|
||||||
A refresh **result is not an exception**: a missing file, or the likely mistake of
|
**Import is now the only thing that refreshes the table on a bridged install**,
|
||||||
pointing at the client's own compressed `Cliloc.enu`, answers `200` with
|
since boot no longer asks the shard. It is the button an operator presses after
|
||||||
`status: "unavailable"` and a reason. A `500` would say only "something broke";
|
patching their client.
|
||||||
the operator needs to be told which file to convert. Setting the path
|
|
||||||
deliberately does **not** import as a side effect — the response carries the
|
A refresh **result is not an exception**, and protocol 8 widened the set of things
|
||||||
refreshed status so the panel can offer that as the next step.
|
that covers: a shard that is down, an asset plane the operator has switched off
|
||||||
|
(`Bridge.AssetsEnabled`), a client with no cliloc file, a client patched halfway
|
||||||
|
through the import, plus everything the file pipeline could already report. Each
|
||||||
|
answers `200` with `status: "unavailable"` and a reason naming what to fix. A
|
||||||
|
`500` would say only "something broke". Setting the path deliberately does **not**
|
||||||
|
import as a side effect — the response carries the refreshed status so the panel
|
||||||
|
can offer that as the next step.
|
||||||
|
|
||||||
|
The sidecar's own statuses are worth knowing when reading a log: **425** is the
|
||||||
|
shard saying it is busy with another asset request (flow control, and the ordinary
|
||||||
|
answer mid-import — the site retries), **403** the asset plane switched off, **404**
|
||||||
|
a client with no such file, **422** a file it has and cannot decode.
|
||||||
|
|||||||
@@ -15,6 +15,14 @@ UO actions, the integrations, the authoring UI, the public surface — needed no
|
|||||||
Those fourteen phases reach the game only to *announce*, over verbs the write plane already carries; nothing in them creates or
|
Those fourteen phases reach the game only to *announce*, over verbs the write plane already carries; nothing in them creates or
|
||||||
changes a thing in the world.
|
changes a thing in the world.
|
||||||
|
|
||||||
|
**COMPLETE as of 2026-09-10.** All seventeen phases are built and on `main` in every repository they
|
||||||
|
touch. P16 ran as three legs — **16a** the acceptance walk from `edge`, **16b** the six-step cutover
|
||||||
|
and the re-verify against released artefacts, **16c** `runicgateway.com` and `.profile` — and 16c
|
||||||
|
opened by closing the seventh cutover step 16b had left standing (`docs#232`). The platform the
|
||||||
|
workstream leaves behind is sidecar **v2.2.0**, overlay **v1.2.0**, bundle **2026.09.10** on
|
||||||
|
**protocol 7**, `Module-uo` **v1.2.2**, and `MODULE_API_VERSION` **1.10.0**. Each phase's record is
|
||||||
|
in its own section below; `edge` stays standing, unused, in every repository.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Before anything: three facts about the ground
|
## Before anything: three facts about the ground
|
||||||
@@ -2025,6 +2033,75 @@ core, docs, then the kit's re-pin and `runicgateway.com`.
|
|||||||
- **`.profile`** — the org landing page is updated when the *shape* of the project changes, which a new
|
- **`.profile`** — the org landing page is updated when the *shape* of the project changes, which a new
|
||||||
subsystem is.
|
subsystem is.
|
||||||
|
|
||||||
|
> **16c BUILT (2026-09-09/10) — and it began by closing the cutover's missing seventh step.**
|
||||||
|
> `runicgateway.com#30` and `.profile#6`, both onto `main`; the leg's first act was `docs#232`.
|
||||||
|
>
|
||||||
|
> #### The step 16b left on `edge`
|
||||||
|
>
|
||||||
|
> Six repositories were cut over and every one of them showed **0 commits on `edge` that are not on
|
||||||
|
> `main`**. This one showed **46**. Step 6 (`#231`) landed the record *on* `edge` rather than cutting
|
||||||
|
> `edge` over — an easy thing to miss, because the step's own PR merged green and closed. The
|
||||||
|
> consequence was quiet and total: `docs` `main` opened `EVENTS.md` with *"revision 5. **No code
|
||||||
|
> written.** Read against … `MODULE_API_VERSION` 1.9.0 · sidecar protocol 5"* while six repositories
|
||||||
|
> shipped the engine on protocol 7, and `link/v6.md` and `v7.md` — the specs of record for two
|
||||||
|
> protocol versions — existed on no default branch anywhere.
|
||||||
|
>
|
||||||
|
> **Nothing in this workstream could have caught it.** Every check that guards a contract lives in
|
||||||
|
> the repository that *depends* on the contract, and a documentation repository has no dependants.
|
||||||
|
> What found it was the one check that reads `docs` from outside: `runicgateway.com`'s
|
||||||
|
> `checkReference.mjs` asserts every canonical document it names still exists on `main`, and adding
|
||||||
|
> the current protocol spec failed with `✗ canonical doc link/v7.md`. **The site is the docs
|
||||||
|
> repository's only dependant, and 16c is the only phase that would ever have run that check.**
|
||||||
|
>
|
||||||
|
> The merge was clean, and both `PROJECT_TREE.md` files stayed on `main`'s newer automated syncs —
|
||||||
|
> `edge` never edited them, so git kept `main`'s side. `edge` stays standing, per 16b's decision,
|
||||||
|
> now four generated commits behind.
|
||||||
|
>
|
||||||
|
> #### The site
|
||||||
|
>
|
||||||
|
> The checks were red before the phase started and named their own answers, which is the whole
|
||||||
|
> bargain §12 of that repository's plan struck: nine `checkFacts` values (protocol 5 → **7** in all
|
||||||
|
> three declaration sites, `moduleApi` → **1.10.0**, the bundle triple, `link` **v2.2.0**, `Module-uo`
|
||||||
|
> **v1.2.2** — a third module release, one past the v1.2.1 the cutover cut), and **twenty-seven
|
||||||
|
> `Bridge.cfg` keys** the site listed nowhere. Those became five groups rather than an appendix,
|
||||||
|
> because `EventsEnabled` is a *second consent switch* and belongs beside the ceilings it governs
|
||||||
|
> rather than filed under `AdminWriteEnabled`.
|
||||||
|
>
|
||||||
|
> Two pages, the treatment Teams has: **Scheduled events** under Administration and **Events
|
||||||
|
> architecture**. Two capability entries, so the homepage, `/features/` and `/modules/` stop omitting
|
||||||
|
> the subsystem — and the calendar one is deliberately **not** `needsModule`, because a bare core can
|
||||||
|
> author and run an event and that marker means "present, correct and permanently empty".
|
||||||
|
>
|
||||||
|
> **`/privacy` owed a row and had none.** `event_run_participants` is personal data — scores and
|
||||||
|
> ranks against a module-opaque member key, linked to an account where one is linked, feeding a
|
||||||
|
> participant's own history. The new `deploy-events` row states the retention exactly, including the
|
||||||
|
> asymmetry that matters: the diagnostic log is swept after 90 days **and only on terminal runs**,
|
||||||
|
> while the run, its steps and its participants are never swept, because they are the record of what
|
||||||
|
> was done to a shared world.
|
||||||
|
>
|
||||||
|
> **A naming collision worth fixing while it was cheap.** `reference/event-catalog` is about what a
|
||||||
|
> shard *emits*; with a scheduled-event system shipped, two things in the documentation were called
|
||||||
|
> an event catalog. Retitled **"Shard event catalog"**, with the route left alone so nothing outside
|
||||||
|
> that repository breaks — and the page now opens by saying which of the two it is, since the kinds
|
||||||
|
> it lists are exactly what a phase can wait for.
|
||||||
|
>
|
||||||
|
> **No screenshots**, and stated as a choice: capturing the events surfaces means standing the whole
|
||||||
|
> rig back up for images no check requires, and the engagement workstream's site leg added none
|
||||||
|
> either.
|
||||||
|
>
|
||||||
|
> #### `.profile`
|
||||||
|
>
|
||||||
|
> One bullet, and two stale numbers. The bullet says the posture rather than the feature list — off
|
||||||
|
> by default, caps in the database, cleanup generated from a ledger, and *an event does not edit the
|
||||||
|
> world, it holds a lease the game restores on its own deadline*. The numbers are protocol **5 → 7**
|
||||||
|
> in the four values the installer prints (the block a reader copies into Admin → Shard, where a
|
||||||
|
> wrong number is a pairing failure with no obvious cause) and **module-uo v1.1.0 → v1.2.2**.
|
||||||
|
>
|
||||||
|
> #### What 16c did not need
|
||||||
|
>
|
||||||
|
> No `EVENTS.md` change, no `MODULE_API_VERSION` change, no protocol change, no release. The phase
|
||||||
|
> moves no contract — it makes the ones already moved legible from outside the organisation.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## What this plan does not do
|
## What this plan does not do
|
||||||
|
|||||||
@@ -169,6 +169,8 @@ website/
|
|||||||
│ │ ├── lib/
|
│ │ ├── lib/
|
||||||
│ │ │ ├── adminNav.js
|
│ │ │ ├── adminNav.js
|
||||||
│ │ │ ├── engagementRules.js
|
│ │ │ ├── engagementRules.js
|
||||||
|
│ │ │ ├── eventAuthoring.js
|
||||||
|
│ │ │ ├── eventCalendar.js
|
||||||
│ │ │ ├── format.js
|
│ │ │ ├── format.js
|
||||||
│ │ │ ├── heroLayout.js
|
│ │ │ ├── heroLayout.js
|
||||||
│ │ │ ├── moduleAdmin.js
|
│ │ │ ├── moduleAdmin.js
|
||||||
@@ -216,6 +218,11 @@ website/
|
|||||||
│ │ │ │ │ ├── EngagementSuppressions.jsx
|
│ │ │ │ │ ├── EngagementSuppressions.jsx
|
||||||
│ │ │ │ │ ├── EngagementTemplates.jsx
|
│ │ │ │ │ ├── EngagementTemplates.jsx
|
||||||
│ │ │ │ │ ├── EngagementTriggers.jsx
|
│ │ │ │ │ ├── EngagementTriggers.jsx
|
||||||
|
│ │ │ │ │ ├── EventActions.jsx
|
||||||
|
│ │ │ │ │ ├── EventEditor.jsx
|
||||||
|
│ │ │ │ │ ├── EventRun.jsx
|
||||||
|
│ │ │ │ │ ├── EventsAdmin.jsx
|
||||||
|
│ │ │ │ │ ├── EventsCalendar.jsx
|
||||||
│ │ │ │ │ ├── HeroEditor.jsx
|
│ │ │ │ │ ├── HeroEditor.jsx
|
||||||
│ │ │ │ │ ├── InvitesAdmin.jsx
|
│ │ │ │ │ ├── InvitesAdmin.jsx
|
||||||
│ │ │ │ │ ├── Moderation.jsx
|
│ │ │ │ │ ├── Moderation.jsx
|
||||||
@@ -246,6 +253,7 @@ website/
|
|||||||
│ │ │ │ ├── ForgotPassword.jsx
|
│ │ │ │ ├── ForgotPassword.jsx
|
||||||
│ │ │ │ ├── PlayerAccount.jsx
|
│ │ │ │ ├── PlayerAccount.jsx
|
||||||
│ │ │ │ ├── PlayerAppeals.jsx
|
│ │ │ │ ├── PlayerAppeals.jsx
|
||||||
|
│ │ │ │ ├── PlayerEvents.jsx
|
||||||
│ │ │ │ ├── PlayerInbox.jsx
|
│ │ │ │ ├── PlayerInbox.jsx
|
||||||
│ │ │ │ ├── PlayerLogin.jsx
|
│ │ │ │ ├── PlayerLogin.jsx
|
||||||
│ │ │ │ ├── PlayerNotifications.jsx
|
│ │ │ │ ├── PlayerNotifications.jsx
|
||||||
@@ -258,6 +266,9 @@ website/
|
|||||||
│ │ │ ├── public/
|
│ │ │ ├── public/
|
||||||
│ │ │ │ ├── About.jsx
|
│ │ │ │ ├── About.jsx
|
||||||
│ │ │ │ ├── CmsPage.jsx
|
│ │ │ │ ├── CmsPage.jsx
|
||||||
|
│ │ │ │ ├── EventPage.jsx
|
||||||
|
│ │ │ │ ├── Events.jsx
|
||||||
|
│ │ │ │ ├── EventSeries.jsx
|
||||||
│ │ │ │ ├── FiveOnFriday.jsx
|
│ │ │ │ ├── FiveOnFriday.jsx
|
||||||
│ │ │ │ ├── Maintenance.jsx
|
│ │ │ │ ├── Maintenance.jsx
|
||||||
│ │ │ │ ├── News.jsx
|
│ │ │ │ ├── News.jsx
|
||||||
@@ -279,6 +290,8 @@ website/
|
|||||||
│ │ ├── apiClient.test.js
|
│ │ ├── apiClient.test.js
|
||||||
│ │ ├── emailTemplates.test.js
|
│ │ ├── emailTemplates.test.js
|
||||||
│ │ ├── engagementRules.test.js
|
│ │ ├── engagementRules.test.js
|
||||||
|
│ │ ├── eventAuthoring.test.js
|
||||||
|
│ │ ├── eventCalendar.test.js
|
||||||
│ │ ├── featureGate.test.js
|
│ │ ├── featureGate.test.js
|
||||||
│ │ ├── format.test.js
|
│ │ ├── format.test.js
|
||||||
│ │ ├── heroLayout.test.js
|
│ │ ├── heroLayout.test.js
|
||||||
@@ -352,6 +365,7 @@ website/
|
|||||||
│ │ │ └── validateBlocks.js
|
│ │ │ └── validateBlocks.js
|
||||||
│ │ ├── config/
|
│ │ ├── config/
|
||||||
│ │ │ ├── brand.js
|
│ │ │ ├── brand.js
|
||||||
|
│ │ │ ├── coreEventActions.js
|
||||||
│ │ │ ├── coreStreams.js
|
│ │ │ ├── coreStreams.js
|
||||||
│ │ │ ├── coreTriggers.js
|
│ │ │ ├── coreTriggers.js
|
||||||
│ │ │ ├── csp.js
|
│ │ │ ├── csp.js
|
||||||
@@ -393,6 +407,18 @@ website/
|
|||||||
│ │ │ ├── suppressions.js
|
│ │ │ ├── suppressions.js
|
||||||
│ │ │ ├── templates.js
|
│ │ │ ├── templates.js
|
||||||
│ │ │ └── templateSeeds.js
|
│ │ │ └── templateSeeds.js
|
||||||
|
│ │ ├── events/
|
||||||
|
│ │ │ ├── announce.js
|
||||||
|
│ │ │ ├── authorize.js
|
||||||
|
│ │ │ ├── cleanup.js
|
||||||
|
│ │ │ ├── dispatch.js
|
||||||
|
│ │ │ ├── gates.js
|
||||||
|
│ │ │ ├── ledger.js
|
||||||
|
│ │ │ ├── participants.js
|
||||||
|
│ │ │ ├── price.js
|
||||||
|
│ │ │ ├── recurrence.js
|
||||||
|
│ │ │ ├── spec.js
|
||||||
|
│ │ │ └── verify.js
|
||||||
│ │ ├── middleware/
|
│ │ ├── middleware/
|
||||||
│ │ │ ├── botScore.js
|
│ │ │ ├── botScore.js
|
||||||
│ │ │ ├── loginProtection.js
|
│ │ │ ├── loginProtection.js
|
||||||
@@ -442,6 +468,25 @@ website/
|
|||||||
│ │ │ │ ├── engagementSuppressions.db.js
|
│ │ │ │ ├── engagementSuppressions.db.js
|
||||||
│ │ │ │ ├── engagementTemplates.db.js
|
│ │ │ │ ├── engagementTemplates.db.js
|
||||||
│ │ │ │ └── engagementTemplates.model.js
|
│ │ │ │ └── engagementTemplates.model.js
|
||||||
|
│ │ │ ├── events/
|
||||||
|
│ │ │ │ ├── eventActionSettings.db.js
|
||||||
|
│ │ │ │ ├── eventCalendar.model.js
|
||||||
|
│ │ │ │ ├── eventDefinitions.db.js
|
||||||
|
│ │ │ │ ├── eventDefinitions.model.js
|
||||||
|
│ │ │ │ ├── eventJson.js
|
||||||
|
│ │ │ │ ├── eventPhaseGates.db.js
|
||||||
|
│ │ │ │ ├── eventPublic.model.js
|
||||||
|
│ │ │ │ ├── eventRunBudget.db.js
|
||||||
|
│ │ │ │ ├── eventRunControls.model.js
|
||||||
|
│ │ │ │ ├── eventRunLog.db.js
|
||||||
|
│ │ │ │ ├── eventRunParticipants.db.js
|
||||||
|
│ │ │ │ ├── eventRunResources.db.js
|
||||||
|
│ │ │ │ ├── eventRuns.db.js
|
||||||
|
│ │ │ │ ├── eventRuns.model.js
|
||||||
|
│ │ │ │ ├── eventRunSteps.db.js
|
||||||
|
│ │ │ │ ├── eventSeries.db.js
|
||||||
|
│ │ │ │ ├── eventSeries.model.js
|
||||||
|
│ │ │ │ └── eventVersions.db.js
|
||||||
│ │ │ ├── invites/
|
│ │ │ ├── invites/
|
||||||
│ │ │ │ ├── invites.db.js
|
│ │ │ │ ├── invites.db.js
|
||||||
│ │ │ │ └── invites.model.js
|
│ │ │ │ └── invites.model.js
|
||||||
@@ -559,6 +604,8 @@ website/
|
|||||||
│ │ │ │ │ ├── emailConfig.controller.js
|
│ │ │ │ │ ├── emailConfig.controller.js
|
||||||
│ │ │ │ │ ├── engagement.controller.js
|
│ │ │ │ │ ├── engagement.controller.js
|
||||||
│ │ │ │ │ ├── engagement.router.js
|
│ │ │ │ │ ├── engagement.router.js
|
||||||
|
│ │ │ │ │ ├── events.controller.js
|
||||||
|
│ │ │ │ │ ├── events.router.js
|
||||||
│ │ │ │ │ ├── imageUpload.js
|
│ │ │ │ │ ├── imageUpload.js
|
||||||
│ │ │ │ │ ├── index.js
|
│ │ │ │ │ ├── index.js
|
||||||
│ │ │ │ │ ├── invites.controller.js
|
│ │ │ │ │ ├── invites.controller.js
|
||||||
@@ -607,6 +654,8 @@ website/
|
|||||||
│ │ │ │ ├── player/
|
│ │ │ │ ├── player/
|
||||||
│ │ │ │ │ ├── appeals.controller.js
|
│ │ │ │ │ ├── appeals.controller.js
|
||||||
│ │ │ │ │ ├── appeals.router.js
|
│ │ │ │ │ ├── appeals.router.js
|
||||||
|
│ │ │ │ │ ├── events.controller.js
|
||||||
|
│ │ │ │ │ ├── events.router.js
|
||||||
│ │ │ │ │ ├── index.js
|
│ │ │ │ │ ├── index.js
|
||||||
│ │ │ │ │ ├── teamForum.controller.js
|
│ │ │ │ │ ├── teamForum.controller.js
|
||||||
│ │ │ │ │ ├── teamForum.router.js
|
│ │ │ │ │ ├── teamForum.router.js
|
||||||
@@ -615,6 +664,8 @@ website/
|
|||||||
│ │ │ │ ├── public/
|
│ │ │ │ ├── public/
|
||||||
│ │ │ │ │ ├── engagement.controller.js
|
│ │ │ │ │ ├── engagement.controller.js
|
||||||
│ │ │ │ │ ├── engagement.router.js
|
│ │ │ │ │ ├── engagement.router.js
|
||||||
|
│ │ │ │ │ ├── events.controller.js
|
||||||
|
│ │ │ │ │ ├── events.router.js
|
||||||
│ │ │ │ │ ├── index.js
|
│ │ │ │ │ ├── index.js
|
||||||
│ │ │ │ │ ├── modules.controller.js
|
│ │ │ │ │ ├── modules.controller.js
|
||||||
│ │ │ │ │ ├── modules.router.js
|
│ │ │ │ │ ├── modules.router.js
|
||||||
@@ -646,6 +697,7 @@ website/
|
|||||||
│ │ │ ├── engagementEmit.js
|
│ │ │ ├── engagementEmit.js
|
||||||
│ │ │ ├── engagementRetentionPrune.js
|
│ │ │ ├── engagementRetentionPrune.js
|
||||||
│ │ │ ├── engagementWorker.js
|
│ │ │ ├── engagementWorker.js
|
||||||
|
│ │ │ ├── eventRunner.js
|
||||||
│ │ │ ├── forumHtml.js
|
│ │ │ ├── forumHtml.js
|
||||||
│ │ │ ├── htmlShell.js
|
│ │ │ ├── htmlShell.js
|
||||||
│ │ │ ├── logger.js
|
│ │ │ ├── logger.js
|
||||||
@@ -718,6 +770,27 @@ website/
|
|||||||
│ │ ├── engagementRetentionSql.test.js
|
│ │ ├── engagementRetentionSql.test.js
|
||||||
│ │ ├── engagementTemplatesAdmin.test.js
|
│ │ ├── engagementTemplatesAdmin.test.js
|
||||||
│ │ ├── engagementTriggers.test.js
|
│ │ ├── engagementTriggers.test.js
|
||||||
|
│ │ ├── eventActionRegistry.test.js
|
||||||
|
│ │ ├── eventAnnounce.test.js
|
||||||
|
│ │ ├── eventAuthorize.test.js
|
||||||
|
│ │ ├── eventCleanup.test.js
|
||||||
|
│ │ ├── eventGates.test.js
|
||||||
|
│ │ ├── eventIntegrations.test.js
|
||||||
|
│ │ ├── eventLedger.test.js
|
||||||
|
│ │ ├── eventModuleContract.test.js
|
||||||
|
│ │ ├── eventParticipants.test.js
|
||||||
|
│ │ ├── eventPrice.test.js
|
||||||
|
│ │ ├── eventPublic.test.js
|
||||||
|
│ │ ├── eventRecurrence.test.js
|
||||||
|
│ │ ├── eventRunControls.test.js
|
||||||
|
│ │ ├── eventRunner.test.js
|
||||||
|
│ │ ├── eventRunnerSql.test.js
|
||||||
|
│ │ ├── eventsAdmin.test.js
|
||||||
|
│ │ ├── eventSchedule.test.js
|
||||||
|
│ │ ├── eventSeries.test.js
|
||||||
|
│ │ ├── eventSpec.test.js
|
||||||
|
│ │ ├── eventsRoles.test.js
|
||||||
|
│ │ ├── eventVerify.test.js
|
||||||
│ │ ├── honeypot.test.js
|
│ │ ├── honeypot.test.js
|
||||||
│ │ ├── htmlShell.test.js
|
│ │ ├── htmlShell.test.js
|
||||||
│ │ ├── inviteController.test.js
|
│ │ ├── inviteController.test.js
|
||||||
|
|||||||
@@ -213,28 +213,60 @@ implicitly commit, defeating the all-or-nothing guarantee. Point ids are assigne
|
|||||||
explicitly rather than left to `AUTO_INCREMENT`, because the join rows need them
|
explicitly rather than left to `AUTO_INCREMENT`, because the join rows need them
|
||||||
and `conn.batch()` reports no usable `insertId`.
|
and `conn.batch()` reports no usable `insertId`.
|
||||||
|
|
||||||
## Artwork — operator-supplied, never shipped
|
## Artwork — the shard extracts it now (Protocol 8)
|
||||||
|
|
||||||
**This project ships no creature art and no extraction tooling, and never will.**
|
**This project ships no creature art and no extraction tooling, and never will.**
|
||||||
UO sprites live in the operator's own client `.mul`/`.uop` files. They are the
|
UO sprites live in the operator's own client `.mul`/`.uop` files. They are the
|
||||||
operator's, not ours to redistribute.
|
operator's, not ours to redistribute.
|
||||||
|
|
||||||
The atlas is fully functional as text. `shard_spawn_creatures.art` is nullable
|
What changed in protocol 8 is not that rule — it is who does the extracting. The
|
||||||
and is NULL on every fresh import; pages render without images, which is the
|
shard already has those files (a ServUO server cannot boot without a UO client),
|
||||||
normal and supported state, not a degraded one.
|
so as of [`v8.md`](../link/v8.md) phase 3 it decodes them itself and hands the
|
||||||
|
pictures to the website over the bridge. Nobody installs UOFiddler and nobody
|
||||||
|
copies images to a web host.
|
||||||
|
|
||||||
An operator who wants art — step-by-step, with the UOFiddler side spelled out, in
|
**Admin → Shard → Import.** The import walks the shard's asset manifest, fetches
|
||||||
[`UOFIDDLER.md`](UOFIDDLER.md) §Part 2:
|
only the sprites whose hash changed, writes them under `uploads/atlas/`, asks the
|
||||||
|
shard for a body id per creature (§8 — the shard constructs the creature and
|
||||||
|
reads `Body.BodyID`, which is the only thing that is right for a shard's own
|
||||||
|
custom creatures) and points each `shard_spawn_creatures.art` at its picture.
|
||||||
|
Boot never calls the shard for this: the files change when an operator patches
|
||||||
|
their client, which is an event they know about and the site does not.
|
||||||
|
|
||||||
1. Extracts it from **their own** client files (UOFiddler, ClassicUO tooling, or
|
On this machine's stock client that is **1,022 creature portraits**, about a
|
||||||
any art extractor).
|
megabyte in total — 787 out of the legacy `anim*.mul` files and 235 more out of
|
||||||
2. Drops the images under `server/uploads/atlas/`.
|
`AnimationFrame*.uop`, which ServUO's own decoder never opens
|
||||||
3. Copies `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json`
|
([`../link/v8.md`](../link/v8.md) §4.9).
|
||||||
and maps creature slugs to file names.
|
|
||||||
4. Restarts, or runs `npm run atlas:import -- --force`.
|
|
||||||
|
|
||||||
Both `spawnAtlas.art.json` and `server/uploads/` are gitignored, so neither the
|
**NULL stays a first-class state, and always will be.** An install with no shard
|
||||||
map nor the images can be committed by accident.
|
link has never imported one; a Linux shard host without `libgdiplus` cannot
|
||||||
|
render a sprite at all (a named `NO_IMAGING` status, not an error); and about
|
||||||
|
half the addressable body range has no art in any client file. Pages render
|
||||||
|
without images, which is normal and supported, not degraded.
|
||||||
|
|
||||||
|
### The operator's own artwork still wins
|
||||||
|
|
||||||
|
An operator who has drawn their own portraits keeps them. The map is unchanged:
|
||||||
|
|
||||||
|
1. Drop the images under `server/uploads/atlas/`.
|
||||||
|
2. Copy `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json` and
|
||||||
|
map creature slugs to file names.
|
||||||
|
3. Restart, or run the import.
|
||||||
|
|
||||||
|
`spawnAtlas.art.json` is applied **over** anything imported, per slug, so a sprite
|
||||||
|
rip never replaces a hand-drawn portrait on the next Update. Both it and
|
||||||
|
`server/uploads/` are gitignored, so neither the map nor the images can be
|
||||||
|
committed by accident.
|
||||||
|
|
||||||
|
### Why the imported art is not stored on the creature row
|
||||||
|
|
||||||
|
`shard_spawn_creatures` is emptied and refilled by every atlas refresh, and a
|
||||||
|
refresh happens on every boot. So the body ids and the imported files live in
|
||||||
|
`shard_creature_bodies` and `shard_assets`, outside that blast radius, and the
|
||||||
|
atlas import re-derives `art` from them on the way past. Storing it on the row
|
||||||
|
would mean an ordinary re-parse of the ServUO tree silently deleting every
|
||||||
|
portrait — with the next asset Update finding the client files unchanged,
|
||||||
|
reporting "nothing to do", and never putting them back.
|
||||||
|
|
||||||
## Code layout
|
## Code layout
|
||||||
|
|
||||||
|
|||||||
@@ -1,282 +0,0 @@
|
|||||||
# Extracting from your own UO client (UOFiddler)
|
|
||||||
|
|
||||||
**Audience:** the shard operator, once, at setup time.
|
|
||||||
**Related:** [`CLILOCS.md`](CLILOCS.md) (why the cliloc conversion is unavoidable),
|
|
||||||
[`SPAWN_ATLAS.md`](SPAWN_ATLAS.md) (where creature art fits).
|
|
||||||
|
|
||||||
Two features read data that **only exists inside a UO client**, and a UO client's
|
|
||||||
files are EA's, not ours to redistribute. So neither this repo nor any image we
|
|
||||||
publish can ship them — the operator extracts from **their own** client, once,
|
|
||||||
and points the site at the result.
|
|
||||||
|
|
||||||
| Feature | What it needs | Required? | Without it |
|
|
||||||
|---|---|---|---|
|
|
||||||
| **Item / title names** ([`CLILOCS.md`](CLILOCS.md)) | `Cliloc.enu`, converted | No | Names render as raw ids — `id 1023721` instead of *quarter staff* |
|
|
||||||
| **Creature art** ([`SPAWN_ATLAS.md`](SPAWN_ATLAS.md)) | Sprites from `.mul`/`.uop` | No | Atlas pages render as text, which is the normal state |
|
|
||||||
|
|
||||||
**Both are optional and neither is load-bearing.** A shard that never does any of
|
|
||||||
this is fully supported. Do part one and skip part two if art is not worth your
|
|
||||||
time — they share only the tool.
|
|
||||||
|
|
||||||
Everything you extract stays **outside the repository**: the converted cliloc
|
|
||||||
file lives at a path you choose, and `spawnAtlas.art.json` plus `server/uploads/`
|
|
||||||
are gitignored, so none of it can be committed by accident.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Part 0 — Get UOFiddler
|
|
||||||
|
|
||||||
[UOFiddler](https://github.com/polserver/UOFiddler) is the community client-file
|
|
||||||
editor. We use it because its `Ultima.dll` already contains the cliloc
|
|
||||||
decompressor, maintained by people who do this for a living.
|
|
||||||
|
|
||||||
1. Download the latest release zip from
|
|
||||||
<https://github.com/polserver/UOFiddler/releases/latest> — one asset, named
|
|
||||||
`UOFiddler-<version>.zip` (4.22.2 is ~2 MB).
|
|
||||||
2. Extract it. The zip contains a single top-level folder, and the two files that
|
|
||||||
matter are at **its root**:
|
|
||||||
|
|
||||||
```
|
|
||||||
UOFiddler-4.22.2/
|
|
||||||
Ultima.dll ← the decompressor (Part 1 needs this path)
|
|
||||||
UoFiddler.exe ← the GUI (Part 2 needs this)
|
|
||||||
plugins/
|
|
||||||
…
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Runtime:** UOFiddler 4.22.2 is built for **.NET 10**. Running `UoFiddler.exe`
|
|
||||||
needs the .NET 10 **Desktop** Runtime (Windows only); loading `Ultima.dll` from
|
|
||||||
the converter in Part 1 needs the .NET 10 runtime. Install from
|
|
||||||
<https://dotnet.microsoft.com/download/dotnet/10.0>.
|
|
||||||
|
|
||||||
### Finding your client files
|
|
||||||
|
|
||||||
The cliloc file is in your **UO client installation directory**, not in your
|
|
||||||
ServUO tree — the shard server has no copy of it. Look for `Cliloc.enu` (English;
|
|
||||||
the other seven are `chs`, `cht`, `deu`, `esp`, `fra`, `jpn`, `kor`) beside
|
|
||||||
`art.mul` / `artLegacyMUL.uop`. The EA Classic Client's default location is:
|
|
||||||
|
|
||||||
```
|
|
||||||
C:\Program Files (x86)\Electronic Arts\Ultima Online Classic\
|
|
||||||
```
|
|
||||||
|
|
||||||
**If your shard distributes its own patched client to players, use that copy.**
|
|
||||||
Any cliloc edits you shipped to players are then already in the base table and
|
|
||||||
you need no overlay for them (see [`CLILOCS.md`](CLILOCS.md) §Shard-added and
|
|
||||||
shard-edited items).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Part 1 — Convert the cliloc table
|
|
||||||
|
|
||||||
**Goal:** turn the client's compressed `Cliloc.enu` into a file the site can
|
|
||||||
read, and point the site at it.
|
|
||||||
|
|
||||||
The site cannot read `Cliloc.enu` directly. Every modern client compresses it
|
|
||||||
(the "Mythic" container), and so does ServUO's own bundled `Ultima.StringList` —
|
|
||||||
which is why the shard cannot supply names on our behalf either. The full
|
|
||||||
reasoning is in [`CLILOCS.md`](CLILOCS.md) §Why the operator has to convert the
|
|
||||||
file; this section is just the procedure.
|
|
||||||
|
|
||||||
Two routes. **The bundled tool is the recommended one** — the GUI export needs a
|
|
||||||
fixup step, described below.
|
|
||||||
|
|
||||||
### Route A — the bundled converter (recommended)
|
|
||||||
|
|
||||||
Needs a .NET SDK (any version 8 or newer — the project targets `net8.0` and rolls
|
|
||||||
forward, so whatever you have works) **plus** the .NET 10 runtime from Part 0,
|
|
||||||
which is what actually loads `Ultima.dll`.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd website/server/tools/cliloc-export
|
|
||||||
dotnet build -c Release
|
|
||||||
|
|
||||||
# plain binary — recommended, exact
|
|
||||||
dotnet run -c Release -- \
|
|
||||||
"/path/to/UOFiddler-4.22.2/Ultima.dll" \
|
|
||||||
"/path/to/UO client/Cliloc.enu" \
|
|
||||||
/srv/uo-data/clilocs.plain
|
|
||||||
|
|
||||||
# or tab-delimited text, if you want to eyeball or hand-edit it
|
|
||||||
dotnet run -c Release -- \
|
|
||||||
"/path/to/UOFiddler-4.22.2/Ultima.dll" \
|
|
||||||
"/path/to/UO client/Cliloc.enu" \
|
|
||||||
/srv/uo-data/clilocs.tsv --tsv
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected output for a stock English client:
|
|
||||||
|
|
||||||
```
|
|
||||||
wrote 123490 entries to /srv/uo-data/clilocs.plain (maxTextBytes=12150, skippedOversize=0)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Sanity-check that number.** A stock `Cliloc.enu` is ~123,000 entries. A few
|
|
||||||
hundred means it read something else and you should not ship the result. The
|
|
||||||
tool exits non-zero and says `no entries were written — is that a cliloc file?`
|
|
||||||
when it gets nothing at all.
|
|
||||||
|
|
||||||
The conversion runs on whatever machine has the client (usually Windows), and the
|
|
||||||
site reads the output wherever it runs — so **copy the output file to the server**
|
|
||||||
if those are different machines. It is a single self-contained file (~5 MB); the
|
|
||||||
`--tsv` form is larger but diff-able.
|
|
||||||
|
|
||||||
<details>
|
|
||||||
<summary>Errors you may hit</summary>
|
|
||||||
|
|
||||||
| Message | Cause |
|
|
||||||
|---|---|
|
|
||||||
| `Ultima.StringList not found — is that really UOFiddler's Ultima.dll?` | First argument points at some other `Ultima.dll` (ServUO ships one too — it is **not** the same assembly and cannot do this) |
|
|
||||||
| `You must install .NET to run this application` | Missing the .NET 10 runtime from Part 0 step 3 |
|
|
||||||
| `Unexpected Ultima.StringList API` | UOFiddler older than 4.21 |
|
|
||||||
| `usage: clilocexport …` | Fewer than three arguments |
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
||||||
### Route B — the UOFiddler GUI
|
|
||||||
|
|
||||||
Use this if you would rather not install a .NET SDK. **It needs one extra step**,
|
|
||||||
so do not skip the fixup.
|
|
||||||
|
|
||||||
1. Launch `UoFiddler.exe` and point it at your client directory when it asks
|
|
||||||
(or **Options → Path Settings**).
|
|
||||||
2. Open the **Cliloc** tab and use its **export to CSV** action.
|
|
||||||
3. It writes `CliLoc.csv` to UOFiddler's configured output path, in **three**
|
|
||||||
columns with a header row:
|
|
||||||
|
|
||||||
```
|
|
||||||
Number;Text;Flag
|
|
||||||
1023721;quarter staff;0
|
|
||||||
```
|
|
||||||
|
|
||||||
4. **Strip the trailing flag column.** The site's text parser reads
|
|
||||||
`number<TAB|,|;>text`, so that third field is otherwise absorbed into the name
|
|
||||||
and every item on the site renders as `quarter staff;0`.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sed -E 's/;[0-9]+$//' CliLoc.csv > clilocs.csv
|
|
||||||
```
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
Get-Content CliLoc.csv |
|
|
||||||
ForEach-Object { $_ -replace ';\d+$','' } |
|
|
||||||
Set-Content -Encoding utf8 clilocs.csv
|
|
||||||
```
|
|
||||||
|
|
||||||
The header row needs no removal — a line whose first field is not an integer
|
|
||||||
is skipped. Blank entries (`1005008;`) survive the fixup correctly and are
|
|
||||||
dropped at import, as intended.
|
|
||||||
|
|
||||||
5. Copy `clilocs.csv` to the server.
|
|
||||||
|
|
||||||
**Why the fixup is not just done for us:** the parser already handles
|
|
||||||
`number,flag,text` — the flag in the *middle*, which is what several exports
|
|
||||||
emit. UOFiddler puts it at the *end*, where it is indistinguishable from a name
|
|
||||||
that genuinely ends in `;0`. One `sed` on the operator's side beats a parser
|
|
||||||
heuristic that would corrupt real names.
|
|
||||||
|
|
||||||
### Point the site at it
|
|
||||||
|
|
||||||
Two ways, the setting winning over the environment:
|
|
||||||
|
|
||||||
| Where | How |
|
|
||||||
|---|---|
|
|
||||||
| **Admin → Shard → cliloc path** | Takes effect on the next refresh, no redeploy |
|
|
||||||
| `UO_CLIENT_PATH` env var | The deploy-time default |
|
|
||||||
|
|
||||||
The value may be **the file itself or a directory to search** — both are natural
|
|
||||||
answers to "where is it", and overlays are picked up either way.
|
|
||||||
|
|
||||||
Setting the path deliberately does **not** import as a side effect. Click
|
|
||||||
**Import** (or `POST /api/v1/admin/shard/clilocs/import`) to load it.
|
|
||||||
|
|
||||||
### Verify
|
|
||||||
|
|
||||||
`GET /api/v1/admin/shard/clilocs`, or the Admin → Shard panel, reports what each
|
|
||||||
source contributed:
|
|
||||||
|
|
||||||
```json
|
|
||||||
"sources": [
|
|
||||||
{ "label": "clilocs.plain", "kind": "base", "entries": 123490, "added": 123490, "overrode": 0 }
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
Roughly **67,500 rows stored** from a stock table is correct — about half a
|
|
||||||
cliloc table is empty strings for ids the client reserves and never uses.
|
|
||||||
|
|
||||||
Then load any character sheet with equipment: items should show names rather than
|
|
||||||
`id 1023721`.
|
|
||||||
|
|
||||||
<details>
|
|
||||||
<summary>What a refusal means</summary>
|
|
||||||
|
|
||||||
A bad file answers `200` with a `status` and a named reason, not a `500` — you
|
|
||||||
need to be told *which file* to fix.
|
|
||||||
|
|
||||||
| `code` | Meaning |
|
|
||||||
|---|---|
|
|
||||||
| `COMPRESSED` | You pointed at the raw client `Cliloc.enu`. Convert it — this whole page. |
|
|
||||||
| `TRUNCATED` | Half-copied file. Re-copy; the loaded table is untouched. |
|
|
||||||
| `EMPTY` | A text source with no parseable rows — the file is named in the reason. |
|
|
||||||
| `status: needsReview` + `missingSources` | A previously-loaded source has vanished (unmounted volume? deliberate deletion?). Nothing changes until you re-import with `{ "approve": true }`. |
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
||||||
### Custom items — do *not* re-export for these
|
|
||||||
|
|
||||||
Shard-added items carry ids no client table has. Drop a small delimited file in a
|
|
||||||
`custom/` directory beside the base file and re-import:
|
|
||||||
|
|
||||||
```
|
|
||||||
/srv/uo-data/
|
|
||||||
clilocs.plain ← base, from this guide
|
|
||||||
custom/
|
|
||||||
01-uomysticmoon.tsv ← your additions and overrides
|
|
||||||
```
|
|
||||||
|
|
||||||
Files are read in sorted order and **later sources win**, so an overlay both adds
|
|
||||||
new ids and overrides stock ones you have re-purposed. **Adding one item never
|
|
||||||
means re-exporting a 5 MB client file.** Details in [`CLILOCS.md`](CLILOCS.md).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Part 2 — Creature art for the spawn atlas (optional)
|
|
||||||
|
|
||||||
**Goal:** put sprites on atlas pages. Purely cosmetic — the atlas is fully
|
|
||||||
functional as text, and `art` is NULL on every fresh import.
|
|
||||||
|
|
||||||
**This project ships no art and no art-extraction tooling, and never will.**
|
|
||||||
|
|
||||||
1. In `UoFiddler.exe` (paths configured as in Route B step 1), open the
|
|
||||||
**Animations** tab for creature sprites — or **Items** for object art — find
|
|
||||||
the creature, and export as PNG. Right-click an entry for its export options,
|
|
||||||
or use the tab's *Export All* action for a batch. (4.22.2 added an export
|
|
||||||
option to the Animation tab's thumbnail list, which is the convenient one
|
|
||||||
here.)
|
|
||||||
2. Put the images under `server/uploads/atlas/`.
|
|
||||||
3. Copy `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json` in
|
|
||||||
the same directory and map creature slugs to file names:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"lizardman": "lizardman.png",
|
|
||||||
"orc": "orc.png"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Keys are the slugs the atlas API reports**, derived from the type names in
|
|
||||||
your own shard's `Spawns/*.xml` — read them off the atlas rather than guessing.
|
|
||||||
A creature with no entry renders without art, which is the default.
|
|
||||||
|
|
||||||
4. Restart, or `npm run atlas:import -- --force`.
|
|
||||||
|
|
||||||
The art map is re-read on every atlas refresh, so adding one image is an edit plus
|
|
||||||
a refresh. Both `spawnAtlas.art.json` and `server/uploads/` are gitignored.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Licensing, briefly
|
|
||||||
|
|
||||||
UO's strings and sprites are EA's. Extracting from **your own** client for
|
|
||||||
**your own** shard is the arrangement here; redistributing the extracted files is
|
|
||||||
not something this project does or can advise on. That is the whole reason this
|
|
||||||
page exists instead of a download link.
|
|
||||||
Reference in New Issue
Block a user