docs(link): clilocs come over the bridge now, and UOFiddler's first job is gone
Phase 2 of the Asset Bridge is built, so the documentation stops telling an operator to install a GUI tool. `v8.md` gains §9.1 and §9.2 — what the port cost, what it measured, and where the base table comes from now. The measurement worth keeping: **67,496 rows in 290 ms**, which is exactly what UOFiddler's own `Ultima.dll` produced from this same client through the converter this phase deletes. An independent implementation agreeing to the row is the strongest check available that a format decoder is correct, and it is not something a subtly-wrong one produces. §17 records the four shapes the org lead settled before any of it was written. Two departed from the recommendation: **the bridge always wins** (no source setting — there is no version of that question an operator benefits from answering) and **import is admin-triggered** (boot does not call the shard at all). `CLILOCS.md` is rewritten around that: where the table comes from, what arrives and in how many pieces, the refusals — including the two the file pipeline had no equivalent of (a client patched mid-import, and the base's exemption from the vanished-source rule, which exists so an upgraded install is not asked to approve a change the upgrade itself made). `UOFIDDLER.md` loses Part 1 entirely rather than having it rewritten. What is left is creature art, which phase 5 takes, after which the page goes away. `v3.md` §8.6 keeps its reasoning with a note saying what superseded it, because the argument for why the manual step existed is still the argument for why this was worth building. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
@@ -1,7 +1,8 @@
|
||||
# Cliloc table (item and title names)
|
||||
|
||||
**Status:** Complete on `edge` — website [#115](https://gitea.whitlocktech.com/RunicGateway/website/pulls/115), docs [#70](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/70).
|
||||
**Design:** [`docs/link/v3.md` §8.6](../link/v3.md) — Protocol 3.0, the dependency Part B/3 was sequenced behind.
|
||||
**Status:** Complete. The base table now arrives **over the bridge** — protocol 8, phase 2.
|
||||
**Design:** [`docs/link/v3.md` §8.6](../link/v3.md) (the table itself, Protocol 3.0) and
|
||||
[`docs/link/v8.md` §9](../link/v8.md) (the Asset Bridge, which retired the manual conversion).
|
||||
|
||||
A "cliloc" is UO's localization table: an integer id mapped to a display string.
|
||||
**Items on the wire carry a `LabelNumber`, not a name.** The bridge has always
|
||||
@@ -12,86 +13,70 @@ render `id 1023721` where the game renders **"quarter staff"**.
|
||||
|
||||
The number was never the missing piece. The table was.
|
||||
|
||||
## Why the operator has to convert the file
|
||||
## Where the table comes from
|
||||
|
||||
This is the awkward part, and it is not avoidable:
|
||||
**The shard reads its own client.** A ServUO server cannot boot without a UO
|
||||
client — `Config/DataPath.cfg` resolves into `Core.DataDirectories` at run time —
|
||||
so the file this table is made of is already sitting on the shard host. Since
|
||||
protocol 8 the plugin decompresses it and serves it over the bridge, and the site
|
||||
imports it like any other shard read:
|
||||
|
||||
**Every current UO client ships its cliloc files compressed.** All eight
|
||||
`Cliloc.*` files in a modern client (`chs`, `cht`, `deu`, `enu`, `esp`, `fra`,
|
||||
`jpn`, `kor`) begin with a DWORD whose high byte is `0x8E` — the "Mythic"
|
||||
compressed container. The plain layout this site parses is what those files
|
||||
looked like *before* that change.
|
||||
|
||||
Decompressing it means an inverse-BWT coder with a frequency header — a few
|
||||
hundred lines of bit-level work whose failure mode is plausible-looking garbage
|
||||
rather than an error. The site has no business carrying that at runtime.
|
||||
|
||||
Two facts make the alternatives worse, not better:
|
||||
|
||||
- **ServUO cannot read it either.** Its bundled `Ultima.StringList` implements
|
||||
only the plain layout, so on a modern client `VendorSearch.StringList` is null
|
||||
and `VendorSearch.GetItemName` returns `item.Name` — usually nothing. The
|
||||
shard cannot supply names on our behalf; the in-game Vendor Search gump has the
|
||||
same gap.
|
||||
- **Nothing client-derived may be committed.** UO's strings are EA's. The repo
|
||||
ships no string table for the same reason it ships no artwork and no map
|
||||
snapshot — see [`SPAWN_ATLAS.md`](SPAWN_ATLAS.md).
|
||||
|
||||
So the conversion happens **once, on the operator's machine, against their own
|
||||
client**, and the site reads the result from a path it is given. A shard that
|
||||
never does this is in a fully supported state: names render as ids, exactly as
|
||||
they did before the table existed.
|
||||
|
||||
## Converting
|
||||
|
||||
> **Step-by-step operator instructions — where to get UOFiddler, where your
|
||||
> client files are, and how to verify the import — are in
|
||||
> [`UOFIDDLER.md`](UOFIDDLER.md).** This section covers the formats and the
|
||||
> reasoning behind them.
|
||||
|
||||
Either format below is accepted; the site sniffs which one it was handed.
|
||||
|
||||
| Format | Fidelity | Notes |
|
||||
|---|---|---|
|
||||
| **Plain binary** (recommended) | Exact | 6-byte header, then `{int32 number, byte flag, uint16 length, UTF-8}` records |
|
||||
| Delimited text | Loses leading/trailing whitespace | `number<TAB\|,\|;>text` per line; a header row, blank lines and `#` comments are ignored |
|
||||
|
||||
The whitespace caveat is real but cosmetic: ~1,300 of the 123,490 entries in a
|
||||
stock `Cliloc.enu` are label prefixes like `"max = "` whose trailing space is
|
||||
meaningful when the client concatenates a value onto them. Nothing on this site
|
||||
concatenates, and every consumer passes through `displayText()`, which trims.
|
||||
|
||||
### Using the bundled tool
|
||||
|
||||
`server/tools/cliloc-export/` is a small .NET console app that drives
|
||||
[UOFiddler](https://github.com/polserver/UOFiddler)'s `Ultima.dll` — the
|
||||
decompressor that already exists and is already maintained — and writes the plain
|
||||
format. It loads that DLL **reflectively** so it compiles against any SDK, and it
|
||||
writes the records by hand because UOFiddler's own `SaveStringList` *re-compresses*
|
||||
on save (its purpose is round-tripping a file back into the client, so its output
|
||||
is byte-identical to its input — a trap worth knowing about).
|
||||
|
||||
```bash
|
||||
cd website/server/tools/cliloc-export
|
||||
dotnet build -c Release
|
||||
|
||||
# binary (recommended)
|
||||
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.plain
|
||||
|
||||
# or tab-delimited
|
||||
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv
|
||||
```
|
||||
ServUO shard ──`cliloc.table`──▶ uo-link sidecar ──`GET /cliloc`──▶ website
|
||||
reads Cliloc.enu, forwards, keeps merges overlays,
|
||||
decompresses, pages nothing replaces the table
|
||||
```
|
||||
|
||||
A UOFiddler GUI export works too, but **not unmodified**: its Cliloc tab writes
|
||||
`Number;Text;Flag` — three columns, the flag *last* — and the parser reads
|
||||
`number<separator>text`, so the trailing field is absorbed into the name and
|
||||
every item renders as `quarter staff;0`. Stripping it is one `sed`, given in
|
||||
[`UOFIDDLER.md`](UOFIDDLER.md) §Route B.
|
||||
**Why that was worth building.** Every current UO client ships its cliloc files
|
||||
compressed: all eight `Cliloc.*` files in a modern client (`chs`, `cht`, `deu`,
|
||||
`enu`, `esp`, `fra`, `jpn`, `kor`) begin with a DWORD whose high byte is `0x8E` —
|
||||
the "Mythic" container. The plain layout is what those files looked like *before*
|
||||
that change, and **ServUO's own bundled `Ultima.StringList` cannot read the new
|
||||
one either**, which is why `VendorSearch.GetItemName` is inert on a modern shard
|
||||
and the in-game Vendor Search gump has the same gap.
|
||||
|
||||
The parser already tolerates `number,flag,text`, with the flag in the *middle*.
|
||||
It is not extended to cover the trailing form because a final `;0` is
|
||||
indistinguishable from a name that genuinely ends that way — a heuristic there
|
||||
would corrupt real names to save the operator one command.
|
||||
So until protocol 8 an operator had to install UOFiddler, build a converter
|
||||
against its `Ultima.dll`, run it over their client and copy a 5 MB file to the web
|
||||
host — every time they patched. The decompressor now lives in the overlay
|
||||
(`overlay/Scripts/Custom/Bridge/BridgeCliloc.cs`, ported from UOFiddler, which is
|
||||
Beerware and therefore clean to bring into a GPL tree), so **none of that is a
|
||||
step any more**.
|
||||
|
||||
Two things are unchanged and remain the point:
|
||||
|
||||
- **Nothing client-derived is committed.** UO's strings are EA's. The repo ships
|
||||
no string table for the same reason it ships no artwork and no map snapshot —
|
||||
see [`SPAWN_ATLAS.md`](SPAWN_ATLAS.md). The extraction happens on the
|
||||
operator's own host, from their own files, for their own shard.
|
||||
- **A shard with no table is fully supported.** Names render as ids, exactly as
|
||||
they did before the table existed.
|
||||
|
||||
### What arrives, and in how many pieces
|
||||
|
||||
The sidecar's reply timeout is 10 s and its inbound line cap is 1 MiB, so the
|
||||
table is **paged**: the shard cuts at a 512 KiB byte budget and hands back a
|
||||
cursor, and the site walks it until a page says `more: false`. Measured on a stock
|
||||
English client: **67,496 rows in about eleven pages**, decoded on the shard in
|
||||
**290 ms**.
|
||||
|
||||
The rows are `{ n, f, t }` — number, flag, text. **Blanks never leave the shard**:
|
||||
roughly 56,000 of a stock table's 123,490 entries are empty strings the client
|
||||
reserves and never uses, the site drops them at import anyway, and sending them
|
||||
would double the transfer for data that is discarded on arrival.
|
||||
|
||||
Only `cut: "end"` means the table finished. A short page can equally mean the byte
|
||||
budget was spent, and importing a table that stopped early is the one failure that
|
||||
is invisible downstream — some items named, some not, which is exactly what *no
|
||||
table* looks like.
|
||||
|
||||
### The file pipeline is deprecated, not removed
|
||||
|
||||
An install with **no uo-link configured** can still be pointed at a converted file
|
||||
and works exactly as it did. That path exists for shards with no bridge and for
|
||||
development without a running ServUO, it accepts the same two formats it always
|
||||
did (plain binary, or `number<TAB|,|;>text` delimited text), and passing an
|
||||
explicit path to a refresh still selects it as a one-off. Nothing new should be
|
||||
built on it.
|
||||
|
||||
## Shard-added and shard-edited items
|
||||
|
||||
@@ -103,12 +88,16 @@ atlas, which reads `Regions.xml` + `Locations/*.xml` + `Spawns/*.xml` +
|
||||
|
||||
```
|
||||
<cliloc path>/
|
||||
clilocs.plain ← base: the converted client table
|
||||
custom/
|
||||
01-uomysticmoon.tsv ← overlays: shard additions and overrides
|
||||
02-events.tsv
|
||||
```
|
||||
|
||||
The base is no longer a file in that directory — it comes from the shard — so on
|
||||
a bridged install the configured path selects **only** where `custom/` is read
|
||||
from. (Without a shard link, a converted base file sitting beside `custom/` is
|
||||
still found, which is the deprecated pipeline above.)
|
||||
|
||||
Overlays use the same delimited-text format, are read in **sorted order**, and
|
||||
**later sources win** — so an overlay both *adds* ids the client never had and
|
||||
*overrides* stock ones the shard has re-purposed. Any `.tsv`, `.csv`, `.txt`,
|
||||
@@ -116,8 +105,9 @@ Overlays use the same delimited-text format, are read in **sorted order**, and
|
||||
say) is ignored.
|
||||
|
||||
Adding, editing or removing any overlay counts as drift, so a new custom item
|
||||
needs only a file edit and a restart — or the admin panel's Import button.
|
||||
**Adding one item never means re-exporting a 5 MB client file.**
|
||||
needs only a file edit and the admin panel's Import button. **Adding one item
|
||||
never means re-reading the client table** — though on the bridge that is now
|
||||
cheap enough not to matter much.
|
||||
|
||||
The import result reports what each source contributed, which is how you confirm
|
||||
an overlay took effect — `overrode: 0` on a file meant to re-label stock items
|
||||
@@ -125,11 +115,14 @@ says it did not:
|
||||
|
||||
```json
|
||||
"sources": [
|
||||
{ "label": "clilocs.plain", "kind": "base", "entries": 123490, "added": 123490, "overrode": 0 },
|
||||
{ "label": "custom/uomysticmoon.tsv", "kind": "custom", "entries": 2, "added": 1, "overrode": 1 }
|
||||
{ "label": "cliloc.enu", "kind": "shard", "entries": 67496, "added": 67496, "overrode": 0 },
|
||||
{ "label": "custom/uomysticmoon.tsv", "kind": "custom", "entries": 2, "added": 1, "overrode": 1 }
|
||||
]
|
||||
```
|
||||
|
||||
`kind` says where a source came from: `shard` over the bridge, `base` a converted
|
||||
file on disk, `custom` an overlay.
|
||||
|
||||
**Why a convention rather than discovery.** Everywhere else this pipeline follows
|
||||
the shard's own files, but **ServUO has no server-side notion of a custom
|
||||
cliloc** — they live in the patched client a shard distributes to its players,
|
||||
@@ -143,28 +136,29 @@ ids and only **37** are absent from the stock client table — tens of entries
|
||||
against a 67k base, which is what makes an overlay the right shape rather than a
|
||||
second full table.
|
||||
|
||||
## Configuring the path
|
||||
## Choosing the source
|
||||
|
||||
Two ways to point at the sources, the setting winning over the environment:
|
||||
Nothing to configure: **the shard wins whenever uo-link is configured and
|
||||
enabled.** There is no mode setting, because there is no version of this question
|
||||
an operator benefits from answering — a shard that can serve its own client table
|
||||
is strictly better than a file somebody converted by hand months ago.
|
||||
|
||||
| Source | Notes |
|
||||
The two escape hatches, both deliberate:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `cliloc_client_path` setting | Admin-editable (Admin → Shard); takes effect on the next refresh without a redeploy |
|
||||
| `UO_CLIENT_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |
|
||||
| No uo-link configured | The file pipeline, exactly as before |
|
||||
| An explicit `path` passed to a refresh | A one-off "import from this file", which the shard never overrules |
|
||||
|
||||
The value may be **the base file itself or a directory to search**, because both
|
||||
are natural answers to "where is it". Overlays are read from a `custom/`
|
||||
directory beside the base **either way** — pointing at a file does not forfeit
|
||||
them.
|
||||
|
||||
A directory is searched case-insensitively (the client writes `Cliloc.enu` on
|
||||
Windows; the site usually runs on Linux) for, in order: `clilocs.tsv`,
|
||||
`clilocs.csv`, `clilocs.plain`, `cliloc.plain`, `cliloc.plain.enu`,
|
||||
`cliloc.enu.plain`, `clilocs.txt`, `cliloc.enu`.
|
||||
|
||||
That ordering puts explicitly-converted names first on purpose. Pointing the
|
||||
setting straight at an unconverted client directory finds `cliloc.enu`, which is
|
||||
compressed — and the site says so by name rather than failing obscurely:
|
||||
`cliloc_client_path` (setting, admin-editable) and `UO_CLIENT_PATH` (env
|
||||
deploy-time default) still name a path, and the setting still wins over the
|
||||
environment. What that path *means* narrowed: on a bridged install it is where
|
||||
`custom/` overlays live. Without a link it is also searched for a base file,
|
||||
case-insensitively (the client writes `Cliloc.enu` on Windows; the site usually
|
||||
runs on Linux), in order: `clilocs.tsv`, `clilocs.csv`, `clilocs.plain`,
|
||||
`cliloc.plain`, `cliloc.plain.enu`, `cliloc.enu.plain`, `clilocs.txt`,
|
||||
`cliloc.enu` — and pointing it straight at an unconverted client directory still
|
||||
answers by name rather than failing obscurely:
|
||||
|
||||
```
|
||||
status: unavailable
|
||||
@@ -175,16 +169,25 @@ reason: This is a compressed (Mythic-format) cliloc file, which the site cannot
|
||||
|
||||
## Refresh contract
|
||||
|
||||
Identical in shape to the spawn atlas, and for the same reasons:
|
||||
|
||||
- **It never blocks startup.** No path, an unreadable file, a wrong-format file,
|
||||
a database error — all caught and logged. The site comes up either way.
|
||||
- **Hash-gated.** The boot path hashes the file and skips the parse entirely when
|
||||
it matches what is loaded, which is every restart that did not follow a client
|
||||
patch. Measured on a stock table: **14 ms** for the no-op, **663 ms** for a full
|
||||
parse and replace.
|
||||
- **A `PARSER_VERSION` bump also counts as drift**, so a corrected parse reaches
|
||||
an install whose client never patches.
|
||||
- **It never blocks startup**, and on the bridge it never *touches* startup: boot
|
||||
imports nothing when the shard is the source. A sidecar round trip in the boot
|
||||
sequence would be spent answering "no" on every restart but the one after a
|
||||
client patch — and patching a client is an operator action, so importing is an
|
||||
operator action. **Admin → Shard → Import** is the button. Whatever table is
|
||||
loaded keeps serving until then.
|
||||
- **Still hash-gated**, so pressing Import when nothing changed costs one small
|
||||
call. The gate is the shard's own `assets.sources`: size, mtime and content
|
||||
hash of `Cliloc.enu` plus the shard's `EXTRACTOR_VERSION`. A `sha256` of `null`
|
||||
with `hashing: true` means the shard has not computed it yet (it hashes off the
|
||||
request path, because the art and animation files it also reports are 343 MB)
|
||||
— that means *ask again*, never *changed*, and the comparison falls back to
|
||||
(size, mtime) meanwhile.
|
||||
- **Without a link** the file path is unchanged: boot hashes the local file and
|
||||
skips the parse when it matches. Measured on a stock table: **14 ms** for the
|
||||
no-op, **663 ms** for a full parse and replace.
|
||||
- **A `PARSER_VERSION` bump counts as drift**, and so does an `EXTRACTOR_VERSION`
|
||||
bump on the shard — same argument at the other end of the wire: a corrected
|
||||
reader must reach an install whose client never patches.
|
||||
|
||||
### Two ways a refresh is refused
|
||||
|
||||
@@ -201,7 +204,7 @@ and the rows already loaded are untouched. A malformed overlay names the file it
|
||||
came from (`custom/broken.tsv: No cliloc entries found…`), because "which of my
|
||||
six overlay files is broken" is otherwise a guessing game.
|
||||
|
||||
**A source that has VANISHED** is the hazard a single file did not have. It
|
||||
**An OVERLAY that has VANISHED** is the hazard a single file did not have. It
|
||||
parses perfectly and imports a table quietly missing everything that file
|
||||
contributed — and an unmounted volume looks exactly like a deliberate deletion
|
||||
from here. This is the same ambiguity the atlas stages a facet removal for, so it
|
||||
@@ -209,11 +212,28 @@ is escalated rather than applied:
|
||||
|
||||
```
|
||||
status: needsReview
|
||||
reason: 1 previously-loaded cliloc source(s) are missing;
|
||||
reason: 1 previously-loaded cliloc overlay(s) are missing;
|
||||
the existing table is unchanged
|
||||
missingSources: ["custom/uomysticmoon.tsv"]
|
||||
```
|
||||
|
||||
**The base is deliberately exempt from that question**, and that is an upgrade
|
||||
detail worth stating: an install that used the converted-file pipeline carries its
|
||||
base file's label in the stored fingerprint, and on the bridge that label is
|
||||
*supposed* to disappear. Counting it as a vanished source would make the first
|
||||
import after the upgrade demand an approval for a change the upgrade itself made.
|
||||
|
||||
**A client patched mid-import** is refused outright rather than staged, because
|
||||
there is nothing to decide: every page echoes the source file's size and mtime, and
|
||||
if they move between pages then half of what arrived came from a file that no
|
||||
longer exists and nothing later can tell which half.
|
||||
|
||||
```
|
||||
status: unavailable
|
||||
code: SOURCE_CHANGED
|
||||
reason: The shard's cliloc file changed while it was being read; nothing was imported
|
||||
```
|
||||
|
||||
`status()` reports `missingSources` too, so the panel can show it before anyone
|
||||
clicks Import. An admin accepts it by re-running the import with
|
||||
`{ "approve": true }`.
|
||||
@@ -229,15 +249,20 @@ admin was already going to run.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Parsed from a stock `Cliloc.enu` | **123,490** entries |
|
||||
| Entries in a stock `Cliloc.enu` | **123,490** |
|
||||
| Of those, empty strings | **55,994** (ids the client reserves and never uses) |
|
||||
| Stored in `shard_clilocs` | **67,496** |
|
||||
|
||||
Blank entries are dropped at import. A row resolving to no name is
|
||||
indistinguishable from no row at all to every caller, and dropping them makes the
|
||||
binary and text imports converge on **identical** content — the binary format
|
||||
carries the blanks explicitly and a text export may or may not, depending on the
|
||||
tool. Verified: both formats import to the same 67,496 rows with the same keys.
|
||||
Blank entries are dropped, and since protocol 8 they are dropped **on the shard**,
|
||||
before they reach the wire. A row resolving to no name is indistinguishable from
|
||||
no row at all to every caller, so sending 56,000 of them would double the
|
||||
transfer for data discarded on arrival. The site still filters at import, because
|
||||
an overlay file can carry one and because the file pipeline still exists.
|
||||
|
||||
That the shard's own decoder lands on **exactly 67,496** is also the strongest
|
||||
check there is that the ported decompressor is correct: the number was measured
|
||||
first through UOFiddler's `Ultima.dll` against this same client, by an entirely
|
||||
different implementation.
|
||||
|
||||
`text` is `TEXT`, not `VARCHAR`: the long property descriptions reach 12 KB, and
|
||||
silently truncating them would be worse than storing them. The index that matters
|
||||
@@ -297,13 +322,24 @@ All admin-only, alongside the atlas under Admin → Shard:
|
||||
|
||||
| Route | Purpose |
|
||||
|---|---|
|
||||
| `GET /api/v1/admin/shard/clilocs` | Sources found, what each contributed at the last import, readability, drift, entry count, `missingSources` |
|
||||
| `POST /api/v1/admin/shard/clilocs/import` | Reload after a client patch or an overlay edit; `{ "force": true }` reimports an unchanged set, `{ "approve": true }` accepts a vanished source |
|
||||
| `PUT /api/v1/admin/shard/clilocs/path` | Set the path; blank disables resolution |
|
||||
| `GET /api/v1/admin/shard/clilocs` | Which source is in use (`bridge` / `file`), the shard's client-file fingerprint, what each source contributed at the last import, drift, entry count, `missingSources` |
|
||||
| `POST /api/v1/admin/shard/clilocs/import` | Reload after a client patch or an overlay edit; `{ "force": true }` reimports an unchanged set, `{ "approve": true }` accepts a vanished overlay |
|
||||
| `PUT /api/v1/admin/shard/clilocs/path` | Set the overlay path (and, with no shard link, the base file's); blank clears it |
|
||||
|
||||
A refresh **result is not an exception**: a missing file, or the likely mistake of
|
||||
pointing at the client's own compressed `Cliloc.enu`, answers `200` with
|
||||
`status: "unavailable"` and a reason. A `500` would say only "something broke";
|
||||
the operator needs to be told which file to convert. Setting the path
|
||||
deliberately does **not** import as a side effect — the response carries the
|
||||
refreshed status so the panel can offer that as the next step.
|
||||
**Import is now the only thing that refreshes the table on a bridged install**,
|
||||
since boot no longer asks the shard. It is the button an operator presses after
|
||||
patching their client.
|
||||
|
||||
A refresh **result is not an exception**, and protocol 8 widened the set of things
|
||||
that covers: a shard that is down, an asset plane the operator has switched off
|
||||
(`Bridge.AssetsEnabled`), a client with no cliloc file, a client patched halfway
|
||||
through the import, plus everything the file pipeline could already report. Each
|
||||
answers `200` with `status: "unavailable"` and a reason naming what to fix. A
|
||||
`500` would say only "something broke". Setting the path deliberately does **not**
|
||||
import as a side effect — the response carries the refreshed status so the panel
|
||||
can offer that as the next step.
|
||||
|
||||
The sidecar's own statuses are worth knowing when reading a log: **425** is the
|
||||
shard saying it is busy with another asset request (flow control, and the ordinary
|
||||
answer mid-import — the site retries), **403** the asset plane switched off, **404**
|
||||
a client with no such file, **422** a file it has and cannot decode.
|
||||
|
||||
@@ -1,34 +1,45 @@
|
||||
# Extracting from your own UO client (UOFiddler)
|
||||
|
||||
**Audience:** the shard operator, once, at setup time.
|
||||
**Related:** [`CLILOCS.md`](CLILOCS.md) (why the cliloc conversion is unavoidable),
|
||||
[`SPAWN_ATLAS.md`](SPAWN_ATLAS.md) (where creature art fits).
|
||||
**Related:** [`SPAWN_ATLAS.md`](SPAWN_ATLAS.md) (where creature art fits),
|
||||
[`../link/v8.md`](../link/v8.md) (the Asset Bridge, which is replacing this page).
|
||||
|
||||
Two features read data that **only exists inside a UO client**, and a UO client's
|
||||
files are EA's, not ours to redistribute. So neither this repo nor any image we
|
||||
publish can ship them — the operator extracts from **their own** client, once,
|
||||
> ### The cliloc conversion is gone — you no longer do this by hand
|
||||
>
|
||||
> **Protocol 8, phase 2.** The shard reads its own client's `Cliloc.enu` and
|
||||
> serves the table over the bridge, so there is nothing to install, convert or
|
||||
> copy. Press **Admin → Shard → Import** after you patch your client and that is
|
||||
> the whole procedure; see [`CLILOCS.md`](CLILOCS.md).
|
||||
>
|
||||
> Part 1 of this guide has been deleted rather than rewritten. If you already
|
||||
> have a converted file it keeps working on an install with no uo-link
|
||||
> configured, but nobody should make a new one.
|
||||
>
|
||||
> **Creature art is next** (phase 5), after which this page goes away entirely.
|
||||
|
||||
What remains here reads data that **only exists inside a UO client**, and a UO
|
||||
client's files are EA's, not ours to redistribute. So neither this repo nor any
|
||||
image we publish can ship them — the operator extracts from **their own** client
|
||||
and points the site at the result.
|
||||
|
||||
| Feature | What it needs | Required? | Without it |
|
||||
|---|---|---|---|
|
||||
| **Item / title names** ([`CLILOCS.md`](CLILOCS.md)) | `Cliloc.enu`, converted | No | Names render as raw ids — `id 1023721` instead of *quarter staff* |
|
||||
| **Creature art** ([`SPAWN_ATLAS.md`](SPAWN_ATLAS.md)) | Sprites from `.mul`/`.uop` | No | Atlas pages render as text, which is the normal state |
|
||||
|
||||
**Both are optional and neither is load-bearing.** A shard that never does any of
|
||||
this is fully supported. Do part one and skip part two if art is not worth your
|
||||
time — they share only the tool.
|
||||
**It is optional and it is not load-bearing.** A shard that never does any of this
|
||||
is fully supported.
|
||||
|
||||
Everything you extract stays **outside the repository**: the converted cliloc
|
||||
file lives at a path you choose, and `spawnAtlas.art.json` plus `server/uploads/`
|
||||
are gitignored, so none of it can be committed by accident.
|
||||
Everything you extract stays **outside the repository**: `spawnAtlas.art.json` and
|
||||
`server/uploads/` are gitignored, so none of it can be committed by accident.
|
||||
|
||||
---
|
||||
|
||||
## Part 0 — Get UOFiddler
|
||||
|
||||
[UOFiddler](https://github.com/polserver/UOFiddler) is the community client-file
|
||||
editor. We use it because its `Ultima.dll` already contains the cliloc
|
||||
decompressor, maintained by people who do this for a living.
|
||||
editor. What is left here uses its GUI to export sprites; the `Ultima.dll` this
|
||||
guide used to reach for was the cliloc decompressor, and the shard has its own
|
||||
now.
|
||||
|
||||
1. Download the latest release zip from
|
||||
<https://github.com/polserver/UOFiddler/releases/latest> — one asset, named
|
||||
@@ -38,204 +49,30 @@ decompressor, maintained by people who do this for a living.
|
||||
|
||||
```
|
||||
UOFiddler-4.22.2/
|
||||
Ultima.dll ← the decompressor (Part 1 needs this path)
|
||||
UoFiddler.exe ← the GUI (Part 2 needs this)
|
||||
Ultima.dll
|
||||
plugins/
|
||||
…
|
||||
```
|
||||
|
||||
3. **Runtime:** UOFiddler 4.22.2 is built for **.NET 10**. Running `UoFiddler.exe`
|
||||
needs the .NET 10 **Desktop** Runtime (Windows only); loading `Ultima.dll` from
|
||||
the converter in Part 1 needs the .NET 10 runtime. Install from
|
||||
<https://dotnet.microsoft.com/download/dotnet/10.0>.
|
||||
3. **Runtime:** UOFiddler 4.22.2 is built for **.NET 10**, so running
|
||||
`UoFiddler.exe` needs the .NET 10 **Desktop** Runtime (Windows only). Install
|
||||
from <https://dotnet.microsoft.com/download/dotnet/10.0>.
|
||||
|
||||
### Finding your client files
|
||||
|
||||
The cliloc file is in your **UO client installation directory**, not in your
|
||||
ServUO tree — the shard server has no copy of it. Look for `Cliloc.enu` (English;
|
||||
the other seven are `chs`, `cht`, `deu`, `esp`, `fra`, `jpn`, `kor`) beside
|
||||
`art.mul` / `artLegacyMUL.uop`. The EA Classic Client's default location is:
|
||||
The art files are in your **UO client installation directory**. Look for
|
||||
`art.mul` / `artLegacyMUL.uop` and the `anim*.mul` set. The EA Classic Client's
|
||||
default location is:
|
||||
|
||||
```
|
||||
C:\Program Files (x86)\Electronic Arts\Ultima Online Classic\
|
||||
```
|
||||
|
||||
**If your shard distributes its own patched client to players, use that copy.**
|
||||
Any cliloc edits you shipped to players are then already in the base table and
|
||||
you need no overlay for them (see [`CLILOCS.md`](CLILOCS.md) §Shard-added and
|
||||
shard-edited items).
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — Convert the cliloc table
|
||||
|
||||
**Goal:** turn the client's compressed `Cliloc.enu` into a file the site can
|
||||
read, and point the site at it.
|
||||
|
||||
The site cannot read `Cliloc.enu` directly. Every modern client compresses it
|
||||
(the "Mythic" container), and so does ServUO's own bundled `Ultima.StringList` —
|
||||
which is why the shard cannot supply names on our behalf either. The full
|
||||
reasoning is in [`CLILOCS.md`](CLILOCS.md) §Why the operator has to convert the
|
||||
file; this section is just the procedure.
|
||||
|
||||
Two routes. **The bundled tool is the recommended one** — the GUI export needs a
|
||||
fixup step, described below.
|
||||
|
||||
### Route A — the bundled converter (recommended)
|
||||
|
||||
Needs a .NET SDK (any version 8 or newer — the project targets `net8.0` and rolls
|
||||
forward, so whatever you have works) **plus** the .NET 10 runtime from Part 0,
|
||||
which is what actually loads `Ultima.dll`.
|
||||
|
||||
```bash
|
||||
cd website/server/tools/cliloc-export
|
||||
dotnet build -c Release
|
||||
|
||||
# plain binary — recommended, exact
|
||||
dotnet run -c Release -- \
|
||||
"/path/to/UOFiddler-4.22.2/Ultima.dll" \
|
||||
"/path/to/UO client/Cliloc.enu" \
|
||||
/srv/uo-data/clilocs.plain
|
||||
|
||||
# or tab-delimited text, if you want to eyeball or hand-edit it
|
||||
dotnet run -c Release -- \
|
||||
"/path/to/UOFiddler-4.22.2/Ultima.dll" \
|
||||
"/path/to/UO client/Cliloc.enu" \
|
||||
/srv/uo-data/clilocs.tsv --tsv
|
||||
```
|
||||
|
||||
Expected output for a stock English client:
|
||||
|
||||
```
|
||||
wrote 123490 entries to /srv/uo-data/clilocs.plain (maxTextBytes=12150, skippedOversize=0)
|
||||
```
|
||||
|
||||
**Sanity-check that number.** A stock `Cliloc.enu` is ~123,000 entries. A few
|
||||
hundred means it read something else and you should not ship the result. The
|
||||
tool exits non-zero and says `no entries were written — is that a cliloc file?`
|
||||
when it gets nothing at all.
|
||||
|
||||
The conversion runs on whatever machine has the client (usually Windows), and the
|
||||
site reads the output wherever it runs — so **copy the output file to the server**
|
||||
if those are different machines. It is a single self-contained file (~5 MB); the
|
||||
`--tsv` form is larger but diff-able.
|
||||
|
||||
<details>
|
||||
<summary>Errors you may hit</summary>
|
||||
|
||||
| Message | Cause |
|
||||
|---|---|
|
||||
| `Ultima.StringList not found — is that really UOFiddler's Ultima.dll?` | First argument points at some other `Ultima.dll` (ServUO ships one too — it is **not** the same assembly and cannot do this) |
|
||||
| `You must install .NET to run this application` | Missing the .NET 10 runtime from Part 0 step 3 |
|
||||
| `Unexpected Ultima.StringList API` | UOFiddler older than 4.21 |
|
||||
| `usage: clilocexport …` | Fewer than three arguments |
|
||||
|
||||
</details>
|
||||
|
||||
### Route B — the UOFiddler GUI
|
||||
|
||||
Use this if you would rather not install a .NET SDK. **It needs one extra step**,
|
||||
so do not skip the fixup.
|
||||
|
||||
1. Launch `UoFiddler.exe` and point it at your client directory when it asks
|
||||
(or **Options → Path Settings**).
|
||||
2. Open the **Cliloc** tab and use its **export to CSV** action.
|
||||
3. It writes `CliLoc.csv` to UOFiddler's configured output path, in **three**
|
||||
columns with a header row:
|
||||
|
||||
```
|
||||
Number;Text;Flag
|
||||
1023721;quarter staff;0
|
||||
```
|
||||
|
||||
4. **Strip the trailing flag column.** The site's text parser reads
|
||||
`number<TAB|,|;>text`, so that third field is otherwise absorbed into the name
|
||||
and every item on the site renders as `quarter staff;0`.
|
||||
|
||||
```bash
|
||||
sed -E 's/;[0-9]+$//' CliLoc.csv > clilocs.csv
|
||||
```
|
||||
|
||||
```powershell
|
||||
Get-Content CliLoc.csv |
|
||||
ForEach-Object { $_ -replace ';\d+$','' } |
|
||||
Set-Content -Encoding utf8 clilocs.csv
|
||||
```
|
||||
|
||||
The header row needs no removal — a line whose first field is not an integer
|
||||
is skipped. Blank entries (`1005008;`) survive the fixup correctly and are
|
||||
dropped at import, as intended.
|
||||
|
||||
5. Copy `clilocs.csv` to the server.
|
||||
|
||||
**Why the fixup is not just done for us:** the parser already handles
|
||||
`number,flag,text` — the flag in the *middle*, which is what several exports
|
||||
emit. UOFiddler puts it at the *end*, where it is indistinguishable from a name
|
||||
that genuinely ends in `;0`. One `sed` on the operator's side beats a parser
|
||||
heuristic that would corrupt real names.
|
||||
|
||||
### Point the site at it
|
||||
|
||||
Two ways, the setting winning over the environment:
|
||||
|
||||
| Where | How |
|
||||
|---|---|
|
||||
| **Admin → Shard → cliloc path** | Takes effect on the next refresh, no redeploy |
|
||||
| `UO_CLIENT_PATH` env var | The deploy-time default |
|
||||
|
||||
The value may be **the file itself or a directory to search** — both are natural
|
||||
answers to "where is it", and overlays are picked up either way.
|
||||
|
||||
Setting the path deliberately does **not** import as a side effect. Click
|
||||
**Import** (or `POST /api/v1/admin/shard/clilocs/import`) to load it.
|
||||
|
||||
### Verify
|
||||
|
||||
`GET /api/v1/admin/shard/clilocs`, or the Admin → Shard panel, reports what each
|
||||
source contributed:
|
||||
|
||||
```json
|
||||
"sources": [
|
||||
{ "label": "clilocs.plain", "kind": "base", "entries": 123490, "added": 123490, "overrode": 0 }
|
||||
]
|
||||
```
|
||||
|
||||
Roughly **67,500 rows stored** from a stock table is correct — about half a
|
||||
cliloc table is empty strings for ids the client reserves and never uses.
|
||||
|
||||
Then load any character sheet with equipment: items should show names rather than
|
||||
`id 1023721`.
|
||||
|
||||
<details>
|
||||
<summary>What a refusal means</summary>
|
||||
|
||||
A bad file answers `200` with a `status` and a named reason, not a `500` — you
|
||||
need to be told *which file* to fix.
|
||||
|
||||
| `code` | Meaning |
|
||||
|---|---|
|
||||
| `COMPRESSED` | You pointed at the raw client `Cliloc.enu`. Convert it — this whole page. |
|
||||
| `TRUNCATED` | Half-copied file. Re-copy; the loaded table is untouched. |
|
||||
| `EMPTY` | A text source with no parseable rows — the file is named in the reason. |
|
||||
| `status: needsReview` + `missingSources` | A previously-loaded source has vanished (unmounted volume? deliberate deletion?). Nothing changes until you re-import with `{ "approve": true }`. |
|
||||
|
||||
</details>
|
||||
|
||||
### Custom items — do *not* re-export for these
|
||||
|
||||
Shard-added items carry ids no client table has. Drop a small delimited file in a
|
||||
`custom/` directory beside the base file and re-import:
|
||||
|
||||
```
|
||||
/srv/uo-data/
|
||||
clilocs.plain ← base, from this guide
|
||||
custom/
|
||||
01-uomysticmoon.tsv ← your additions and overrides
|
||||
```
|
||||
|
||||
Files are read in sorted order and **later sources win**, so an overlay both adds
|
||||
new ids and overrides stock ones you have re-purposed. **Adding one item never
|
||||
means re-exporting a 5 MB client file.** Details in [`CLILOCS.md`](CLILOCS.md).
|
||||
**If your shard distributes its own patched client to players, use that copy** —
|
||||
it is what your players actually see. (It is also the copy the shard itself reads
|
||||
from, since a ServUO server resolves `Config/DataPath.cfg` into its own client
|
||||
path at boot, which is the premise the Asset Bridge is built on.)
|
||||
|
||||
---
|
||||
|
||||
@@ -246,7 +83,8 @@ functional as text, and `art` is NULL on every fresh import.
|
||||
|
||||
**This project ships no art and no art-extraction tooling, and never will.**
|
||||
|
||||
1. In `UoFiddler.exe` (paths configured as in Route B step 1), open the
|
||||
1. Open `UoFiddler.exe` and point it at your client directory
|
||||
(**Settings → Paths**), then open the
|
||||
**Animations** tab for creature sprites — or **Items** for object art — find
|
||||
the creature, and export as PNG. Right-click an entry for its export options,
|
||||
or use the tab's *Export All* action for a batch. (4.22.2 added an export
|
||||
|
||||
Reference in New Issue
Block a user