Merge pull request 'docs(website): an operator runbook for extracting from your own UO client' (#75) from docs/operator-uofiddler-guide into edge
Reviewed-on: #75
This commit is contained in:
@@ -22,6 +22,7 @@ ci/ cross-cutting CI/quality notes
|
||||
| [SHARD_VISIBILITY.md](website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework |
|
||||
| [SPAWN_ATLAS.md](website/SPAWN_ATLAS.md) | The bestiary / spawn atlas: what the shard contains, parsed from its own ServUO tree |
|
||||
| [CLILOCS.md](website/CLILOCS.md) | UO's id → name table: converting one from your client so items have names |
|
||||
| [UOFIDDLER.md](website/UOFIDDLER.md) | **Operator runbook** — step-by-step extraction from your own UO client (cliloc table, creature art) |
|
||||
| [MARKETPLACE.md](website/MARKETPLACE.md) | The player-vendor index: how it is gathered, what it costs, how to tune it |
|
||||
| [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) |
|
||||
| [PROJECT_TREE.md](website/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout |
|
||||
|
||||
@@ -44,6 +44,11 @@ they did before the table existed.
|
||||
|
||||
## Converting
|
||||
|
||||
> **Step-by-step operator instructions — where to get UOFiddler, where your
|
||||
> client files are, and how to verify the import — are in
|
||||
> [`UOFIDDLER.md`](UOFIDDLER.md).** This section covers the formats and the
|
||||
> reasoning behind them.
|
||||
|
||||
Either format below is accepted; the site sniffs which one it was handed.
|
||||
|
||||
| Format | Fidelity | Notes |
|
||||
@@ -77,8 +82,16 @@ dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/cli
|
||||
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv
|
||||
```
|
||||
|
||||
A UOFiddler GUI export works equally well — anything producing one of the two
|
||||
shapes above is fine.
|
||||
A UOFiddler GUI export works too, but **not unmodified**: its Cliloc tab writes
|
||||
`Number;Text;Flag` — three columns, the flag *last* — and the parser reads
|
||||
`number<separator>text`, so the trailing field is absorbed into the name and
|
||||
every item renders as `quarter staff;0`. Stripping it is one `sed`, given in
|
||||
[`UOFIDDLER.md`](UOFIDDLER.md) §Route B.
|
||||
|
||||
The parser already tolerates `number,flag,text`, with the flag in the *middle*.
|
||||
It is not extended to cover the trailing form because a final `;0` is
|
||||
indistinguishable from a name that genuinely ends that way — a heuristic there
|
||||
would corrupt real names to save the operator one command.
|
||||
|
||||
## Shard-added and shard-edited items
|
||||
|
||||
|
||||
@@ -223,7 +223,8 @@ The atlas is fully functional as text. `shard_spawn_creatures.art` is nullable
|
||||
and is NULL on every fresh import; pages render without images, which is the
|
||||
normal and supported state, not a degraded one.
|
||||
|
||||
An operator who wants art:
|
||||
An operator who wants art — step-by-step, with the UOFiddler side spelled out, in
|
||||
[`UOFIDDLER.md`](UOFIDDLER.md) §Part 2:
|
||||
|
||||
1. Extracts it from **their own** client files (UOFiddler, ClassicUO tooling, or
|
||||
any art extractor).
|
||||
|
||||
282
website/UOFIDDLER.md
Normal file
282
website/UOFIDDLER.md
Normal file
@@ -0,0 +1,282 @@
|
||||
# Extracting from your own UO client (UOFiddler)
|
||||
|
||||
**Audience:** the shard operator, once, at setup time.
|
||||
**Related:** [`CLILOCS.md`](CLILOCS.md) (why the cliloc conversion is unavoidable),
|
||||
[`SPAWN_ATLAS.md`](SPAWN_ATLAS.md) (where creature art fits).
|
||||
|
||||
Two features read data that **only exists inside a UO client**, and a UO client's
|
||||
files are EA's, not ours to redistribute. So neither this repo nor any image we
|
||||
publish can ship them — the operator extracts from **their own** client, once,
|
||||
and points the site at the result.
|
||||
|
||||
| Feature | What it needs | Required? | Without it |
|
||||
|---|---|---|---|
|
||||
| **Item / title names** ([`CLILOCS.md`](CLILOCS.md)) | `Cliloc.enu`, converted | No | Names render as raw ids — `id 1023721` instead of *quarter staff* |
|
||||
| **Creature art** ([`SPAWN_ATLAS.md`](SPAWN_ATLAS.md)) | Sprites from `.mul`/`.uop` | No | Atlas pages render as text, which is the normal state |
|
||||
|
||||
**Both are optional and neither is load-bearing.** A shard that never does any of
|
||||
this is fully supported. Do part one and skip part two if art is not worth your
|
||||
time — they share only the tool.
|
||||
|
||||
Everything you extract stays **outside the repository**: the converted cliloc
|
||||
file lives at a path you choose, and `spawnAtlas.art.json` plus `server/uploads/`
|
||||
are gitignored, so none of it can be committed by accident.
|
||||
|
||||
---
|
||||
|
||||
## Part 0 — Get UOFiddler
|
||||
|
||||
[UOFiddler](https://github.com/polserver/UOFiddler) is the community client-file
|
||||
editor. We use it because its `Ultima.dll` already contains the cliloc
|
||||
decompressor, maintained by people who do this for a living.
|
||||
|
||||
1. Download the latest release zip from
|
||||
<https://github.com/polserver/UOFiddler/releases/latest> — one asset, named
|
||||
`UOFiddler-<version>.zip` (4.22.2 is ~2 MB).
|
||||
2. Extract it. The zip contains a single top-level folder, and the two files that
|
||||
matter are at **its root**:
|
||||
|
||||
```
|
||||
UOFiddler-4.22.2/
|
||||
Ultima.dll ← the decompressor (Part 1 needs this path)
|
||||
UoFiddler.exe ← the GUI (Part 2 needs this)
|
||||
plugins/
|
||||
…
|
||||
```
|
||||
|
||||
3. **Runtime:** UOFiddler 4.22.2 is built for **.NET 10**. Running `UoFiddler.exe`
|
||||
needs the .NET 10 **Desktop** Runtime (Windows only); loading `Ultima.dll` from
|
||||
the converter in Part 1 needs the .NET 10 runtime. Install from
|
||||
<https://dotnet.microsoft.com/download/dotnet/10.0>.
|
||||
|
||||
### Finding your client files
|
||||
|
||||
The cliloc file is in your **UO client installation directory**, not in your
|
||||
ServUO tree — the shard server has no copy of it. Look for `Cliloc.enu` (English;
|
||||
the other seven are `chs`, `cht`, `deu`, `esp`, `fra`, `jpn`, `kor`) beside
|
||||
`art.mul` / `artLegacyMUL.uop`. The EA Classic Client's default location is:
|
||||
|
||||
```
|
||||
C:\Program Files (x86)\Electronic Arts\Ultima Online Classic\
|
||||
```
|
||||
|
||||
**If your shard distributes its own patched client to players, use that copy.**
|
||||
Any cliloc edits you shipped to players are then already in the base table and
|
||||
you need no overlay for them (see [`CLILOCS.md`](CLILOCS.md) §Shard-added and
|
||||
shard-edited items).
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — Convert the cliloc table
|
||||
|
||||
**Goal:** turn the client's compressed `Cliloc.enu` into a file the site can
|
||||
read, and point the site at it.
|
||||
|
||||
The site cannot read `Cliloc.enu` directly. Every modern client compresses it
|
||||
(the "Mythic" container), and so does ServUO's own bundled `Ultima.StringList` —
|
||||
which is why the shard cannot supply names on our behalf either. The full
|
||||
reasoning is in [`CLILOCS.md`](CLILOCS.md) §Why the operator has to convert the
|
||||
file; this section is just the procedure.
|
||||
|
||||
Two routes. **The bundled tool is the recommended one** — the GUI export needs a
|
||||
fixup step, described below.
|
||||
|
||||
### Route A — the bundled converter (recommended)
|
||||
|
||||
Needs a .NET SDK (any version 8 or newer — the project targets `net8.0` and rolls
|
||||
forward, so whatever you have works) **plus** the .NET 10 runtime from Part 0,
|
||||
which is what actually loads `Ultima.dll`.
|
||||
|
||||
```bash
|
||||
cd website/server/tools/cliloc-export
|
||||
dotnet build -c Release
|
||||
|
||||
# plain binary — recommended, exact
|
||||
dotnet run -c Release -- \
|
||||
"/path/to/UOFiddler-4.22.2/Ultima.dll" \
|
||||
"/path/to/UO client/Cliloc.enu" \
|
||||
/srv/uo-data/clilocs.plain
|
||||
|
||||
# or tab-delimited text, if you want to eyeball or hand-edit it
|
||||
dotnet run -c Release -- \
|
||||
"/path/to/UOFiddler-4.22.2/Ultima.dll" \
|
||||
"/path/to/UO client/Cliloc.enu" \
|
||||
/srv/uo-data/clilocs.tsv --tsv
|
||||
```
|
||||
|
||||
Expected output for a stock English client:
|
||||
|
||||
```
|
||||
wrote 123490 entries to /srv/uo-data/clilocs.plain (maxTextBytes=12150, skippedOversize=0)
|
||||
```
|
||||
|
||||
**Sanity-check that number.** A stock `Cliloc.enu` is ~123,000 entries. A few
|
||||
hundred means it read something else and you should not ship the result. The
|
||||
tool exits non-zero and says `no entries were written — is that a cliloc file?`
|
||||
when it gets nothing at all.
|
||||
|
||||
The conversion runs on whatever machine has the client (usually Windows), and the
|
||||
site reads the output wherever it runs — so **copy the output file to the server**
|
||||
if those are different machines. It is a single self-contained file (~5 MB); the
|
||||
`--tsv` form is larger but diff-able.
|
||||
|
||||
<details>
|
||||
<summary>Errors you may hit</summary>
|
||||
|
||||
| Message | Cause |
|
||||
|---|---|
|
||||
| `Ultima.StringList not found — is that really UOFiddler's Ultima.dll?` | First argument points at some other `Ultima.dll` (ServUO ships one too — it is **not** the same assembly and cannot do this) |
|
||||
| `You must install .NET to run this application` | Missing the .NET 10 runtime from Part 0 step 3 |
|
||||
| `Unexpected Ultima.StringList API` | UOFiddler older than 4.21 |
|
||||
| `usage: clilocexport …` | Fewer than three arguments |
|
||||
|
||||
</details>
|
||||
|
||||
### Route B — the UOFiddler GUI
|
||||
|
||||
Use this if you would rather not install a .NET SDK. **It needs one extra step**,
|
||||
so do not skip the fixup.
|
||||
|
||||
1. Launch `UoFiddler.exe` and point it at your client directory when it asks
|
||||
(or **Options → Path Settings**).
|
||||
2. Open the **Cliloc** tab and use its **export to CSV** action.
|
||||
3. It writes `CliLoc.csv` to UOFiddler's configured output path, in **three**
|
||||
columns with a header row:
|
||||
|
||||
```
|
||||
Number;Text;Flag
|
||||
1023721;quarter staff;0
|
||||
```
|
||||
|
||||
4. **Strip the trailing flag column.** The site's text parser reads
|
||||
`number<TAB|,|;>text`, so that third field is otherwise absorbed into the name
|
||||
and every item on the site renders as `quarter staff;0`.
|
||||
|
||||
```bash
|
||||
sed -E 's/;[0-9]+$//' CliLoc.csv > clilocs.csv
|
||||
```
|
||||
|
||||
```powershell
|
||||
Get-Content CliLoc.csv |
|
||||
ForEach-Object { $_ -replace ';\d+$','' } |
|
||||
Set-Content -Encoding utf8 clilocs.csv
|
||||
```
|
||||
|
||||
The header row needs no removal — a line whose first field is not an integer
|
||||
is skipped. Blank entries (`1005008;`) survive the fixup correctly and are
|
||||
dropped at import, as intended.
|
||||
|
||||
5. Copy `clilocs.csv` to the server.
|
||||
|
||||
**Why the fixup is not just done for us:** the parser already handles
|
||||
`number,flag,text` — the flag in the *middle*, which is what several exports
|
||||
emit. UOFiddler puts it at the *end*, where it is indistinguishable from a name
|
||||
that genuinely ends in `;0`. One `sed` on the operator's side beats a parser
|
||||
heuristic that would corrupt real names.
|
||||
|
||||
### Point the site at it
|
||||
|
||||
Two ways, the setting winning over the environment:
|
||||
|
||||
| Where | How |
|
||||
|---|---|
|
||||
| **Admin → Shard → cliloc path** | Takes effect on the next refresh, no redeploy |
|
||||
| `UO_CLIENT_PATH` env var | The deploy-time default |
|
||||
|
||||
The value may be **the file itself or a directory to search** — both are natural
|
||||
answers to "where is it", and overlays are picked up either way.
|
||||
|
||||
Setting the path deliberately does **not** import as a side effect. Click
|
||||
**Import** (or `POST /api/v1/admin/shard/clilocs/import`) to load it.
|
||||
|
||||
### Verify
|
||||
|
||||
`GET /api/v1/admin/shard/clilocs`, or the Admin → Shard panel, reports what each
|
||||
source contributed:
|
||||
|
||||
```json
|
||||
"sources": [
|
||||
{ "label": "clilocs.plain", "kind": "base", "entries": 123490, "added": 123490, "overrode": 0 }
|
||||
]
|
||||
```
|
||||
|
||||
Roughly **67,500 rows stored** from a stock table is correct — about half a
|
||||
cliloc table is empty strings for ids the client reserves and never uses.
|
||||
|
||||
Then load any character sheet with equipment: items should show names rather than
|
||||
`id 1023721`.
|
||||
|
||||
<details>
|
||||
<summary>What a refusal means</summary>
|
||||
|
||||
A bad file answers `200` with a `status` and a named reason, not a `500` — you
|
||||
need to be told *which file* to fix.
|
||||
|
||||
| `code` | Meaning |
|
||||
|---|---|
|
||||
| `COMPRESSED` | You pointed at the raw client `Cliloc.enu`. Convert it — this whole page. |
|
||||
| `TRUNCATED` | Half-copied file. Re-copy; the loaded table is untouched. |
|
||||
| `EMPTY` | A text source with no parseable rows — the file is named in the reason. |
|
||||
| `status: needsReview` + `missingSources` | A previously-loaded source has vanished (unmounted volume? deliberate deletion?). Nothing changes until you re-import with `{ "approve": true }`. |
|
||||
|
||||
</details>
|
||||
|
||||
### Custom items — do *not* re-export for these
|
||||
|
||||
Shard-added items carry ids no client table has. Drop a small delimited file in a
|
||||
`custom/` directory beside the base file and re-import:
|
||||
|
||||
```
|
||||
/srv/uo-data/
|
||||
clilocs.plain ← base, from this guide
|
||||
custom/
|
||||
01-uomysticmoon.tsv ← your additions and overrides
|
||||
```
|
||||
|
||||
Files are read in sorted order and **later sources win**, so an overlay both adds
|
||||
new ids and overrides stock ones you have re-purposed. **Adding one item never
|
||||
means re-exporting a 5 MB client file.** Details in [`CLILOCS.md`](CLILOCS.md).
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Creature art for the spawn atlas (optional)
|
||||
|
||||
**Goal:** put sprites on atlas pages. Purely cosmetic — the atlas is fully
|
||||
functional as text, and `art` is NULL on every fresh import.
|
||||
|
||||
**This project ships no art and no art-extraction tooling, and never will.**
|
||||
|
||||
1. In `UoFiddler.exe` (paths configured as in Route B step 1), open the
|
||||
**Animations** tab for creature sprites — or **Items** for object art — find
|
||||
the creature, and export as PNG. Right-click an entry for its export options,
|
||||
or use the tab's *Export All* action for a batch. (4.22.2 added an export
|
||||
option to the Animation tab's thumbnail list, which is the convenient one
|
||||
here.)
|
||||
2. Put the images under `server/uploads/atlas/`.
|
||||
3. Copy `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json` in
|
||||
the same directory and map creature slugs to file names:
|
||||
|
||||
```json
|
||||
{
|
||||
"lizardman": "lizardman.png",
|
||||
"orc": "orc.png"
|
||||
}
|
||||
```
|
||||
|
||||
**Keys are the slugs the atlas API reports**, derived from the type names in
|
||||
your own shard's `Spawns/*.xml` — read them off the atlas rather than guessing.
|
||||
A creature with no entry renders without art, which is the default.
|
||||
|
||||
4. Restart, or `npm run atlas:import -- --force`.
|
||||
|
||||
The art map is re-read on every atlas refresh, so adding one image is an edit plus
|
||||
a refresh. Both `spawnAtlas.art.json` and `server/uploads/` are gitignored.
|
||||
|
||||
---
|
||||
|
||||
## Licensing, briefly
|
||||
|
||||
UO's strings and sprites are EA's. Extracting from **your own** client for
|
||||
**your own** shard is the arrangement here; redistributing the extracted files is
|
||||
not something this project does or can advise on. That is the whole reason this
|
||||
page exists instead of a download link.
|
||||
Reference in New Issue
Block a user