wtclaude
b61a4d6721
Protocol 3.0 §8.6 (docs/link/v3.md), the dependency order 5 was sequenced
behind. Items on the wire carry a LabelNumber, not a name — the bridge has
always sent it (char.profile.equipment.cliloc, reward titles as a cliloc
number in string form, and one per marketplace listing) but the site had no
table to resolve it against, so a character sheet could only render
`id 1023721` where the game renders "quarter staff".
The number was never the missing piece. The table was.
Sourced from a file the operator converts once from their own client, at a
path from the `cliloc_client_path` setting falling back to UO_CLIENT_PATH.
Nothing client-derived is committed: UO's strings are EA's, exactly as the
creature sprites are. A shard with nothing configured is fully supported —
names render as ids, as they did before.
The conversion step is not avoidable, and that is the substantive finding
here: every current client ships its cliloc files COMPRESSED (first DWORD's
high byte 0x8E, the Mythic container), and ServUO's own bundled
Ultima.StringList cannot read that either — so VendorSearch.GetItemName is
already inert on such a shard and the plugin could not supply names instead.
v3.md's original "read the client's Cliloc.enu" recommendation was therefore
not implementable as written, and its committed db/data/clilocs.json artifact
also predates the Part C corrections (no committed derived snapshots, nothing
EA-derived shipped). Replaced with the spawn-atlas pattern: parse on boot from
an operator-configured path, hash-gated, output gitignored.
- utils/clilocParse.js — pure parsers, fs-free so the suite runs in CI.
Accepts the plain binary layout and delimited text, sniffed by header rather
than extension. Rejects a compressed file BY NAME: without that check the
plain parser reads it as ~19k records of negative ids and 60 KB "strings"
before dying mid-file, and the resulting error names the wrong problem.
displayText() drops the ~1_val~ arguments the bridge never sends.
- utils/clilocSource.js — the fs layer. hashSource reports `compressed` so the
admin panel can flag an unconverted file WITHOUT parsing 5 MB per poll;
otherwise pointing at a client directory reports a healthy file with pending
drift ("ready to import") and the operator only finds out on failure.
- model/shardClilocs — refresh/status/lookup. All-or-nothing replace (DELETE,
not TRUNCATE — TRUNCATE is DDL in MariaDB and implicitly commits). Batched
server-side resolution behind a capped cache; never throws, because a cliloc
lookup is decoration on a character sheet.
- Deliberately NO staged-approval flow, unlike the atlas: the atlas escalates
facet loss because a half-copied tree and a real map change are
indistinguishable from inside the process, whereas a partial cliloc copy
makes the parser fail on a truncated record. The ambiguity the atlas must
escalate is one this parser simply detects.
- No public route. The table is never served AS a table: 67k rows would dwarf
any page using them, and the Android client consumes the same resolved JSON.
Two parser bugs found by building it, both now covered by tests: trimming a
text line before splitting ate the trailing separator on empty-text entries
and silently dropped 55,994 of 123,490 while still reporting success; and
Number('') is 0, not NaN, so a line starting with a separator imported as a
bogus cliloc 0.
Verified against the real client table (123,490 entries) and the live MariaDB:
import 663 ms, hash-gated boot no-op 14 ms, cold resolve 4.2 ms / warm 0.015 ms.
Binary and TSV imports converge on the same 67,496 rows with identical keys
(blank entries — half the table — are dropped at import). A file truncated to
half its length is refused with TRUNCATED and leaves the previous table
serving. Boot logs verified for both the import and the compressed-file
warning; neither blocks startup. All three admin routes exercised over HTTP
with a real session. 629 server tests pass; client builds clean; swagger,
routes.manifest.json and routes.guards.json regenerated.
Not covered by an automated test: the character sheet renders resolved names
in presentational React with no DOM test harness in this repo, and was not
rendered against a live linked-player profile — that needs a logged-in player
with a linked game account and a shard answering a profile RPC.
Co-Authored-By: Claude <noreply@anthropic.com>
2.6 KiB
cliloc-export
Converts a UO client's compressed Cliloc.enu into the plain format the
website can read.
This is a one-off operator utility, not part of the website build. Nothing in the
Node application references it and CI never touches it. Full background —
including why the conversion is necessary at all — is in
docs/website/CLILOCS.md.
The short version
Every current UO client ships its cliloc files in the compressed "Mythic"
container (the first DWORD's high byte is 0x8E). The website parses the plain
layout those files used before that change. Decompressing is an inverse-BWT coder
that the site has no business carrying at runtime — and ServUO's own bundled
Ultima.StringList cannot read it either, so the shard cannot supply item names
on our behalf.
So: convert once, here, using a decompressor that already exists and is already
maintained — UOFiddler's Ultima.dll.
Usage
dotnet build -c Release
# plain binary (recommended — exact)
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.plain
# tab-delimited text (convenient; does not preserve leading/trailing whitespace)
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv
Then point the site at the output: Admin → Shard → cliloc path, or the
UO_CLIENT_PATH environment variable. The setting wins over the environment.
Expected output for a stock English client:
wrote 123490 entries to /srv/uo-data/clilocs.plain (maxTextBytes=12150, skippedOversize=0)
The site stores ~67,500 of those — roughly half a cliloc table is empty strings for ids the client reserves and never uses.
Two implementation notes worth keeping
Ultima.dll is loaded reflectively, not referenced. UOFiddler ships as
net10.0; a project reference from an older SDK fails at compile time with
CS1705. Reflection moves that to run time, where RollForward: LatestMajor
answers it — so this builds on whatever SDK you have and runs on the newest
runtime installed.
StringList.SaveStringList is not the export path, despite looking exactly
like it. It re-compresses on save, because its purpose is round-tripping a file
back into the client — its output is byte-identical to its input. The plain
records are written by hand for that reason.
Output is never committed
UO's strings are EA's. .gitignore covers this project's build output and the
conventional in-repo output location, but the supported arrangement is a path
outside the repository entirely.