feat(shard): resolve cliloc names for items and reward titles
Protocol 3.0 §8.6 (docs/link/v3.md), the dependency order 5 was sequenced
behind. Items on the wire carry a LabelNumber, not a name — the bridge has
always sent it (char.profile.equipment.cliloc, reward titles as a cliloc
number in string form, and one per marketplace listing) but the site had no
table to resolve it against, so a character sheet could only render
`id 1023721` where the game renders "quarter staff".
The number was never the missing piece. The table was.
Sourced from a file the operator converts once from their own client, at a
path from the `cliloc_client_path` setting falling back to UO_CLIENT_PATH.
Nothing client-derived is committed: UO's strings are EA's, exactly as the
creature sprites are. A shard with nothing configured is fully supported —
names render as ids, as they did before.
The conversion step is not avoidable, and that is the substantive finding
here: every current client ships its cliloc files COMPRESSED (first DWORD's
high byte 0x8E, the Mythic container), and ServUO's own bundled
Ultima.StringList cannot read that either — so VendorSearch.GetItemName is
already inert on such a shard and the plugin could not supply names instead.
v3.md's original "read the client's Cliloc.enu" recommendation was therefore
not implementable as written, and its committed db/data/clilocs.json artifact
also predates the Part C corrections (no committed derived snapshots, nothing
EA-derived shipped). Replaced with the spawn-atlas pattern: parse on boot from
an operator-configured path, hash-gated, output gitignored.
- utils/clilocParse.js — pure parsers, fs-free so the suite runs in CI.
Accepts the plain binary layout and delimited text, sniffed by header rather
than extension. Rejects a compressed file BY NAME: without that check the
plain parser reads it as ~19k records of negative ids and 60 KB "strings"
before dying mid-file, and the resulting error names the wrong problem.
displayText() drops the ~1_val~ arguments the bridge never sends.
- utils/clilocSource.js — the fs layer. hashSource reports `compressed` so the
admin panel can flag an unconverted file WITHOUT parsing 5 MB per poll;
otherwise pointing at a client directory reports a healthy file with pending
drift ("ready to import") and the operator only finds out on failure.
- model/shardClilocs — refresh/status/lookup. All-or-nothing replace (DELETE,
not TRUNCATE — TRUNCATE is DDL in MariaDB and implicitly commits). Batched
server-side resolution behind a capped cache; never throws, because a cliloc
lookup is decoration on a character sheet.
- Deliberately NO staged-approval flow, unlike the atlas: the atlas escalates
facet loss because a half-copied tree and a real map change are
indistinguishable from inside the process, whereas a partial cliloc copy
makes the parser fail on a truncated record. The ambiguity the atlas must
escalate is one this parser simply detects.
- No public route. The table is never served AS a table: 67k rows would dwarf
any page using them, and the Android client consumes the same resolved JSON.
Two parser bugs found by building it, both now covered by tests: trimming a
text line before splitting ate the trailing separator on empty-text entries
and silently dropped 55,994 of 123,490 while still reporting success; and
Number('') is 0, not NaN, so a line starting with a separator imported as a
bogus cliloc 0.
Verified against the real client table (123,490 entries) and the live MariaDB:
import 663 ms, hash-gated boot no-op 14 ms, cold resolve 4.2 ms / warm 0.015 ms.
Binary and TSV imports converge on the same 67,496 rows with identical keys
(blank entries — half the table — are dropped at import). A file truncated to
half its length is refused with TRUNCATED and leaves the previous table
serving. Boot logs verified for both the import and the compressed-file
warning; neither blocks startup. All three admin routes exercised over HTTP
with a real session. 629 server tests pass; client builds clean; swagger,
routes.manifest.json and routes.guards.json regenerated.
Not covered by an automated test: the character sheet renders resolved names
in presentational React with no DOM test harness in this repo, and was not
rendered against a live linked-player profile — that needs a logged-in player
with a linked game account and a shard answering a profile RPC.
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
64
server/tools/cliloc-export/README.md
Normal file
64
server/tools/cliloc-export/README.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# cliloc-export
|
||||
|
||||
Converts a UO client's **compressed** `Cliloc.enu` into the plain format the
|
||||
website can read.
|
||||
|
||||
This is a one-off operator utility, not part of the website build. Nothing in the
|
||||
Node application references it and CI never touches it. Full background —
|
||||
including why the conversion is necessary at all — is in
|
||||
[`docs/website/CLILOCS.md`](https://gitea.whitlocktech.com/RunicGateway/docs/src/branch/edge/website/CLILOCS.md).
|
||||
|
||||
## The short version
|
||||
|
||||
Every current UO client ships its cliloc files in the compressed "Mythic"
|
||||
container (the first DWORD's high byte is `0x8E`). The website parses the plain
|
||||
layout those files used before that change. Decompressing is an inverse-BWT coder
|
||||
that the site has no business carrying at runtime — and ServUO's own bundled
|
||||
`Ultima.StringList` cannot read it either, so the shard cannot supply item names
|
||||
on our behalf.
|
||||
|
||||
So: convert once, here, using a decompressor that already exists and is already
|
||||
maintained — [UOFiddler](https://github.com/polserver/UOFiddler)'s `Ultima.dll`.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
dotnet build -c Release
|
||||
|
||||
# plain binary (recommended — exact)
|
||||
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.plain
|
||||
|
||||
# tab-delimited text (convenient; does not preserve leading/trailing whitespace)
|
||||
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv
|
||||
```
|
||||
|
||||
Then point the site at the output: **Admin → Shard → cliloc path**, or the
|
||||
`UO_CLIENT_PATH` environment variable. The setting wins over the environment.
|
||||
|
||||
Expected output for a stock English client:
|
||||
|
||||
```
|
||||
wrote 123490 entries to /srv/uo-data/clilocs.plain (maxTextBytes=12150, skippedOversize=0)
|
||||
```
|
||||
|
||||
The site stores ~67,500 of those — roughly half a cliloc table is empty strings
|
||||
for ids the client reserves and never uses.
|
||||
|
||||
## Two implementation notes worth keeping
|
||||
|
||||
**`Ultima.dll` is loaded reflectively, not referenced.** UOFiddler ships as
|
||||
net10.0; a project reference from an older SDK fails at *compile* time with
|
||||
CS1705. Reflection moves that to run time, where `RollForward: LatestMajor`
|
||||
answers it — so this builds on whatever SDK you have and runs on the newest
|
||||
runtime installed.
|
||||
|
||||
**`StringList.SaveStringList` is not the export path**, despite looking exactly
|
||||
like it. It *re-compresses* on save, because its purpose is round-tripping a file
|
||||
back into the client — its output is byte-identical to its input. The plain
|
||||
records are written by hand for that reason.
|
||||
|
||||
## Output is never committed
|
||||
|
||||
UO's strings are EA's. `.gitignore` covers this project's build output and the
|
||||
conventional in-repo output location, but the supported arrangement is a path
|
||||
**outside** the repository entirely.
|
||||
Reference in New Issue
Block a user