Files
docs/website/CLILOCS.md
wtclaude be9f5019fa docs(link): the cliloc table, and why §8.6's recommendation was not implementable
Protocol 3.0 §8.6 resolved as its own website-only change, landing ahead of
§8 so the marketplace ships with real item names. Matching documentation for
website #TBD.

NEW website/CLILOCS.md — operator-facing: why the conversion step exists, how
to convert, how to configure the path, the refresh contract, what gets stored
and how names are applied.

link/v3.md §8.6 rewritten. Two things in the original recommendation turned
out to be wrong, and both are recorded because the reasoning generalises:

1. The committed db/data/clilocs.json artifact predates the Part C
   corrections (§6.1) and violates both — no committed snapshot of derived
   content, and nothing EA-derived ever shipped. UO's strings are EA's,
   exactly as the creature sprites are.
2. "scripts/buildClilocs.js reads the UO client's Cliloc.enu" is not
   possible. EVERY current client ships its cliloc files compressed (first
   DWORD's high byte 0x8E, the Mythic container); the plain layout is what
   those files looked like before that change, and parsing one as the other
   does not fail cleanly — it yields ~19k records with negative ids, 1,722
   distinct keys out of 19,508, one 62 KB "string", and a truncation
   somewhere in the middle. ServUO's own Ultima.StringList cannot read it
   either, so VendorSearch.GetItemName is already inert on such a shard and
   the work could not be pushed to the plugin.

That second point also retires an open question in §8.2: the warning never to
call GetItemName in the market sweep costs us nothing we could otherwise have
had, because the in-game Vendor Search gump has the same gap.

Three traps found by building it are recorded: StringList.SaveStringList
RE-COMPRESSES on save (its output is byte-identical to its compressed input,
because its purpose is round-tripping a file back into the client); trimming a
text line before splitting silently drops the ~half of a table that is empty
strings; and Number('') is 0, not NaN.

Also updated:
- Progress and §9 sequencing tables: order 5 split into 5a (this, website
  only) and 5b (the four-repo wire change).
- website/BACKEND_DESIGN.md — shard_clilocs / shard_cliloc_meta, the three
  admin routes, and why there is no staged-approval flow and no public route.
- link/INTEGRATION.md — the char.profile field note now says explicitly not to
  expect the shard to resolve clilocs, and points at CLILOCS.md.
- §10 documentation obligations list CLILOCS.md.

Documentation only. Every claim was written after the corresponding behaviour
was observed running: the compressed-format finding and the parse failures
come from the real client files on this machine, and the counts (123,490
parsed → 67,496 stored) and timings from importing them into the live MariaDB.

PROJECT_TREE.md files are deliberately untouched — they are CI-generated by
the sync-project-tree workflow and say so in their header.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-29 04:22:05 -05:00

10 KiB

Cliloc table (item and title names)

Status: Complete on edge — website #TBD. Design: docs/link/v3.md §8.6 — Protocol 3.0, the dependency Part B/3 was sequenced behind.

A "cliloc" is UO's localization table: an integer id mapped to a display string. Items on the wire carry a LabelNumber, not a name. The bridge has always sent that number — char.profile.equipment has a cliloc field, reward titles arrive as a cliloc number in string form, and every marketplace listing carries one — but the site had no table to look it up in, so a character sheet could only render id 1023721 where the game renders "quarter staff".

The number was never the missing piece. The table was.

Why the operator has to convert the file

This is the awkward part, and it is not avoidable:

Every current UO client ships its cliloc files compressed. The four Cliloc.* files in a modern client all begin with a DWORD whose high byte is 0x8E — the "Mythic" compressed container. The plain layout this site parses is what those files looked like before that change.

Decompressing it means an inverse-BWT coder with a frequency header — a few hundred lines of bit-level work whose failure mode is plausible-looking garbage rather than an error. The site has no business carrying that at runtime.

Two facts make the alternatives worse, not better:

  • ServUO cannot read it either. Its bundled Ultima.StringList implements only the plain layout, so on a modern client VendorSearch.StringList is null and VendorSearch.GetItemName returns item.Name — usually nothing. The shard cannot supply names on our behalf; the in-game Vendor Search gump has the same gap.
  • Nothing client-derived may be committed. UO's strings are EA's. The repo ships no string table for the same reason it ships no artwork and no map snapshot — see SPAWN_ATLAS.md.

So the conversion happens once, on the operator's machine, against their own client, and the site reads the result from a path it is given. A shard that never does this is in a fully supported state: names render as ids, exactly as they did before the table existed.

Converting

Either format below is accepted; the site sniffs which one it was handed.

Format Fidelity Notes
Plain binary (recommended) Exact 6-byte header, then {int32 number, byte flag, uint16 length, UTF-8} records
Delimited text Loses leading/trailing whitespace number<TAB|,|;>text per line; a header row, blank lines and # comments are ignored

The whitespace caveat is real but cosmetic: ~1,300 of the 123,490 entries in a stock Cliloc.enu are label prefixes like "max = " whose trailing space is meaningful when the client concatenates a value onto them. Nothing on this site concatenates, and every consumer passes through displayText(), which trims.

Using the bundled tool

server/tools/cliloc-export/ is a small .NET console app that drives UOFiddler's Ultima.dll — the decompressor that already exists and is already maintained — and writes the plain format. It loads that DLL reflectively so it compiles against any SDK, and it writes the records by hand because UOFiddler's own SaveStringList re-compresses on save (its purpose is round-tripping a file back into the client, so its output is byte-identical to its input — a trap worth knowing about).

cd website/server/tools/cliloc-export
dotnet build -c Release

# binary (recommended)
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.plain

# or tab-delimited
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv

A UOFiddler GUI export works equally well — anything producing one of the two shapes above is fine.

Configuring the path

Two ways to point at the converted file, the setting winning over the environment:

Source Notes
cliloc_client_path setting Admin-editable (Admin → Shard); takes effect on the next refresh without a redeploy
UO_CLIENT_PATH env var The deploy-time default, since the path usually describes a mount the deployment sets up

The value may be the file itself or a directory to search, because both are natural answers to "where is it". A directory is searched case-insensitively (the client writes Cliloc.enu on Windows; the site usually runs on Linux) for, in order: clilocs.tsv, clilocs.csv, cliloc.plain.enu, cliloc.enu.plain, clilocs.txt, cliloc.enu.

That ordering puts explicitly-converted names first on purpose. Pointing the setting straight at an unconverted client directory finds cliloc.enu, which is compressed — and the site says so by name rather than failing obscurely:

status: unavailable
code:   COMPRESSED
reason: This is a compressed (Mythic-format) cliloc file, which the site cannot
        read. Convert it to the plain format first — see docs/website/CLILOCS.md.

Refresh contract

Identical in shape to the spawn atlas, and for the same reasons:

  • It never blocks startup. No path, an unreadable file, a wrong-format file, a database error — all caught and logged. The site comes up either way.
  • Hash-gated. The boot path hashes the file and skips the parse entirely when it matches what is loaded, which is every restart that did not follow a client patch. Measured on a stock table: 14 ms for the no-op, 663 ms for a full parse and replace.
  • A PARSER_VERSION bump also counts as drift, so a corrected parse reaches an install whose client never patches.

Why there is no staged-approval flow

The atlas stages a refresh that would remove a facet, because a half-copied tree and a real map change are indistinguishable from inside the process. A cliloc file is one file with one hash, and its realistic corruption — a partial copy — makes the parser fail on a truncated record instead of yielding a plausible-but-short table. The ambiguity the atlas has to escalate to a human is one this parser can simply detect, so it refuses the import and leaves the previous table serving. Verified: a file truncated to half its length reports

code:   TRUNCATED
reason: Truncated record header at byte 2486759 (74909 entries read)

and the 67,496 rows already loaded are untouched.

What gets stored

Parsed from a stock Cliloc.enu 123,490 entries
Of those, empty strings 55,994 (ids the client reserves and never uses)
Stored in shard_clilocs 67,496

Blank entries are dropped at import. A row resolving to no name is indistinguishable from no row at all to every caller, and dropping them makes the binary and text imports converge on identical content — the binary format carries the blanks explicitly and a text export may or may not, depending on the tool. Verified: both formats import to the same 67,496 rows with the same keys.

text is TEXT, not VARCHAR: the long property descriptions reach 12 KB, and silently truncating them would be worse than storing them. The index that matters for marketplace search is on the denormalized shard_vendor_items.display_name, not here.

How names are applied

Resolution happens server-side. The table is never served as a table and there is no public route for it. Two reasons: 67k rows would dwarf any page that used them, and the Android client consumes the same JSON and would otherwise need its own copy.

resolveMany() takes a batch of ids and returns a Map holding only those that resolved to something displayable, so "no such id" and "id with no usable name" collapse into one branch at the call site. It never throws — a cliloc lookup is decoration on someone's character sheet, and a database blip must not fail the sheet. A capped in-process cache fronts it; measured cold 4.2 ms, warm 0.015 ms.

displayText()

Cliloc strings interpolate arguments the client pulls from an item's property list — ~1_val~, ~2_NAME~. We never have those: the bridge sends the id, not the packet. So a name carrying them is reduced to what is actually knowable.

Raw Displayed
quarter staff quarter staff
cold damage ~1_val~% cold damage
[~1_stuff~] (nothing — the whole string was the argument)
50% 50%

The trailing % in row two is dropped because a placeholder was removed — it is the unit belonging to the number we never had. Row four shows why that condition matters: stripping % unconditionally would corrupt a string that legitimately ends in one.

Consumers

  • Character sheet equipment. enrichCharProfile attaches clilocName to each item. A player-given name always wins — "Bob's lucky axe" must not be relabelled "hatchet" — and the client re-states that precedence.
  • Reward titles. titles.rewardResolved is a parallel array with the numeric entries turned into words (null where nothing resolved). The sheet used to skip numeric reward titles entirely, having no way to render them.
  • Marketplace listings (Protocol 3.0 §8) denormalize the resolved name into shard_vendor_items.display_name so search can index it.

Admin surface

All admin-only, alongside the atlas under Admin → Shard:

Route Purpose
GET /api/v1/admin/shard/clilocs Path, resolved file, readability, drift, entry count
POST /api/v1/admin/shard/clilocs/import Reload after a client patch; { "force": true } reimports an unchanged file
PUT /api/v1/admin/shard/clilocs/path Set the path; blank disables resolution

A refresh result is not an exception: a missing file, or the likely mistake of pointing at the client's own compressed Cliloc.enu, answers 200 with status: "unavailable" and a reason. A 500 would say only "something broke"; the operator needs to be told which file to convert. Setting the path deliberately does not import as a side effect — the response carries the refreshed status so the panel can offer that as the next step.