docs/website/CLILOCS.md

Cliloc table (item and title names)

Status: Complete on edge — website #115, docs #70. Design: docs/link/v3.md §8.6 — Protocol 3.0, the dependency Part B/3 was sequenced behind.

A "cliloc" is UO's localization table: an integer id mapped to a display string. Items on the wire carry a LabelNumber, not a name. The bridge has always sent that number — char.profile.equipment has a cliloc field, reward titles arrive as a cliloc number in string form, and every marketplace listing carries one — but the site had no table to look it up in, so a character sheet could only render id 1023721 where the game renders "quarter staff".

The number was never the missing piece. The table was.

Why the operator has to convert the file

This is the awkward part, and it is not avoidable:

Every current UO client ships its cliloc files compressed. All eight Cliloc.* files in a modern client (chs, cht, deu, enu, esp, fra, jpn, kor) begin with a DWORD whose high byte is 0x8E — the "Mythic" compressed container. The plain layout this site parses is what those files looked like before that change.

Decompressing it means an inverse-BWT coder with a frequency header — a few hundred lines of bit-level work whose failure mode is plausible-looking garbage rather than an error. The site has no business carrying that at runtime.

Two facts make the alternatives worse, not better:

• ServUO cannot read it either. Its bundled Ultima.StringList implements only the plain layout, so on a modern client VendorSearch.StringList is null and VendorSearch.GetItemName returns item.Name — usually nothing. The shard cannot supply names on our behalf; the in-game Vendor Search gump has the same gap.
• Nothing client-derived may be committed. UO's strings are EA's. The repo ships no string table for the same reason it ships no artwork and no map snapshot — see SPAWN_ATLAS.md.

So the conversion happens once, on the operator's machine, against their own client, and the site reads the result from a path it is given. A shard that never does this is in a fully supported state: names render as ids, exactly as they did before the table existed.

Converting

Either format below is accepted; the site sniffs which one it was handed.

Format	Fidelity	Notes
Plain binary (recommended)	Exact	6-byte header, then {int32 number, byte flag, uint16 length, UTF-8} records
Delimited text	Loses leading/trailing whitespace	number<TAB|,|;>text per line; a header row, blank lines and # comments are ignored

The whitespace caveat is real but cosmetic: ~1,300 of the 123,490 entries in a stock Cliloc.enu are label prefixes like "max = " whose trailing space is meaningful when the client concatenates a value onto them. Nothing on this site concatenates, and every consumer passes through displayText(), which trims.

Using the bundled tool

server/tools/cliloc-export/ is a small .NET console app that drives UOFiddler's Ultima.dll — the decompressor that already exists and is already maintained — and writes the plain format. It loads that DLL reflectively so it compiles against any SDK, and it writes the records by hand because UOFiddler's own SaveStringList re-compresses on save (its purpose is round-tripping a file back into the client, so its output is byte-identical to its input — a trap worth knowing about).

cd website/server/tools/cliloc-export
dotnet build -c Release

# binary (recommended)
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.plain

# or tab-delimited
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv

A UOFiddler GUI export works equally well — anything producing one of the two shapes above is fine.

Shard-added and shard-edited items

Shards edit items and add new ones, and those carry cliloc ids no stock client table has. The table is therefore built from a set of sources, all re-read on every boot and hash-gated together — the same shape as the spawn atlas, which reads Regions.xml + Locations/*.xml + Spawns/*.xml + ChampionSpawns.xml and merges them:

<cliloc path>/
  clilocs.plain            ← base: the converted client table
  custom/
    01-uomysticmoon.tsv    ← overlays: shard additions and overrides
    02-events.tsv

Overlays use the same delimited-text format, are read in sorted order, and later sources win — so an overlay both adds ids the client never had and overrides stock ones the shard has re-purposed. Any .tsv, .csv, .txt, .enu or .plain file in custom/ is picked up; anything else (a README.md, say) is ignored.

Adding, editing or removing any overlay counts as drift, so a new custom item needs only a file edit and a restart — or the admin panel's Import button. Adding one item never means re-exporting a 5 MB client file.

The import result reports what each source contributed, which is how you confirm an overlay took effect — overrode: 0 on a file meant to re-label stock items says it did not:

"sources": [
  { "label": "clilocs.plain",           "kind": "base",   "entries": 123490, "added": 123490, "overrode": 0 },
  { "label": "custom/uomysticmoon.tsv", "kind": "custom", "entries": 2,      "added": 1,      "overrode": 1 }
]

Why a convention rather than discovery. Everywhere else this pipeline follows the shard's own files, but ServUO has no server-side notion of a custom cliloc — they live in the patched client a shard distributes to its players, and nothing in the tree declares them. There is nothing to discover, so custom/ is the one thing here that is our convention rather than the shard's. (An operator who does patch their client cliloc needs no overlay at all: convert the patched file and their edits are simply in the base.)

Measured on the live shard for scale: its script tree references 16,434 cliloc ids and only 37 are absent from the stock client table — tens of entries against a 67k base, which is what makes an overlay the right shape rather than a second full table.

Configuring the path

Two ways to point at the sources, the setting winning over the environment:

Source	Notes
cliloc_client_path setting	Admin-editable (Admin → Shard); takes effect on the next refresh without a redeploy
UO_CLIENT_PATH env var	The deploy-time default, since the path usually describes a mount the deployment sets up

The value may be the base file itself or a directory to search, because both are natural answers to "where is it". Overlays are read from a custom/ directory beside the base either way — pointing at a file does not forfeit them.

A directory is searched case-insensitively (the client writes Cliloc.enu on Windows; the site usually runs on Linux) for, in order: clilocs.tsv, clilocs.csv, clilocs.plain, cliloc.plain, cliloc.plain.enu, cliloc.enu.plain, clilocs.txt, cliloc.enu.

That ordering puts explicitly-converted names first on purpose. Pointing the setting straight at an unconverted client directory finds cliloc.enu, which is compressed — and the site says so by name rather than failing obscurely:

status: unavailable
code:   COMPRESSED
reason: This is a compressed (Mythic-format) cliloc file, which the site cannot
        read. Convert it to the plain format first — see docs/website/CLILOCS.md.

Refresh contract

Identical in shape to the spawn atlas, and for the same reasons:

• It never blocks startup. No path, an unreadable file, a wrong-format file, a database error — all caught and logged. The site comes up either way.
• Hash-gated. The boot path hashes the file and skips the parse entirely when it matches what is loaded, which is every restart that did not follow a client patch. Measured on a stock table: 14 ms for the no-op, 663 ms for a full parse and replace.
• A PARSER_VERSION bump also counts as drift, so a corrected parse reaches an install whose client never patches.

Two ways a refresh is refused

A corrupt file — the realistic failure for any single source — makes the parser fail on a truncated record rather than yield a plausible-but-short table, so it is caught outright. Verified: a file truncated to half its length reports

code:   TRUNCATED
reason: Truncated record header at byte 2486759 (74909 entries read)

and the rows already loaded are untouched. A malformed overlay names the file it came from (custom/broken.tsv: No cliloc entries found…), because "which of my six overlay files is broken" is otherwise a guessing game.

A source that has VANISHED is the hazard a single file did not have. It parses perfectly and imports a table quietly missing everything that file contributed — and an unmounted volume looks exactly like a deliberate deletion from here. This is the same ambiguity the atlas stages a facet removal for, so it is escalated rather than applied:

status:         needsReview
reason:         1 previously-loaded cliloc source(s) are missing;
                the existing table is unchanged
missingSources: ["custom/uomysticmoon.tsv"]

status() reports missingSources too, so the panel can show it before anyone clicks Import. An admin accepts it by re-running the import with { "approve": true }.

Why that is a flag and not the atlas's approve/reject pair. The atlas stores a pending decision in its own table so that approving re-parses the tree, which is what keeps a multi-megabyte blob out of the database and makes the applied result match the tree at approval time. Here nothing is stored, so re-reading at approval time is automatic — the decision is a single boolean on the import an admin was already going to run.

What gets stored

Parsed from a stock Cliloc.enu	123,490 entries
Of those, empty strings	55,994 (ids the client reserves and never uses)
Stored in shard_clilocs	67,496

Blank entries are dropped at import. A row resolving to no name is indistinguishable from no row at all to every caller, and dropping them makes the binary and text imports converge on identical content — the binary format carries the blanks explicitly and a text export may or may not, depending on the tool. Verified: both formats import to the same 67,496 rows with the same keys.

text is TEXT, not VARCHAR: the long property descriptions reach 12 KB, and silently truncating them would be worse than storing them. The index that matters for marketplace search is on the denormalized shard_vendor_items.display_name, not here.

How names are applied

Resolution happens server-side. The table is never served as a table and there is no public route for it. Two reasons: 67k rows would dwarf any page that used them, and the Android client consumes the same JSON and would otherwise need its own copy.

resolveMany() takes a batch of ids and returns a Map holding only those that resolved to something displayable, so "no such id" and "id with no usable name" collapse into one branch at the call site. It never throws — a cliloc lookup is decoration on someone's character sheet, and a database blip must not fail the sheet. A capped in-process cache fronts it; measured cold 4.2 ms, warm 0.015 ms.

displayText()

Cliloc strings interpolate arguments the client pulls from an item's property list — ~1_val~, ~2_NAME~. We never have those: the bridge sends the id, not the packet. So a name carrying them is reduced to what is actually knowable.

Raw	Displayed
quarter staff	quarter staff
cold damage ~1_val~%	cold damage
[~1_stuff~]	(nothing — the whole string was the argument)
50%	50%
Runic Gateway Sigil (v2)	Runic Gateway Sigil (v2)

Punctuation is only tidied when a placeholder was actually removed. The trailing % in row two is the unit belonging to the number we never had, and the brackets in row three only ever wrapped the argument — but a string with no placeholder has no such debris, and trimming it anyway corrupts real names. Rows four and five are the ones that caught it: a shard's custom "Runic Gateway Sigil (v2)" rendered as "(v2" while the bracket trim was unconditional.

Consumers

• Character sheet equipment. enrichCharProfile attaches clilocName to each item. A player-given name always wins — "Bob's lucky axe" must not be relabelled "hatchet" — and the client re-states that precedence.
• Reward titles. titles.rewardResolved is a parallel array with the numeric entries turned into words (null where nothing resolved). The sheet used to skip numeric reward titles entirely, having no way to render them.
• Marketplace listings (Protocol 3.0 §8) denormalize the resolved name into shard_vendor_items.display_name so search can index it.

Admin surface

All admin-only, alongside the atlas under Admin → Shard:

Route	Purpose
GET /api/v1/admin/shard/clilocs	Sources found, what each contributed at the last import, readability, drift, entry count, missingSources
POST /api/v1/admin/shard/clilocs/import	Reload after a client patch or an overlay edit; { "force": true } reimports an unchanged set, { "approve": true } accepts a vanished source
PUT /api/v1/admin/shard/clilocs/path	Set the path; blank disables resolution

A refresh result is not an exception: a missing file, or the likely mistake of pointing at the client's own compressed Cliloc.enu, answers 200 with status: "unavailable" and a reason. A 500 would say only "something broke"; the operator needs to be told which file to convert. Setting the path deliberately does not import as a side effect — the response carries the refreshed status so the panel can offer that as the next step.