docs/website/CLILOCS.md

# Cliloc table (item and title names)

**Status:** Complete on `edge` — website [#115](https://gitea.whitlocktech.com/RunicGateway/website/pulls/115), docs [#70](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/70).
**Design:** [`docs/link/v3.md` §8.6](../link/v3.md) — Protocol 3.0, the dependency Part B/3 was sequenced behind.

A "cliloc" is UO's localization table: an integer id mapped to a display string.
**Items on the wire carry a `LabelNumber`, not a name.** The bridge has always
sent that number — `char.profile.equipment` has a `cliloc` field, reward titles
arrive as a cliloc number in string form, and every marketplace listing carries
one — but the site had no table to look it up in, so a character sheet could only
render `id 1023721` where the game renders **"quarter staff"**.

The number was never the missing piece. The table was.

## Why the operator has to convert the file

This is the awkward part, and it is not avoidable:

**Every current UO client ships its cliloc files compressed.** All eight
`Cliloc.*` files in a modern client (`chs`, `cht`, `deu`, `enu`, `esp`, `fra`,
`jpn`, `kor`) begin with a DWORD whose high byte is `0x8E` — the "Mythic"
compressed container. The plain layout this site parses is what those files
looked like *before* that change.

Decompressing it means an inverse-BWT coder with a frequency header — a few
hundred lines of bit-level work whose failure mode is plausible-looking garbage
rather than an error. The site has no business carrying that at runtime.

Two facts make the alternatives worse, not better:

- **ServUO cannot read it either.** Its bundled `Ultima.StringList` implements
  only the plain layout, so on a modern client `VendorSearch.StringList` is null
  and `VendorSearch.GetItemName` returns `item.Name` — usually nothing. The
  shard cannot supply names on our behalf; the in-game Vendor Search gump has the
  same gap.
- **Nothing client-derived may be committed.** UO's strings are EA's. The repo
  ships no string table for the same reason it ships no artwork and no map
  snapshot — see [`SPAWN_ATLAS.md`](SPAWN_ATLAS.md).

So the conversion happens **once, on the operator's machine, against their own
client**, and the site reads the result from a path it is given. A shard that
never does this is in a fully supported state: names render as ids, exactly as
they did before the table existed.

## Converting

Either format below is accepted; the site sniffs which one it was handed.

| Format | Fidelity | Notes |
|---|---|---|
| **Plain binary** (recommended) | Exact | 6-byte header, then `{int32 number, byte flag, uint16 length, UTF-8}` records |
| Delimited text | Loses leading/trailing whitespace | `number<TAB\|,\|;>text` per line; a header row, blank lines and `#` comments are ignored |

The whitespace caveat is real but cosmetic: ~1,300 of the 123,490 entries in a
stock `Cliloc.enu` are label prefixes like `"max = "` whose trailing space is
meaningful when the client concatenates a value onto them. Nothing on this site
concatenates, and every consumer passes through `displayText()`, which trims.

### Using the bundled tool

`server/tools/cliloc-export/` is a small .NET console app that drives
[UOFiddler](https://github.com/polserver/UOFiddler)'s `Ultima.dll` — the
decompressor that already exists and is already maintained — and writes the plain
format. It loads that DLL **reflectively** so it compiles against any SDK, and it
writes the records by hand because UOFiddler's own `SaveStringList` *re-compresses*
on save (its purpose is round-tripping a file back into the client, so its output
is byte-identical to its input — a trap worth knowing about).

```bash
cd website/server/tools/cliloc-export
dotnet build -c Release

# binary (recommended)
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.plain

# or tab-delimited
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv
```

A UOFiddler GUI export works equally well — anything producing one of the two
shapes above is fine.

## Configuring the path

Two ways to point at the converted file, the setting winning over the
environment:

| Source | Notes |
|---|---|
| `cliloc_client_path` setting | Admin-editable (Admin → Shard); takes effect on the next refresh without a redeploy |
| `UO_CLIENT_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |

The value may be **the file itself or a directory to search**, because both are
natural answers to "where is it". A directory is searched case-insensitively (the
client writes `Cliloc.enu` on Windows; the site usually runs on Linux) for, in
order: `clilocs.tsv`, `clilocs.csv`, `cliloc.plain.enu`, `cliloc.enu.plain`,
`clilocs.txt`, `cliloc.enu`.

That ordering puts explicitly-converted names first on purpose. Pointing the
setting straight at an unconverted client directory finds `cliloc.enu`, which is
compressed — and the site says so by name rather than failing obscurely:

```
status: unavailable
code:   COMPRESSED
reason: This is a compressed (Mythic-format) cliloc file, which the site cannot
        read. Convert it to the plain format first — see docs/website/CLILOCS.md.
```

## Refresh contract

Identical in shape to the spawn atlas, and for the same reasons:

- **It never blocks startup.** No path, an unreadable file, a wrong-format file,
  a database error — all caught and logged. The site comes up either way.
- **Hash-gated.** The boot path hashes the file and skips the parse entirely when
  it matches what is loaded, which is every restart that did not follow a client
  patch. Measured on a stock table: **14 ms** for the no-op, **663 ms** for a full
  parse and replace.
- **A `PARSER_VERSION` bump also counts as drift**, so a corrected parse reaches
  an install whose client never patches.

### Why there is no staged-approval flow

The atlas stages a refresh that would *remove a facet*, because a half-copied
tree and a real map change are indistinguishable from inside the process. A
cliloc file is one file with one hash, and its realistic corruption — a partial
copy — makes the parser fail on a truncated record instead of yielding a
plausible-but-short table. **The ambiguity the atlas has to escalate to a human is
one this parser can simply detect**, so it refuses the import and leaves the
previous table serving. Verified: a file truncated to half its length reports

```
code:   TRUNCATED
reason: Truncated record header at byte 2486759 (74909 entries read)
```

and the 67,496 rows already loaded are untouched.

## What gets stored

| | |
|---|---|
| Parsed from a stock `Cliloc.enu` | **123,490** entries |
| Of those, empty strings | **55,994** (ids the client reserves and never uses) |
| Stored in `shard_clilocs` | **67,496** |

Blank entries are dropped at import. A row resolving to no name is
indistinguishable from no row at all to every caller, and dropping them makes the
binary and text imports converge on **identical** content — the binary format
carries the blanks explicitly and a text export may or may not, depending on the
tool. Verified: both formats import to the same 67,496 rows with the same keys.

`text` is `TEXT`, not `VARCHAR`: the long property descriptions reach 12 KB, and
silently truncating them would be worse than storing them. The index that matters
for marketplace search is on the denormalized `shard_vendor_items.display_name`,
not here.

## How names are applied

**Resolution happens server-side.** The table is never served *as* a table and
there is no public route for it. Two reasons: 67k rows would dwarf any page that
used them, and the Android client consumes the same JSON and would otherwise need
its own copy.

`resolveMany()` takes a batch of ids and returns a `Map` holding only those that
resolved to something displayable, so "no such id" and "id with no usable name"
collapse into one branch at the call site. It never throws — a cliloc lookup is
decoration on someone's character sheet, and a database blip must not fail the
sheet. A capped in-process cache fronts it; measured cold **4.2 ms**, warm
**0.015 ms**.

### `displayText()`

Cliloc strings interpolate arguments the client pulls from an item's property
list — `~1_val~`, `~2_NAME~`. **We never have those**: the bridge sends the id,
not the packet. So a name carrying them is reduced to what is actually knowable.

| Raw | Displayed |
|---|---|
| `quarter staff` | `quarter staff` |
| `cold damage ~1_val~%` | `cold damage` |
| `[~1_stuff~]` | *(nothing — the whole string was the argument)* |
| `50%` | `50%` |

The trailing `%` in row two is dropped **because a placeholder was removed** — it
is the unit belonging to the number we never had. Row four shows why that
condition matters: stripping `%` unconditionally would corrupt a string that
legitimately ends in one.

### Consumers

- **Character sheet equipment.** `enrichCharProfile` attaches `clilocName` to each
  item. A player-given `name` always wins — "Bob's lucky axe" must not be
  relabelled "hatchet" — and the client re-states that precedence.
- **Reward titles.** `titles.rewardResolved` is a parallel array with the numeric
  entries turned into words (`null` where nothing resolved). The sheet used to
  *skip* numeric reward titles entirely, having no way to render them.
- **Marketplace listings** (Protocol 3.0 §8) denormalize the resolved name into
  `shard_vendor_items.display_name` so search can index it.

## Admin surface

All admin-only, alongside the atlas under Admin → Shard:

| Route | Purpose |
|---|---|
| `GET /api/v1/admin/shard/clilocs` | Path, resolved file, readability, drift, entry count |
| `POST /api/v1/admin/shard/clilocs/import` | Reload after a client patch; `{ "force": true }` reimports an unchanged file |
| `PUT /api/v1/admin/shard/clilocs/path` | Set the path; blank disables resolution |

A refresh **result is not an exception**: a missing file, or the likely mistake of
pointing at the client's own compressed `Cliloc.enu`, answers `200` with
`status: "unavailable"` and a reason. A `500` would say only "something broke";
the operator needs to be told which file to convert. Setting the path
deliberately does **not** import as a side effect — the response carries the
refreshed status so the panel can offer that as the next step.