feat(shard): resolve cliloc names for items and reward titles
Protocol 3.0 §8.6 (docs/link/v3.md), the dependency order 5 was sequenced
behind. Items on the wire carry a LabelNumber, not a name — the bridge has
always sent it (char.profile.equipment.cliloc, reward titles as a cliloc
number in string form, and one per marketplace listing) but the site had no
table to resolve it against, so a character sheet could only render
`id 1023721` where the game renders "quarter staff".
The number was never the missing piece. The table was.
Sourced from a file the operator converts once from their own client, at a
path from the `cliloc_client_path` setting falling back to UO_CLIENT_PATH.
Nothing client-derived is committed: UO's strings are EA's, exactly as the
creature sprites are. A shard with nothing configured is fully supported —
names render as ids, as they did before.
The conversion step is not avoidable, and that is the substantive finding
here: every current client ships its cliloc files COMPRESSED (first DWORD's
high byte 0x8E, the Mythic container), and ServUO's own bundled
Ultima.StringList cannot read that either — so VendorSearch.GetItemName is
already inert on such a shard and the plugin could not supply names instead.
v3.md's original "read the client's Cliloc.enu" recommendation was therefore
not implementable as written, and its committed db/data/clilocs.json artifact
also predates the Part C corrections (no committed derived snapshots, nothing
EA-derived shipped). Replaced with the spawn-atlas pattern: parse on boot from
an operator-configured path, hash-gated, output gitignored.
- utils/clilocParse.js — pure parsers, fs-free so the suite runs in CI.
Accepts the plain binary layout and delimited text, sniffed by header rather
than extension. Rejects a compressed file BY NAME: without that check the
plain parser reads it as ~19k records of negative ids and 60 KB "strings"
before dying mid-file, and the resulting error names the wrong problem.
displayText() drops the ~1_val~ arguments the bridge never sends.
- utils/clilocSource.js — the fs layer. hashSource reports `compressed` so the
admin panel can flag an unconverted file WITHOUT parsing 5 MB per poll;
otherwise pointing at a client directory reports a healthy file with pending
drift ("ready to import") and the operator only finds out on failure.
- model/shardClilocs — refresh/status/lookup. All-or-nothing replace (DELETE,
not TRUNCATE — TRUNCATE is DDL in MariaDB and implicitly commits). Batched
server-side resolution behind a capped cache; never throws, because a cliloc
lookup is decoration on a character sheet.
- Deliberately NO staged-approval flow, unlike the atlas: the atlas escalates
facet loss because a half-copied tree and a real map change are
indistinguishable from inside the process, whereas a partial cliloc copy
makes the parser fail on a truncated record. The ambiguity the atlas must
escalate is one this parser simply detects.
- No public route. The table is never served AS a table: 67k rows would dwarf
any page using them, and the Android client consumes the same resolved JSON.
Two parser bugs found by building it, both now covered by tests: trimming a
text line before splitting ate the trailing separator on empty-text entries
and silently dropped 55,994 of 123,490 while still reporting success; and
Number('') is 0, not NaN, so a line starting with a separator imported as a
bogus cliloc 0.
Verified against the real client table (123,490 entries) and the live MariaDB:
import 663 ms, hash-gated boot no-op 14 ms, cold resolve 4.2 ms / warm 0.015 ms.
Binary and TSV imports converge on the same 67,496 rows with identical keys
(blank entries — half the table — are dropped at import). A file truncated to
half its length is refused with TRUNCATED and leaves the previous table
serving. Boot logs verified for both the import and the compressed-file
warning; neither blocks startup. All three admin routes exercised over HTTP
with a real session. 629 server tests pass; client builds clean; swagger,
routes.manifest.json and routes.guards.json regenerated.
Not covered by an automated test: the character sheet renders resolved names
in presentational React with no DOM test harness in this repo, and was not
rendered against a live linked-player profile — that needs a logged-in player
with a linked game account and a shard answering a profile RPC.
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -25,6 +25,7 @@ const { body, param } = require('express-validator')
|
||||
const shardOps = require('./shardOps.controller')
|
||||
const shardVisibility = require('./shardVisibility.controller')
|
||||
const shardAtlas = require('./shardAtlas.controller')
|
||||
const shardClilocs = require('./shardClilocs.controller')
|
||||
const selfShard = require('../player/shard.controller')
|
||||
const { requireRole } = require('../../../utils/auth')
|
||||
const validate = require('../../../middleware/validate')
|
||||
@@ -304,6 +305,54 @@ shardRouter.put(
|
||||
shardAtlas.setPath,
|
||||
)
|
||||
|
||||
// ── Cliloc table (admin only) ─────────────────────────────────────────────
|
||||
// UO's id → display-string map, converted once by the operator from their own
|
||||
// client (docs/website/CLILOCS.md). Sits beside the atlas for the same reason:
|
||||
// it is static content derived from operator-supplied files rather than anything
|
||||
// the sidecar sends, and operating it is shard administration.
|
||||
//
|
||||
// There is deliberately NO public counterpart. The table is never served as a
|
||||
// table — 123k rows would dwarf any page that used it, and the Android client
|
||||
// consumes the same already-resolved JSON. Names are applied server-side to the
|
||||
// responses that need them.
|
||||
shardRouter.get(
|
||||
'/clilocs',
|
||||
// #swagger.tags = ['Admin · Shard']
|
||||
// #swagger.summary = 'Cliloc table status: path, drift, entry count (admin only)'
|
||||
// #swagger.description = 'Where the converted cliloc file is, whether it can be read, how many entries are loaded, and whether the file on disk has drifted from them. A shard with no cliloc file configured is a supported state — item names simply render as ids.'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.responses[200] = { description: 'Cliloc status', content: { "application/json": { schema: { $ref: "#/components/schemas/ClilocStatus" } } } } */
|
||||
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
|
||||
adminOnly,
|
||||
shardClilocs.getStatus,
|
||||
)
|
||||
shardRouter.post(
|
||||
'/clilocs/import',
|
||||
// #swagger.tags = ['Admin · Shard']
|
||||
// #swagger.summary = 'Re-import the cliloc table from the converted file (admin only)'
|
||||
// #swagger.description = 'Applies a client patch without a restart. `force` reimports even when the source hash matches what is loaded. A missing file — or the common mistake of pointing at the client\'s own COMPRESSED Cliloc.enu — answers 200 with status "unavailable" and the reason, rather than 500: the refresh contract reports outcomes instead of throwing, and the admin needs to be told which file to convert.'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.requestBody = { required: false, content: { "application/json": { schema: { type: "object", properties: { force: { type: "boolean", description: "Reimport even if the file is unchanged." } } } } } } */
|
||||
/* #swagger.responses[200] = { description: 'What happened', content: { "application/json": { schema: { $ref: "#/components/schemas/ClilocRefreshResult" } } } } */
|
||||
adminOnly,
|
||||
body('force').optional().isBoolean(),
|
||||
validate,
|
||||
shardClilocs.importClilocs,
|
||||
)
|
||||
shardRouter.put(
|
||||
'/clilocs/path',
|
||||
// #swagger.tags = ['Admin · Shard']
|
||||
// #swagger.summary = 'Set the cliloc file the site reads from (admin only)'
|
||||
// #swagger.description = 'Accepts either the converted file itself or a directory to search. Persisted as a setting, which wins over the UO_CLIENT_PATH deploy default so the mount can move without a redeploy. Blank clears it and resolution is skipped on the next boot. Deliberately does not import as a side effect — the response carries the refreshed status so the panel can offer that as the next step.'
|
||||
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
|
||||
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["path"], properties: { path: { type: "string", description: "Path to the converted cliloc file, or a directory containing one. Blank disables resolution." } } } } } } */
|
||||
/* #swagger.responses[200] = { description: 'Cliloc status after the change', content: { "application/json": { schema: { $ref: "#/components/schemas/ClilocStatus" } } } } */
|
||||
adminOnly,
|
||||
body('path').isString().isLength({ max: 512 }),
|
||||
validate,
|
||||
shardClilocs.setPath,
|
||||
)
|
||||
|
||||
// ── Feature visibility (admin only) ───────────────────────────────────
|
||||
// Who can see which shard surface, and which sensitive fields within it. This
|
||||
// decides what ANONYMOUS visitors get, so it sits above the moderator tier.
|
||||
|
||||
78
server/src/router/v1/admin/shardClilocs.controller.js
Normal file
78
server/src/router/v1/admin/shardClilocs.controller.js
Normal file
@@ -0,0 +1,78 @@
|
||||
// ── Admin · Cliloc table ───────────────────────────────────────────────────
|
||||
//
|
||||
// Operating the cliloc import: where the converted cliloc file is, whether it
|
||||
// has drifted from what is loaded, and a forced reimport after a client patch
|
||||
// (docs/website/CLILOCS.md).
|
||||
//
|
||||
// The policy lives in the model. This controller does three things and no more:
|
||||
// it validates input, it maps a refresh RESULT onto an HTTP status, and it
|
||||
// records the action in the admin activity log.
|
||||
//
|
||||
// **A refresh result is not an exception.** `shardClilocs.refresh()` reports
|
||||
// `unavailable` / `failed` rather than throwing, because the boot path must never
|
||||
// be stopped by a bad file. That contract is preserved here: a missing file, or
|
||||
// the single most likely operator mistake — pointing at the client's own
|
||||
// COMPRESSED `Cliloc.enu` — is a 200 carrying `status: 'unavailable'` and the
|
||||
// reason, not a 500. A 500 would say only "something broke"; the operator needs
|
||||
// to be told which file to convert.
|
||||
|
||||
const clilocs = require('../../../model/shardClilocs/shardClilocs.model')
|
||||
const activity = require('../../../model/activity/activity.model')
|
||||
|
||||
const log = require('../../../utils/logger')('admin-shard-clilocs')
|
||||
|
||||
// GET /admin/shard/clilocs — what is loaded, what the file looks like, whether
|
||||
// they disagree. There is no public counterpart: the cliloc table is never
|
||||
// served as a table, only applied to names the site already returns.
|
||||
async function getStatus(req, res) {
|
||||
try {
|
||||
return res.json(await clilocs.status())
|
||||
} catch (err) {
|
||||
log.error('getStatus', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// POST /admin/shard/clilocs/import — reload after a client patch without a
|
||||
// restart. `force` reimports even when the source hash matches what is loaded
|
||||
// (the escape hatch for "the database is wrong but the file is not").
|
||||
async function importClilocs(req, res) {
|
||||
try {
|
||||
const force = !!req.body?.force
|
||||
const result = await clilocs.refresh({ force })
|
||||
await activity.log({
|
||||
req,
|
||||
action: 'shard.clilocs.import',
|
||||
detail: { force, status: result.status, count: result.count ?? null },
|
||||
})
|
||||
return res.json(result)
|
||||
} catch (err) {
|
||||
log.error('importClilocs', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
// PUT /admin/shard/clilocs/path — point the site at a different cliloc file.
|
||||
//
|
||||
// Persisted as a setting, which wins over the UO_CLIENT_PATH env default so an
|
||||
// operator can move the mount without a redeploy. Blank clears it, which turns
|
||||
// resolution off (boot skips, the loaded table keeps serving) — a legitimate
|
||||
// thing to want, so it is allowed rather than validated away.
|
||||
//
|
||||
// Deliberately does NOT import as a side effect, for the same reason the atlas
|
||||
// path does not: changing where the table reads from and reloading it are
|
||||
// separate decisions. The response carries the refreshed status so the panel can
|
||||
// offer the import immediately.
|
||||
async function setPath(req, res) {
|
||||
try {
|
||||
const value = String(req.body?.path ?? '').trim()
|
||||
await clilocs.setClientPath(value, req.user?.id ?? null)
|
||||
await activity.log({ req, action: 'shard.clilocs.path', detail: { path: value } })
|
||||
return res.json(await clilocs.status())
|
||||
} catch (err) {
|
||||
log.error('setClilocPath', err)
|
||||
return res.status(500).json({ message: 'Internal Server Error' })
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { getStatus, importClilocs, setPath }
|
||||
@@ -10,6 +10,7 @@
|
||||
const uoLinkClient = require('../../../utils/uoLinkClient')
|
||||
const shardLinks = require('../../../model/shardLinks/shardLinks.model')
|
||||
const shardState = require('../../../model/shardState/shardState.model')
|
||||
const shardClilocs = require('../../../model/shardClilocs/shardClilocs.model')
|
||||
const settings = require('../../../model/settings/settings.model')
|
||||
const { salesForAccounts } = require('../../../utils/shardSales')
|
||||
const activity = require('../../../model/activity/activity.model')
|
||||
@@ -18,9 +19,62 @@ const log = require('../../../utils/logger')('player-shard')
|
||||
|
||||
const SERIAL_RE = /^0x[0-9a-fA-F]+$/
|
||||
|
||||
/**
|
||||
* Resolve the cliloc ids on a profile into display names.
|
||||
*
|
||||
* Items on the wire carry a `LabelNumber`, not a name — `BridgeProfile.WriteItem`
|
||||
* sends `cliloc` on every equipment entry and `name` only for the minority of
|
||||
* items a player has renamed. Reward titles are the same shape: the shard sends
|
||||
* a cliloc number as a string, which the sheet previously had to SKIP because it
|
||||
* had no way to turn it into words.
|
||||
*
|
||||
* Resolution happens here rather than in the browser because the table is ~123k
|
||||
* rows: shipping it to render a dozen names would dwarf the page, and the
|
||||
* Android client consumes this same JSON and would otherwise need its own copy.
|
||||
*
|
||||
* A shard with no cliloc table configured resolves nothing and the sheet renders
|
||||
* ids exactly as it did before — this is decoration, and it is applied in the
|
||||
* same best-effort block as the guild/governor cross-links.
|
||||
*/
|
||||
async function resolveProfileClilocs(profile) {
|
||||
const wanted = []
|
||||
|
||||
const equipment = Array.isArray(profile.equipment) ? profile.equipment : []
|
||||
for (const item of equipment) {
|
||||
if (Number.isInteger(item?.cliloc)) wanted.push(item.cliloc)
|
||||
}
|
||||
|
||||
// Reward titles arrive as strings that may be either a literal ("Knight of
|
||||
// Trinsic") or a cliloc number in string form. Only the numeric ones need us.
|
||||
const reward = Array.isArray(profile.titles?.reward) ? profile.titles.reward : []
|
||||
const rewardNumbers = reward.map((r) => (/^\d+$/.test(String(r)) ? Number(r) : null))
|
||||
for (const n of rewardNumbers) if (n !== null) wanted.push(n)
|
||||
|
||||
if (wanted.length === 0) return
|
||||
|
||||
const names = await shardClilocs.resolveMany(wanted)
|
||||
if (names.size === 0) return
|
||||
|
||||
for (const item of equipment) {
|
||||
// A player-given name always wins over the type name: an item called "Bob's
|
||||
// lucky axe" should not be relabelled "hatchet".
|
||||
if (item?.name) continue
|
||||
const resolved = names.get(item?.cliloc)
|
||||
if (resolved) item.clilocName = resolved
|
||||
}
|
||||
|
||||
if (rewardNumbers.some((n) => n !== null)) {
|
||||
profile.titles.rewardResolved = reward.map((raw, i) => {
|
||||
const n = rewardNumbers[i]
|
||||
return n === null ? String(raw) : names.get(n) ?? null
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Decorate a char.profile with cross-links from our own board data: the guild the
|
||||
// character leads and any city governorship on its account. Best-effort — a
|
||||
// failure here never fails the profile (it's a nicety, not the sheet).
|
||||
// character leads and any city governorship on its account, plus resolved cliloc
|
||||
// names. Best-effort — a failure here never fails the profile (it's a nicety,
|
||||
// not the sheet).
|
||||
async function enrichCharProfile(profile) {
|
||||
if (!profile) return profile
|
||||
try {
|
||||
@@ -30,6 +84,7 @@ async function enrichCharProfile(profile) {
|
||||
const govs = await shardState.listGovernorshipsForAccounts([profile.acct])
|
||||
if (govs.length) profile.governorOf = govs.map((g) => g.city)
|
||||
}
|
||||
await resolveProfileClilocs(profile)
|
||||
} catch (err) {
|
||||
log.warn('enrichCharProfile failed', { serial: profile.serial, message: err.message })
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user