feat(cliloc): import the table from the shard, not from a file someone converted (Phase 2)
All checks were successful
PR Checks / client-build (pull_request) Successful in 34s
PR Checks / frozen-manifest (pull_request) Successful in 53s
PR Checks / server-tests (pull_request) Successful in 8m18s

The base cliloc table now comes over the bridge. `clilocBridge.js` walks
`GET /cliloc` page by page and the model merges the `custom/` overlays over it —
overlays stay on disk because ServUO has no server-side notion of a custom
cliloc, so there is nothing on the shard to ask for.

**The shard wins whenever uo-link is configured and enabled**, with no mode
setting: there is no version of "which source?" an operator benefits from
answering. A file on disk remains the source only where there is no shard link,
plus a one-off explicit `path` — deprecated, not removed, and unchanged.

**Boot no longer imports on the bridge.** The file path could hash 5 MB locally
and skip in 14 ms; a shard round trip in the boot sequence would be spent
answering "no" on every restart but the one after a client patch — and patching a
client is an operator action, so importing became one. Admin → Shard → Import.
Whatever table is loaded keeps serving until then.

Three checks in the walk, each for a way a shard can hand back a table that looks
complete:

  * only `cut: 'end'` finishes it — a short page can equally be a spent budget,
    and a truncated table renders some items named and some not, which is exactly
    what NO table looks like;
  * the cursor must advance, or the walk stops rather than spinning;
  * every page echoes the source's size and mtime, so a client patched mid-import
    is refused outright rather than stitched from two files.

**The base is exempt from the vanished-source rule**, which is an upgrade detail
rather than a preference: an install that used the file pipeline carries its base
file's label in the stored fingerprint, and on the bridge that label is *supposed*
to disappear. Counting it as vanished would demand an approval for a change the
upgrade itself made. Overlays keep the rule in full.

**The protocol pin moves 7 → 8** — the third declaration site, and the one
nothing enforces. Phase 1 moved the sidecar and the overlay together because the
installer refuses a mismatched bundle; this one has to be moved by hand, in the
phase that first calls a protocol-8 route. The schema block above it is the
record of what forgetting costs: two phases of every REST call answered 409.

Verified against a live shard, sidecar and site: 12 pages, 67,496 rows imported
in 1.68 s, the operator's three-row overlay overriding stock strings on top of
it, and the next import correctly `unchanged`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
2026-09-10 11:13:24 -05:00
parent c73d62e93a
commit 893a36618b
13 changed files with 1817 additions and 64 deletions

View File

@@ -490,21 +490,40 @@ module.exports = {
UoClilocStatus: {
type: 'object',
description:
'Admin view of cliloc state: where the converted file is, whether it is readable, how many entries are loaded, and whether the file has drifted from them. `configured: false` is a supported state — item names then render as ids.',
'Admin view of cliloc state: which source the base table comes from, whether it can be read, how many entries are loaded, and whether anything has drifted from them. Nothing configured at all is a supported state — item names then render as ids.',
properties: {
source: {
type: 'string',
enum: ['bridge', 'file'],
description: '`bridge`: the shard reads its own UO client (protocol 8, the normal case). `file`: a converted file on disk — the pre-protocol-8 pipeline, deprecated, kept for installs with no shard link.',
example: 'bridge',
},
configured: { type: 'boolean', example: true },
path: { type: 'string', example: '/srv/uo-client' },
file: { type: 'string', nullable: true, description: 'The file actually resolved, when the path is a directory.', example: '/srv/uo-client/clilocs.tsv' },
path: { type: 'string', description: 'On the bridge: where `custom/` overlays are read from. On a file source: the base path too.', example: '/srv/uo-client' },
file: { type: 'string', nullable: true, description: 'The base file in use — the shards own `cliloc.enu` on the bridge, the resolved local file otherwise.', example: 'cliloc.enu' },
fileReadable: { type: 'boolean', example: true },
problem: { type: 'string', nullable: true, description: 'Why the file cannot be used, when it cannot. Set (with code COMPRESSED) for a readable-but-unconverted client file.', example: null },
code: { type: 'string', nullable: true, description: 'Machine-readable cause of `problem`.', enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED'] },
problem: { type: 'string', nullable: true, description: 'Why the base cannot be used, when it cannot: a shard that is down or has assets switched off, or (on a file source) a missing or still-compressed file.', example: null },
code: { type: 'string', nullable: true, description: 'Machine-readable cause of `problem`.', enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED', 'DISABLED', 'NO_SOURCE', 'SHARD_DOWN', 'PROTOCOL', 'BUSY', 'UNAVAILABLE'] },
shard: {
type: 'object',
nullable: true,
description: 'Present on the bridge: the shards own cliloc file as it is right now. `hashing: true` with a null `sha256` means the hash has not been computed yet — “ask again”, not “changed”.',
properties: {
size: { type: 'integer', example: 4989921 },
mtime: { type: 'integer', description: 'Unix milliseconds.', example: 1757462400000 },
sha256: { type: 'string', nullable: true },
extractorVersion: { type: 'integer', description: 'The version of the shards extraction code. A bump makes everything derived from it drift.', example: 1 },
hashing: { type: 'boolean', example: false },
complete: { type: 'boolean', description: 'Every client file has a hash.', example: true },
},
},
drift: { type: 'boolean', nullable: true, description: 'True when any source hash differs from the loaded table. NULL when the sources could not be read or are not usable.', example: false },
count: { type: 'integer', description: 'Entries currently loaded.', example: 67496 },
sources: {
type: 'array',
items: { type: 'string' },
description: 'Every source found now, root-relative, base first then overlays in merge order.',
example: ['clilocs.plain', 'custom/uomysticmoon.tsv'],
example: ['custom/uomysticmoon.tsv'],
},
loadedSources: {
type: 'array',
@@ -514,7 +533,7 @@ module.exports = {
type: 'object',
properties: {
label: { type: 'string', example: 'custom/uomysticmoon.tsv' },
kind: { type: 'string', enum: ['base', 'custom'], example: 'custom' },
kind: { type: 'string', enum: ['shard', 'base', 'custom'], description: '`shard` is the table read over the bridge; `base` a converted file on disk.', example: 'custom' },
entries: { type: 'integer', example: 37 },
added: { type: 'integer', description: 'Ids this source introduced.', example: 25 },
overrode: { type: 'integer', description: 'Ids it replaced from an earlier source.', example: 12 },
@@ -543,17 +562,22 @@ module.exports = {
example: 'imported',
},
reason: { type: 'string', nullable: true },
source: { type: 'string', nullable: true, enum: ['bridge', 'file'], description: 'Which source this refresh read.', example: 'bridge' },
code: {
type: 'string',
nullable: true,
description: 'Machine-readable cause. `COMPRESSED` means the client\'s own Cliloc.enu was supplied instead of a converted one.',
enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED', 'TRUNCATED', 'EMPTY', 'NOT_BUFFER'],
description: 'Machine-readable cause. Bridge codes describe the shard (`DISABLED`: the operator switched the asset plane off; `NO_SOURCE`: its client has no cliloc file; `SHARD_DOWN`; `SOURCE_CHANGED`: the client was patched mid-import, so nothing was applied). File codes describe the path — `COMPRESSED` means the client\'s own Cliloc.enu was supplied instead of a converted one.',
enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED', 'TRUNCATED', 'EMPTY', 'NOT_BUFFER', 'DISABLED', 'NO_SOURCE', 'SHARD_DOWN', 'PROTOCOL', 'BUSY', 'UNAVAILABLE', 'SOURCE_CHANGED', 'INCOMPLETE', 'STUCK', 'MALFORMED', 'TOO_LARGE'],
},
path: { type: 'string', nullable: true },
file: { type: 'string', nullable: true },
count: { type: 'integer', nullable: true, description: 'Entries stored (blank strings are dropped).', example: 67496 },
parsed: { type: 'integer', nullable: true, description: 'Entries read across every source before blanks were dropped.', example: 123527 },
blank: { type: 'integer', nullable: true, example: 55994 },
blank: { type: 'integer', nullable: true, example: 0 },
pages: { type: 'integer', nullable: true, description: 'Bridge only: how many pages the table arrived in (a stock English table is about eleven).', example: 11 },
reported: { type: 'integer', nullable: true, description: 'Bridge only: how many rows the shard said it holds.', example: 67496 },
received: { type: 'integer', nullable: true, description: 'Bridge only: how many arrived. Disagreeing with `reported` means the walk is wrong.', example: 67496 },
overlayProblem: { type: 'string', nullable: true, description: 'The base imported, but the overlay directory could not be read. Reported rather than fatal.' },
sources: {
type: 'array',
nullable: true,
@@ -562,7 +586,7 @@ module.exports = {
type: 'object',
properties: {
label: { type: 'string' },
kind: { type: 'string', enum: ['base', 'custom'] },
kind: { type: 'string', enum: ['shard', 'base', 'custom'] },
entries: { type: 'integer' },
added: { type: 'integer' },
overrode: { type: 'integer' },