feat(cliloc): import the table from the shard, not from a file someone converted (Phase 2)
All checks were successful
PR Checks / client-build (pull_request) Successful in 34s
PR Checks / frozen-manifest (pull_request) Successful in 53s
PR Checks / server-tests (pull_request) Successful in 8m18s

The base cliloc table now comes over the bridge. `clilocBridge.js` walks
`GET /cliloc` page by page and the model merges the `custom/` overlays over it —
overlays stay on disk because ServUO has no server-side notion of a custom
cliloc, so there is nothing on the shard to ask for.

**The shard wins whenever uo-link is configured and enabled**, with no mode
setting: there is no version of "which source?" an operator benefits from
answering. A file on disk remains the source only where there is no shard link,
plus a one-off explicit `path` — deprecated, not removed, and unchanged.

**Boot no longer imports on the bridge.** The file path could hash 5 MB locally
and skip in 14 ms; a shard round trip in the boot sequence would be spent
answering "no" on every restart but the one after a client patch — and patching a
client is an operator action, so importing became one. Admin → Shard → Import.
Whatever table is loaded keeps serving until then.

Three checks in the walk, each for a way a shard can hand back a table that looks
complete:

  * only `cut: 'end'` finishes it — a short page can equally be a spent budget,
    and a truncated table renders some items named and some not, which is exactly
    what NO table looks like;
  * the cursor must advance, or the walk stops rather than spinning;
  * every page echoes the source's size and mtime, so a client patched mid-import
    is refused outright rather than stitched from two files.

**The base is exempt from the vanished-source rule**, which is an upgrade detail
rather than a preference: an install that used the file pipeline carries its base
file's label in the stored fingerprint, and on the bridge that label is *supposed*
to disappear. Counting it as vanished would demand an approval for a change the
upgrade itself made. Overlays keep the rule in full.

**The protocol pin moves 7 → 8** — the third declaration site, and the one
nothing enforces. Phase 1 moved the sidecar and the overlay together because the
installer refuses a mismatched bundle; this one has to be moved by hand, in the
phase that first calls a protocol-8 route. The schema block above it is the
record of what forgetting costs: two phases of every REST call answered 409.

Verified against a live shard, sidecar and site: 12 pages, 67,496 rows imported
in 1.68 s, the operator's three-row overlay overriding stock strings on top of
it, and the next import correctly `unchanged`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
2026-09-10 11:13:24 -05:00
parent c73d62e93a
commit 893a36618b
13 changed files with 1817 additions and 64 deletions

View File

@@ -307,10 +307,17 @@ shardRouter.put(
)
// ── Cliloc table (admin only) ─────────────────────────────────────────────
// UO's id → display-string map, converted once by the operator from their own
// client (docs/website/CLILOCS.md). Sits beside the atlas for the same reason:
// it is static content derived from operator-supplied files rather than anything
// the sidecar sends, and operating it is shard administration.
// UO's id → display-string map, read from the shard's own UO client over the
// bridge (docs/link/v8.md §9, docs/website/CLILOCS.md). Sits beside the atlas for
// the same reason: it is static content derived from the operator's own files
// rather than anything the sidecar streams, and operating it is shard
// administration.
//
// Protocol 8 changed where the base table comes from, not what these routes are:
// the shard decompresses `Cliloc.enu` and serves it paged, so an operator no
// longer converts anything by hand. Import stays an explicit admin action,
// because the only thing that changes a client's table is an operator patching
// their client.
//
// There is deliberately NO public counterpart. The table is never served as a
// table — 123k rows would dwarf any page that used it, and the Android client
@@ -320,7 +327,7 @@ shardRouter.get(
'/clilocs',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Cliloc table status: sources, drift, entry count (admin only)'
// #swagger.description = 'Where the cliloc sources are, whether they can be read, how many entries are loaded, and whether the files on disk have drifted from them. The table is built from a SET of sources — the converted client table plus every operator-maintained overlay under `custom/`, which is how shard-added and shard-edited items get names. `missingSources` lists any source that was loaded before and is now gone; an import refuses that without `approve`. A shard with nothing configured is a supported state — item names simply render as ids.'
// #swagger.description = 'Where the cliloc sources are, whether they can be read, how many entries are loaded, and whether they have drifted from what is loaded. `source` says which pipeline is in use: `bridge` (the shard reads its own client — the normal case once uo-link is configured) or `file` (a converted file on disk, deprecated, kept for installs with no shard link). On the bridge, `shard` carries the client files size, mtime, hash and the shards extractor version, and `shard.hashing: true` means a null hash is “not computed yet”, not “changed”. The table is always a SET: the base plus every operator-maintained overlay under `custom/`, which is how shard-added and shard-edited items get names. `missingSources` lists any overlay that was loaded before and is now gone; an import refuses that without `approve`. A shard with no source at all is a supported state — item names simply render as ids.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Cliloc status', content: { "application/json": { schema: { $ref: "#/components/schemas/UoClilocStatus" } } } } */
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
@@ -330,8 +337,8 @@ shardRouter.get(
shardRouter.post(
'/clilocs/import',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Re-import the cliloc table from its source files (admin only)'
// #swagger.description = 'Applies a client patch, or a change to the shards own overlay files, without a restart. `force` reimports even when the source hashes match what is loaded. `approve` accepts a refresh in which a previously-loaded source has VANISHED — refused by default, because an unmounted volume and a deliberate deletion are indistinguishable from the server, and the wrong guess silently drops every name that file contributed. A missing path — or the common mistake of pointing at the clients own COMPRESSED Cliloc.enu — answers 200 with status "unavailable" and the reason, rather than 500: the refresh contract reports outcomes instead of throwing, and the admin needs to be told which file to convert.'
// #swagger.summary = 'Re-import the cliloc table from its source (admin only)'
// #swagger.description = 'Applies a client patch, or a change to the shards own overlay files, without a restart. On the bridge this is the ONLY thing that imports — boot deliberately does not call the shard — so it is what an operator presses after patching their client. `force` reimports even when the sources are unchanged. `approve` accepts a refresh in which a previously-loaded overlay has VANISHED — refused by default, because an unmounted volume and a deliberate deletion are indistinguishable from the server, and the wrong guess silently drops every name that file contributed. Nothing here throws for an operator-visible problem: a shard that is down, an asset plane the operator has switched off, a client with no cliloc file, or a malformed overlay all answer 200 with status "unavailable" and a reason naming what to fix.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: false, content: { "application/json": { schema: { type: "object", properties: { force: { type: "boolean", description: "Reimport even if the sources are unchanged." }, approve: { type: "boolean", description: "Accept a refresh in which a previously-loaded source has vanished." } } } } } } */
/* #swagger.responses[200] = { description: 'What happened', content: { "application/json": { schema: { $ref: "#/components/schemas/UoClilocRefreshResult" } } } } */
@@ -344,10 +351,10 @@ shardRouter.post(
shardRouter.put(
'/clilocs/path',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Set the cliloc source the site reads from (admin only)'
// #swagger.description = 'Accepts either the converted base file itself or a directory to search. Overlays are read from a `custom/` directory beside it either way pointing at a file does not forfeit them. Persisted as a setting, which wins over the UO_CLIENT_PATH deploy default so the mount can move without a redeploy. Blank clears it and resolution is skipped on the next boot. Deliberately does not import as a side effect — the response carries the refreshed status so the panel can offer that as the next step.'
// #swagger.summary = 'Set the cliloc path the site reads overlays (and any file base) from (admin only)'
// #swagger.description = 'On an install with uo-link configured this selects only where `custom/` overlays are read from — the base table comes from the shard. Without a shard link it is also where the converted base file is looked for, which is the deprecated pre-protocol-8 pipeline. Accepts either a file or a directory to search; overlays are read from a `custom/` directory beside it either way, so pointing at a file does not forfeit them. Persisted as a setting, which wins over the UO_CLIENT_PATH deploy default so the mount can move without a redeploy. Blank clears it. Deliberately does not import as a side effect — the response carries the refreshed status so the panel can offer that as the next step.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["path"], properties: { path: { type: "string", description: "Path to the converted cliloc file, or a directory containing one. Blank disables resolution." } } } } } } */
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["path"], properties: { path: { type: "string", description: "Directory holding the custom/ overlays (and, with no shard link, a converted base file). Blank clears it." } } } } } } */
/* #swagger.responses[200] = { description: 'Cliloc status after the change', content: { "application/json": { schema: { $ref: "#/components/schemas/UoClilocStatus" } } } } */
adminOnly,
body('path').isString().isLength({ max: 512 }),