feat(cliloc): import the table from the shard, not from a file someone converted (Phase 2)
All checks were successful
PR Checks / client-build (pull_request) Successful in 34s
PR Checks / frozen-manifest (pull_request) Successful in 53s
PR Checks / server-tests (pull_request) Successful in 8m18s

The base cliloc table now comes over the bridge. `clilocBridge.js` walks
`GET /cliloc` page by page and the model merges the `custom/` overlays over it —
overlays stay on disk because ServUO has no server-side notion of a custom
cliloc, so there is nothing on the shard to ask for.

**The shard wins whenever uo-link is configured and enabled**, with no mode
setting: there is no version of "which source?" an operator benefits from
answering. A file on disk remains the source only where there is no shard link,
plus a one-off explicit `path` — deprecated, not removed, and unchanged.

**Boot no longer imports on the bridge.** The file path could hash 5 MB locally
and skip in 14 ms; a shard round trip in the boot sequence would be spent
answering "no" on every restart but the one after a client patch — and patching a
client is an operator action, so importing became one. Admin → Shard → Import.
Whatever table is loaded keeps serving until then.

Three checks in the walk, each for a way a shard can hand back a table that looks
complete:

  * only `cut: 'end'` finishes it — a short page can equally be a spent budget,
    and a truncated table renders some items named and some not, which is exactly
    what NO table looks like;
  * the cursor must advance, or the walk stops rather than spinning;
  * every page echoes the source's size and mtime, so a client patched mid-import
    is refused outright rather than stitched from two files.

**The base is exempt from the vanished-source rule**, which is an upgrade detail
rather than a preference: an install that used the file pipeline carries its base
file's label in the stored fingerprint, and on the bridge that label is *supposed*
to disappear. Counting it as vanished would demand an approval for a change the
upgrade itself made. Overlays keep the rule in full.

**The protocol pin moves 7 → 8** — the third declaration site, and the one
nothing enforces. Phase 1 moved the sidecar and the overlay together because the
installer refuses a mismatched bundle; this one has to be moved by hand, in the
phase that first calls a protocol-8 route. The schema block above it is the
record of what forgetting costs: two phases of every REST call answered 409.

Verified against a live shard, sidecar and site: 12 pages, 67,496 rows imported
in 1.68 s, the operator's three-row overlay overriding stock strings on top of
it, and the next import correctly `unchanged`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
2026-09-10 11:13:24 -05:00
parent c73d62e93a
commit 893a36618b
13 changed files with 1817 additions and 64 deletions

View File

@@ -6,6 +6,24 @@
// - the server, which refreshes the table on boot (`shardClilocs.model.js`)
// - the admin panel, which can force a reimport without a restart
//
// ── What protocol 8 took away, and what it left ───────────────────────────
//
// The BASE table no longer comes from here on a shard that has uo-link
// configured: `clilocBridge.js` asks the shard for it, because the shard has the
// operator's client files already and, since phase 2, the decompressor to read
// them (docs/link/v8.md §9). Nobody converts a file by hand any more.
//
// Two things keep this module alive rather than deleting it:
//
// - **Overlays.** Shard-added items carry cliloc ids no client table has, and
// ServUO has no server-side notion of a custom cliloc — there is nothing on
// the shard to ask for. `custom/` is still a directory the site reads, and
// `readOverlays` below is the entry point the bridge path uses.
// - **Installs with no shard link**, and development. A site that has never
// configured uo-link can still be pointed at a converted file; that path is
// deprecated, not removed, and it stays the whole of this module's base-table
// behaviour.
//
// The files are the OPERATOR'S (see docs/website/CLILOCS.md). Nothing derived
// from them is committed: the repo holds no string table, exactly as it holds no
// map snapshot and no artwork. That rule is why this module reads a configured
@@ -194,6 +212,63 @@ function readSources(configured) {
return { root, files }
}
/**
* Read the OVERLAY files only, with no base table.
*
* The bridge path needs exactly this: the base arrives from the shard and the
* `custom/` directory beside the configured path still has to be merged over it.
* `readSources` cannot answer it, because resolving a base is the first thing it
* does and there may not be one — an operator on the bridge is entitled to point
* this setting at a directory that holds nothing but `custom/`.
*
* **Never throws.** A path that is blank, missing or unreadable is reported as a
* `problem` string and an empty file list, because none of those may stop a base
* table that arrived perfectly well from being imported. The model decides what
* to do about it — and it has a real decision to make, since an overlay that was
* loaded last time and is missing now is the vanished-source hazard, not a
* config typo.
*/
function readOverlays(configured) {
const target = String(configured ?? '').trim()
if (target === '') return { root: null, files: [], problem: null }
let root
try {
const stat = fs.statSync(target)
root = stat.isFile() ? path.dirname(target) : target
} catch {
return { root: null, files: [], problem: `Cliloc path does not exist: ${target}` }
}
let overlays
try {
overlays = listCustom(root)
} catch (err) {
return { root, files: [], problem: err.message }
}
const files = []
for (const file of overlays) {
let buffer
try {
buffer = fs.readFileSync(file)
} catch {
return { root, files: [], problem: `Cliloc overlay is not readable: ${file}` }
}
files.push({
label: path.relative(root, file).split(path.sep).join('/'),
kind: 'custom',
file,
buffer,
sha256: sha256(buffer),
bytes: buffer.length,
compressed: isCompressedCliloc(buffer),
})
}
return { root, files, problem: null }
}
/**
* A fingerprint of every source: `{ "<label>": "<sha256>" }`, plus the base's
* details for the admin panel.
@@ -244,6 +319,25 @@ function missingSources(current, loaded) {
return Object.keys(loaded).filter((label) => !Object.hasOwn(current, label))
}
/**
* The same question asked of OVERLAYS only.
*
* Needed because the base table moved to the bridge. An install upgraded from the
* file pipeline carries a base label (`clilocs.plain`, say) in its loaded
* fingerprint, and that label is *supposed* to disappear when the base starts
* arriving from the shard — reporting it as a vanished source would make every
* first import after the upgrade demand an approval for a change the upgrade
* itself made. Overlay labels are the ones whose absence is genuinely ambiguous,
* and they are exactly the labels under `custom/`.
*/
function missingOverlays(current, loaded) {
if (!loaded) return []
const prefix = `${CUSTOM_DIR}/`
return Object.keys(loaded).filter(
(label) => label.startsWith(prefix) && !Object.hasOwn(current, label),
)
}
/**
* Read and parse every source, merged into one entry list.
*
@@ -309,8 +403,10 @@ module.exports = {
resolveBase,
listCustom,
readSources,
readOverlays,
hashSources,
sameSources,
missingSources,
missingOverlays,
readCliloc,
}