feat(cliloc): import the table from the shard, not from a file someone converted (Phase 2)
The base cliloc table now comes over the bridge. `clilocBridge.js` walks
`GET /cliloc` page by page and the model merges the `custom/` overlays over it —
overlays stay on disk because ServUO has no server-side notion of a custom
cliloc, so there is nothing on the shard to ask for.
**The shard wins whenever uo-link is configured and enabled**, with no mode
setting: there is no version of "which source?" an operator benefits from
answering. A file on disk remains the source only where there is no shard link,
plus a one-off explicit `path` — deprecated, not removed, and unchanged.
**Boot no longer imports on the bridge.** The file path could hash 5 MB locally
and skip in 14 ms; a shard round trip in the boot sequence would be spent
answering "no" on every restart but the one after a client patch — and patching a
client is an operator action, so importing became one. Admin → Shard → Import.
Whatever table is loaded keeps serving until then.
Three checks in the walk, each for a way a shard can hand back a table that looks
complete:
* only `cut: 'end'` finishes it — a short page can equally be a spent budget,
and a truncated table renders some items named and some not, which is exactly
what NO table looks like;
* the cursor must advance, or the walk stops rather than spinning;
* every page echoes the source's size and mtime, so a client patched mid-import
is refused outright rather than stitched from two files.
**The base is exempt from the vanished-source rule**, which is an upgrade detail
rather than a preference: an install that used the file pipeline carries its base
file's label in the stored fingerprint, and on the bridge that label is *supposed*
to disappear. Counting it as vanished would demand an approval for a change the
upgrade itself made. Overlays keep the rule in full.
**The protocol pin moves 7 → 8** — the third declaration site, and the one
nothing enforces. Phase 1 moved the sidecar and the overlay together because the
installer refuses a mismatched bundle; this one has to be moved by hand, in the
phase that first calls a protocol-8 route. The schema block above it is the
record of what forgetting costs: two phases of every REST call answered 409.
Verified against a live shard, sidecar and site: 12 pages, 67,496 rows imported
in 1.68 s, the operator's three-row overlay overriding stock strings on top of
it, and the next import correctly `unchanged`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
@@ -6,6 +6,24 @@
|
||||
// - the server, which refreshes the table on boot (`shardClilocs.model.js`)
|
||||
// - the admin panel, which can force a reimport without a restart
|
||||
//
|
||||
// ── What protocol 8 took away, and what it left ───────────────────────────
|
||||
//
|
||||
// The BASE table no longer comes from here on a shard that has uo-link
|
||||
// configured: `clilocBridge.js` asks the shard for it, because the shard has the
|
||||
// operator's client files already and, since phase 2, the decompressor to read
|
||||
// them (docs/link/v8.md §9). Nobody converts a file by hand any more.
|
||||
//
|
||||
// Two things keep this module alive rather than deleting it:
|
||||
//
|
||||
// - **Overlays.** Shard-added items carry cliloc ids no client table has, and
|
||||
// ServUO has no server-side notion of a custom cliloc — there is nothing on
|
||||
// the shard to ask for. `custom/` is still a directory the site reads, and
|
||||
// `readOverlays` below is the entry point the bridge path uses.
|
||||
// - **Installs with no shard link**, and development. A site that has never
|
||||
// configured uo-link can still be pointed at a converted file; that path is
|
||||
// deprecated, not removed, and it stays the whole of this module's base-table
|
||||
// behaviour.
|
||||
//
|
||||
// The files are the OPERATOR'S (see docs/website/CLILOCS.md). Nothing derived
|
||||
// from them is committed: the repo holds no string table, exactly as it holds no
|
||||
// map snapshot and no artwork. That rule is why this module reads a configured
|
||||
@@ -194,6 +212,63 @@ function readSources(configured) {
|
||||
return { root, files }
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the OVERLAY files only, with no base table.
|
||||
*
|
||||
* The bridge path needs exactly this: the base arrives from the shard and the
|
||||
* `custom/` directory beside the configured path still has to be merged over it.
|
||||
* `readSources` cannot answer it, because resolving a base is the first thing it
|
||||
* does and there may not be one — an operator on the bridge is entitled to point
|
||||
* this setting at a directory that holds nothing but `custom/`.
|
||||
*
|
||||
* **Never throws.** A path that is blank, missing or unreadable is reported as a
|
||||
* `problem` string and an empty file list, because none of those may stop a base
|
||||
* table that arrived perfectly well from being imported. The model decides what
|
||||
* to do about it — and it has a real decision to make, since an overlay that was
|
||||
* loaded last time and is missing now is the vanished-source hazard, not a
|
||||
* config typo.
|
||||
*/
|
||||
function readOverlays(configured) {
|
||||
const target = String(configured ?? '').trim()
|
||||
if (target === '') return { root: null, files: [], problem: null }
|
||||
|
||||
let root
|
||||
try {
|
||||
const stat = fs.statSync(target)
|
||||
root = stat.isFile() ? path.dirname(target) : target
|
||||
} catch {
|
||||
return { root: null, files: [], problem: `Cliloc path does not exist: ${target}` }
|
||||
}
|
||||
|
||||
let overlays
|
||||
try {
|
||||
overlays = listCustom(root)
|
||||
} catch (err) {
|
||||
return { root, files: [], problem: err.message }
|
||||
}
|
||||
|
||||
const files = []
|
||||
for (const file of overlays) {
|
||||
let buffer
|
||||
try {
|
||||
buffer = fs.readFileSync(file)
|
||||
} catch {
|
||||
return { root, files: [], problem: `Cliloc overlay is not readable: ${file}` }
|
||||
}
|
||||
files.push({
|
||||
label: path.relative(root, file).split(path.sep).join('/'),
|
||||
kind: 'custom',
|
||||
file,
|
||||
buffer,
|
||||
sha256: sha256(buffer),
|
||||
bytes: buffer.length,
|
||||
compressed: isCompressedCliloc(buffer),
|
||||
})
|
||||
}
|
||||
|
||||
return { root, files, problem: null }
|
||||
}
|
||||
|
||||
/**
|
||||
* A fingerprint of every source: `{ "<label>": "<sha256>" }`, plus the base's
|
||||
* details for the admin panel.
|
||||
@@ -244,6 +319,25 @@ function missingSources(current, loaded) {
|
||||
return Object.keys(loaded).filter((label) => !Object.hasOwn(current, label))
|
||||
}
|
||||
|
||||
/**
|
||||
* The same question asked of OVERLAYS only.
|
||||
*
|
||||
* Needed because the base table moved to the bridge. An install upgraded from the
|
||||
* file pipeline carries a base label (`clilocs.plain`, say) in its loaded
|
||||
* fingerprint, and that label is *supposed* to disappear when the base starts
|
||||
* arriving from the shard — reporting it as a vanished source would make every
|
||||
* first import after the upgrade demand an approval for a change the upgrade
|
||||
* itself made. Overlay labels are the ones whose absence is genuinely ambiguous,
|
||||
* and they are exactly the labels under `custom/`.
|
||||
*/
|
||||
function missingOverlays(current, loaded) {
|
||||
if (!loaded) return []
|
||||
const prefix = `${CUSTOM_DIR}/`
|
||||
return Object.keys(loaded).filter(
|
||||
(label) => label.startsWith(prefix) && !Object.hasOwn(current, label),
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Read and parse every source, merged into one entry list.
|
||||
*
|
||||
@@ -309,8 +403,10 @@ module.exports = {
|
||||
resolveBase,
|
||||
listCustom,
|
||||
readSources,
|
||||
readOverlays,
|
||||
hashSources,
|
||||
sameSources,
|
||||
missingSources,
|
||||
missingOverlays,
|
||||
readCliloc,
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user