Shards edit items and add new ones, and those carry cliloc ids no stock client
table has. Reading exactly one converted file meant an operator had to
re-export 5 MB every time they added one item — friction enough that the table
would simply go stale, which is the failure the spawn atlas was redesigned to
avoid in the first place.
So this mirrors spawnAtlasSource.readSources(): a BASE (the converted client
table) plus every operator-maintained overlay under `custom/`, all re-read on
every boot and hash-gated as a SET. Later sources win, so an overlay both adds
ids the client never had and overrides stock ones the shard re-purposed.
Adding, editing or removing any overlay counts as drift.
`custom/` is the one convention here that is ours rather than the shard's, and
deliberately so: ServUO has no server-side notion of a custom cliloc — they
live in the patched client a shard distributes, and nothing in the tree
declares them. There is nothing to discover. (An operator who does patch their
client cliloc needs no overlay: convert the patched file and the edits are in
the base.) Scale, measured on the live shard: its script tree references 16,434
cliloc ids and only 37 are absent from stock — tens against a 67k base, which
is why this is an overlay and not a second table.
The set brings back a hazard a single file did not have, and it gets the
atlas's answer. A corrupt source fails the parse loudly, but a source that has
VANISHED parses perfectly and imports a table quietly missing everything it
contributed — an unmounted volume is indistinguishable from a deliberate
deletion. So it is staged, not applied (`needsReview`), reported by both the
import and status(), and accepted with `{approve:true}`. That is a flag rather
than the atlas's approve/reject pair because the atlas stores a pending
decision SO THAT approving re-parses; here nothing is stored, so re-reading at
approval time is automatic.
Also reports a per-source breakdown (entries/added/overrode) on import and in
status, which is how an operator confirms an overlay took effect — "overrode: 0"
on a file meant to re-label stock items says it did not.
Two bugs this surfaced, both found by running a shard-style overlay rather than
by another stock-table fixture:
- displayText tidied punctuation unconditionally, so a custom
"Runic Gateway Sigil (v2)" rendered as "(v2". Stripping leftover brackets is
right after a placeholder is removed and wrong otherwise — the same condition
the `%` rule already had.
- CANDIDATE_NAMES did not include `clilocs.plain`, which is the exact filename
CLILOCS.md and the export tool's README tell operators to write. Pointing at
the directory they were told to create failed with NO_FILE.
Verified end to end against the live MariaDB and a real server boot: base-only
import, overlay adding one id and overriding another (per-source breakdown
correct), unchanged set as a no-op, an edited overlay re-importing and
withdrawing its override, a vanished overlay refused with the table intact,
status reporting missingSources, approve applying it, and a file-path
configuration still finding overlays beside it. All three resolve correctly
through the running server: shard-added, overridden and stock. 646 server tests
pass (16 new in clilocSource.test.js, 3 new in clilocParse.test.js); swagger,
routes.manifest.json and routes.guards.json regenerated.
Co-Authored-By: Claude <noreply@anthropic.com>
109 lines
4.2 KiB
JavaScript
109 lines
4.2 KiB
JavaScript
const { pool, query } = require('../../utils/db')
|
|
|
|
// Raw SQL for the cliloc table. `shard_clilocs` is IMPORT-OWNED: `replaceAll`
|
|
// empties and refills it inside one transaction, and nothing else in the
|
|
// codebase writes to it. No foreign keys, consistent with every other shard_*
|
|
// table.
|
|
|
|
const BATCH = 1000
|
|
|
|
/**
|
|
* Replace the entire cliloc table in one transaction.
|
|
*
|
|
* All-or-nothing on purpose: a failed reload must leave the previous table
|
|
* intact rather than a half-loaded one, because a partially-imported cliloc
|
|
* table is indistinguishable from a complete one to anyone reading it — you
|
|
* would just see some items named and some not, which is also what "no table at
|
|
* all" looks like.
|
|
*
|
|
* `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and implicitly
|
|
* commits, which would defeat exactly that guarantee. (The same trap the spawn
|
|
* atlas import documents; at ~123k rows `DELETE` is still well under a second.)
|
|
*/
|
|
async function replaceAll(entries, meta) {
|
|
const conn = await pool.getConnection()
|
|
try {
|
|
await conn.beginTransaction()
|
|
await conn.query('DELETE FROM shard_clilocs')
|
|
|
|
// Blank entries are dropped rather than stored. Roughly HALF of a real
|
|
// cliloc table is empty strings — ids the client reserves and never uses —
|
|
// and a row that resolves to no name is indistinguishable from no row at
|
|
// all to every caller. Dropping them halves the table (123,490 → ~67,500)
|
|
// and, more importantly, makes the binary and text imports converge on
|
|
// identical content: the binary format carries the blanks explicitly and a
|
|
// text export may or may not, depending on the tool.
|
|
//
|
|
// Later duplicates win. Merging across sources already happened upstream in
|
|
// `readCliloc`, so in practice this collapses nothing — it is kept because
|
|
// the plain format permits a repeated id WITHIN one file and the client's
|
|
// own loader resolves it the same way (its dictionary assignment
|
|
// overwrites). Without it, a file the game itself would load happily would
|
|
// fail the batch insert on a primary-key collision.
|
|
const byNumber = new Map()
|
|
let blank = 0
|
|
for (const entry of entries) {
|
|
if (!Number.isInteger(entry.number)) continue
|
|
if (String(entry.text ?? '').trim() === '') {
|
|
blank++
|
|
continue
|
|
}
|
|
byNumber.set(entry.number, entry)
|
|
}
|
|
|
|
const rows = [...byNumber.values()].map((e) => [e.number, e.flag ?? 0, e.text])
|
|
for (let i = 0; i < rows.length; i += BATCH) {
|
|
await conn.batch('INSERT INTO shard_clilocs (number, flag, text) VALUES (?,?,?)', rows.slice(i, i + BATCH))
|
|
}
|
|
|
|
await conn.query(
|
|
'INSERT INTO shard_cliloc_meta (id, payload) VALUES (1, ?) ' +
|
|
'ON DUPLICATE KEY UPDATE payload = VALUES(payload), imported_at = CURRENT_TIMESTAMP',
|
|
[JSON.stringify({ ...meta, count: rows.length })],
|
|
)
|
|
|
|
await conn.commit()
|
|
return { count: rows.length, blank, duplicates: entries.length - blank - rows.length }
|
|
} catch (err) {
|
|
await conn.rollback().catch(() => {})
|
|
throw err
|
|
} finally {
|
|
conn.release()
|
|
}
|
|
}
|
|
|
|
async function getMeta() {
|
|
const rows = await query('SELECT payload, imported_at FROM shard_cliloc_meta WHERE id = 1')
|
|
if (rows.length === 0) return null
|
|
const payload = typeof rows[0].payload === 'string' ? JSON.parse(rows[0].payload) : rows[0].payload
|
|
return { ...payload, importedAt: rows[0].imported_at }
|
|
}
|
|
|
|
/**
|
|
* Look up a batch of ids.
|
|
*
|
|
* Batched rather than one-at-a-time because every caller has a LIST: a character
|
|
* sheet resolves a dozen equipment ids at once, and a page of marketplace
|
|
* listings resolves fifty. `IN (...)` with generated placeholders keeps it one
|
|
* round trip and one parameterized statement.
|
|
*/
|
|
async function lookup(numbers) {
|
|
if (!Array.isArray(numbers) || numbers.length === 0) return []
|
|
const ids = [...new Set(numbers.filter((n) => Number.isInteger(n)))]
|
|
if (ids.length === 0) return []
|
|
const placeholders = ids.map(() => '?').join(',')
|
|
return query(`SELECT number, text FROM shard_clilocs WHERE number IN (${placeholders})`, ids)
|
|
}
|
|
|
|
async function count() {
|
|
const rows = await query('SELECT COUNT(*) AS n FROM shard_clilocs')
|
|
return Number(rows[0]?.n) || 0
|
|
}
|
|
|
|
module.exports = {
|
|
replaceAll,
|
|
getMeta,
|
|
lookup,
|
|
count,
|
|
}
|