36 Commits

Author SHA1 Message Date
76c224fff1 Merge pull request 'chore(cutover): sync main into edge before the Asset Bridge cutover (Phase 9b, 0 of 5)' (#43) from chore/cutover-sync-main into edge
All checks were successful
PR Checks / client-build (pull_request) Successful in 19s
PR Checks / server-tests (pull_request) Successful in 30s
PR Checks / frozen-manifest (pull_request) Successful in 1m30s
Reviewed-on: #43
2026-09-14 23:08:49 +00:00
f9bbc7a90d chore(cutover): sync main into edge before the Asset Bridge cutover
All checks were successful
PR Checks / client-build (pull_request) Successful in 21s
PR Checks / server-tests (pull_request) Successful in 27s
PR Checks / frozen-manifest (pull_request) Successful in -31s
`edge` was BEHIND `main` by two commits — Module-uo#35 (the atlas keeps its
`UniqueId`, and a landmark option value names one landmark) and the Event
System's core re-pin — so merging `edge` into `main` as the Asset Bridge
cutover would have REVERTED a released fix. Phase 9a's walk measured it:
0 of 6,455 spawners carried a `UniqueId` on an `edge` rig even after the
column existed.

## The one conflict, and why the number had to move

Both sides bumped `PARSER_VERSION` 4 -> 5, for different reasons, and main's
5 is RELEASED in v1.2.2: "a point keeps its `UniqueId`". `edge`'s 5 was
phase 7's canonical label order.

Keeping 5 would have made phase 7's change unreachable. `sameSources` gates
on the tree hash and `currentParser` on the stored number; an install that
imported under v1.2.2 already stores 5, so a phase-7 build declaring 5 would
be called current and would never re-read. That is precisely the trap this
constant exists to defeat, so the merged file carries BOTH notes: 5 is main's
released meaning, 6 is phase 7's, with the renumbering explained in place.

Everything else merged clean and keeps #35's files verbatim.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 17:34:13 -05:00
e7b3412b36 Merge pull request 'fix(assets): a busy shard is not a broken one, and a column that reached no existing install (Phase 9a)' (#42) from feat/asset-bridge-p9 into edge
Reviewed-on: #42
2026-09-14 22:26:29 +00:00
3c087a43cd fix(assets): a busy shard is not a broken one, and a column that reached no existing install (Phase 9a)
All checks were successful
PR Checks / client-build (pull_request) Successful in 21s
PR Checks / frozen-manifest (pull_request) Successful in 1m20s
PR Checks / server-tests (pull_request) Successful in 8m4s
Two defects the Phase 9 acceptance walk found on a real rig, one of them ours and
one of them released (docs/link/v8.md §17.14).

## "The shard is not answering for client files" about a shard that was fine

The status call exhausts its 425 backoff whenever something else holds the
shard's single asset slot -- an import the operator started, or the item-art warm
pass refilling itself after a client patch. Phase 8's panel rendered that with
the same banner as a shard that is down or has the plane switched off, and left
it standing, because the page only re-reads after an action. On the rig it was up
for a quarter of an hour while the warm pass refilled 313 pictures and every
direct call to the same route answered normally.

BUSY now says what it is, and one automatic re-read four seconds later clears the
ordinary case. One per mount, guarded by a ref: a page that retried forever would
be holding the slot it is waiting for. DOWN, DISABLED and NO_IMAGING read exactly
as they did.

## Every spawn-atlas import on an upgraded install has failed since v1.2.0

`shard_spawn_points.unique_id` (Events Phase 12b) was added to the CREATE TABLE
and nowhere else. `CREATE TABLE IF NOT EXISTS` does not add a column to a table
that already exists -- which is what the twenty-odd `ADD COLUMN IF NOT EXISTS`
lines in this same file are for -- so it reached fresh installs and no existing
one, and `replaceAtlas` inserts the column unconditionally:

    Unknown column 'unique_id' in 'INSERT INTO'

No bestiary refresh, no spawn map, no champion altars, on every install whose
tables predate 12b. A fresh install cannot reproduce it and neither can a test
whose schema is this file applied to an empty database; it took a rig with old
tables. Org lead, weighing that it is already released: it ships here on edge
rather than as a hotfix to main.

Verified by dropping the column, rebooting, watching the replay put it back, and
importing 6,455 spawners over the bridge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 13:07:04 -05:00
5c35b4fe97 Merge pull request 'feat(assets): the panel that operates the client-file imports (Phase 8)' (#41) from feat/asset-bridge-p8 into edge
Reviewed-on: #41
2026-09-14 16:15:15 +00:00
5705aa9c23 ci: re-run the client checks
All checks were successful
PR Checks / server-tests (pull_request) Successful in 27s
PR Checks / frozen-manifest (pull_request) Successful in 1m1s
PR Checks / client-build (pull_request) Successful in 7m59s
Run 68's client-build spent 14m21s in "Set up job" and then failed every step
at 0s with no log uploaded — the runner died during container setup. The same
commit's server-tests and frozen-manifest jobs passed, and frozen-manifest built
this very chunk on the same runner. Nothing in the tree changed; this is the
push the workflow needs to run again (pr-checks has no workflow_dispatch).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 11:07:51 -05:00
675e879b48 feat(assets): the panel that operates the client-file imports (Phase 8)
Some checks failed
PR Checks / frozen-manifest (pull_request) Successful in 1m3s
PR Checks / server-tests (pull_request) Successful in 8m4s
PR Checks / client-build (pull_request) Failing after 14m21s
Admin -> Client Files: one page over the three things that come out of the
operator's UO client -- creature portraits, item and land pictures, and the
cliloc table. One page rather than three because they are one job: same client
install, same bridge, and all of them change at the same moment, when the
operator patches that client. Boot never asks the shard for any of it, so these
buttons are the only thing that imports.

The cliloc pair had had no UI at all since phase 2. On a bridged install, where
boot deliberately stopped calling the shard, that meant `curl` was the only way
to load 67,496 names.

Update and Re-import everything are section 6's two stages as two buttons rather
than one button and a checkbox, because they cost wildly different things. A
vanished key is reviewed in the page and not in a table -- an asset import only
happens because someone pressed a button here, so the review is already in front
of the person who caused it -- and it shows each key's PICTURE, since
`body/820/a23` names nothing a human recognises. `shard_asset_meta` gained a
`last` block (what the import did, who ran it) so the panel can answer "did last
week's import do anything" without scrolling core's whole activity log.

The live walk against a real shard imported 1,095 portraits in 3.5 s, warmed 313
item pictures in 0.6 s and reloaded 67,496 cliloc rows in 1.7 s -- and found two
DELETIONS that predate this phase and that no test could see, because only a
screen showing the numbers together makes them visible:

  * The body import diffed its manifest against every family's rows. Phase 5 put
    item and land art in the same table, and a body manifest never mentions
    them, so all 313 item pictures were staged for deletion with a sentence
    saying the shard had stopped offering them.
  * An approved vanish unlinked the sprite and kept the row. The catalogue went
    on counting a picture that was gone, the atlas could point a creature page at
    a missing file, and the next forced import offered the same key for review
    again -- reporting "nothing was changed" about a file it had deleted.

Both fixed here, with the removals now inside `saveAssets`'s own transaction.
The same whole-table read made the panel announce a 1,408-row creature catalogue
on an install holding 1,095 portraits and 313 item pictures.

Protocol stays 8 and EXTRACTOR_VERSION stays 3: nothing on the wire changed.

Refs: docs/link/v8.md sections 12.2, 14, 16 (phase 8)

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 08:10:16 -05:00
7dbdaa14ad Merge pull request 'feat(atlas): the spawn atlas reads the shard, not the shard's filesystem (Phase 7)' (#40) from feat/asset-bridge-p7 into edge
Reviewed-on: #40
2026-09-14 07:37:18 +00:00
d4d5989926 feat(atlas): the spawn atlas reads the shard, not the shard's filesystem (Phase 7)
All checks were successful
PR Checks / server-tests (pull_request) Successful in 23s
PR Checks / frozen-manifest (pull_request) Successful in 1m8s
PR Checks / client-build (pull_request) Successful in 7m59s
`spawnAtlasSource.js` gains a second backend behind its existing interface
(docs/link/v8.md 10). Where a shard is linked and enabled the tree arrives over
the sidecar; where there is none, a local ServUO tree is read exactly as before.
An explicit --servuo path is an instruction and overrules both.

The parsers do not move. spawnAtlasParse.js is still pure, still fs-free and
still CI-covered without a ServUO tree anywhere near it; `buildFromFiles` is now
where the parse starts, and both readers feed it the same shape.

treeBridge.js walks the manifest and then the chunks. Three of its checks are
not decoration -- each is a way this ends in a tree that LOOKS imported, and XML
is forgiving enough that a mis-assembled spawn file parses cleanly and simply
has fewer spawns in it:

  - every chunk re-declares its address and carries the hash of its own
    uncompressed bytes, and chunks are placed by declared index rather than
    arrival order
  - the whole file is hashed after reassembly against its manifest row
  - the catalog must not move mid-walk, or the import is refused rather than
    stitched out of two trees

Boot does not call the shard. The same answer 17.7 gave the cliloc table, and
the same reasoning: a local tree hashes in ~120 ms and skips, while a round trip
in the boot sequence would answer "no" on every restart that did not follow a
map edit. Editing spawn files is an operator action, so importing is one --
Admin -> Spawn Atlas -> Import. What that costs is real and is said out loud in
the panel, the CLI and the log: an install on the bridge has NO automatic
refresh at all.

Two things the live walk found that the unit tests could not:

  - PARSER_VERSION 4 -> 5. The parse is order-sensitive in one place -- the
    decoration index keeps the FIRST item id it sees for a type -- and the two
    readers agreed on a stock tree by coincidence, since the filesystem reader
    walks each directory with localeCompare while the shard sorts whole relative
    paths. buildFromFiles now sorts by label, ordinally, once, whatever order
    the files arrived in. Identical input, a different answer for a handful of
    types: exactly what the version number exists to push through the hash gate.
    The parity test asserted deepEqual, which ignores key order; it now asserts
    serialised equality too.
  - The source fingerprint is taken over RAW BYTES at both ends. Hashing decoded
    text hashes a UTF-8 re-encoding -- identical for valid UTF-8, different for a
    file that is not, because an undecodable byte becomes U+FFFD and never comes
    back. One Latin-1 character in a creature name would have made the drift gate
    report a change on every import, forever, with the tree untouched.

A 200 from assets.sources also stopped meaning "the client files are on offer":
a shard may now serve its configuration tree while declining to serve its UO
client. Both client-file readers check `assetsEnabled` and say DISABLED, instead
of reading an empty file list as "your client has no cliloc.enu" and sending an
operator to their client install for a setting that lives on their shard.

Measured end to end against a live shard and the real sidecar: 141 files,
11.9 MB, 158 chunks, 3 pages, 1.33 MB on the wire, 512 ms; every file
byte-identical to disk; and the atlas built over the bridge identical to the one
built off it -- 6,455 points, 800 creatures, 387 regions, 558 landmarks,
25 champions, 309 decoration types.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 02:00:43 -05:00
679762b643 Merge pull request 'feat(assets): a creature's picture is whichever action has one (Phase 6)' (#39) from feat/asset-bridge-p6 into edge
Reviewed-on: #39
2026-09-14 06:16:09 +00:00
c6c51b190d feat(assets): a creature's picture is whichever action has one (Phase 6)
Some checks failed
PR Checks / frozen-manifest (pull_request) Successful in 51s
PR Checks / client-build (pull_request) Successful in 8m9s
PR Checks / server-tests (pull_request) Failing after 14m27s
The shard's catalogue can now answer for 73 bodies it used to report absent —
they have no art at action 0 and real art at a later one, and their key says
which (`body/820/a23` is a horse). This side stores that action and stops
assuming `a0` anywhere.

The atlas join is the part that mattered. It read

  a.asset_key = CONCAT('body/', b.body, '/a0')

which would have silently dropped exactly the creatures this phase adds. It now
reads the row's own action, with COALESCE for rows written before the column
existed — a NULL inside CONCAT makes the whole comparison NULL, which would have
taken every portrait off the site on upgrade with the database perfectly correct
and nothing in any log. It still matches at most one row per slug: a deeper key
(`body/820/a23/f4`) does not equal the catalogue key.

Verified against a real MariaDB with the live shard's own 1,095-row manifest: the
ALTER applies to an installed-shape table and is idempotent, the horse joins to
its a23 picture, a pre-phase-6 NULL-action row keeps its portrait, and a stored
frame key does not become a second candidate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 01:09:50 -05:00
53b3aca0c1 Merge pull request 'feat(assets): item pictures on the marketplace and the character sheet (Phase 5)' (#38) from feat/asset-bridge-p5 into edge
Reviewed-on: #38
2026-09-11 11:27:03 +00:00
f335531538 feat(assets): item pictures on the marketplace and the character sheet (Phase 5)
All checks were successful
PR Checks / client-build (pull_request) Successful in 17s
PR Checks / frozen-manifest (pull_request) Successful in 41s
PR Checks / server-tests (pull_request) Successful in 8m23s
Both places this site already knew an item's (ItemID, hue) and could only print
it as text now show the picture, hued the way the client would draw it. The
shard does the hueing: whether a hue repaints every pixel or only the grey ones
is a flag in `tiledata.mul`, which a browser has no way to read.

**Ingest warms; the route only serves** (org lead, 2026-09-11). A page never
waits on the shard and never causes a fetch -- it renders what is stored and
leaves out what is not, which is the state every install was in before this
phase. Fetching happens behind that, on a timer, from the keys the site's own
rows name. The alternative, fetching on first request, was rejected on one
number: the shard's asset plane serves ONE request at a time, so a URL that
fetched would let any visitor walk 49,152 ids times 3,000 hues through that slot
and park an operator's own import behind it.

The wanted set is DERIVED (`SELECT DISTINCT item_id, hue`) rather than queued, so
it is self-healing: a restart loses nothing, and a key stops being wanted the
moment the vendor row naming it is deleted. The in-memory hint set on top is only
for the character sheet, which is fetched live from the shard and stored nowhere
-- nothing on disk would ever name those keys.

Staleness without a manifest (§7): every row records the shard's `catalog` id, a
hash of the files that decide its bytes. A client patch changes it and a restart
does not, so "is this out of date?" is a per-row question -- and pictures nobody
looks at any more are simply never re-fetched, which is why this is lazy rather
than a sweep. `shard_asset_meta` is deliberately NOT written here: it is the body
catalogue's singleton, and a warm pass touching it would tell the body import
that a client it never looked at is unchanged.

A key the shard has no art for writes no row at all. An empty row would make the
key held and it would never be asked again -- including after the operator
patches in the graphic that was missing.

`assets.sources` now reports which families an overlay serves, so an overlay
older than phase 5 is one reported state with a sentence naming the fix, instead
of a refusal per pass forever with no picture ever appearing.

688 server tests pass (14 new); client builds; the frozen manifest regenerates
with one added route, all documented, no core URL moved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-11 06:13:08 -05:00
6ece48f7d3 Merge pull request 'feat(assets): creature artwork from the shard's own client (Phase 3)' (#37) from feat/asset-bridge-p3 into edge
Reviewed-on: #37
2026-09-10 23:58:11 +00:00
a194ec68e0 feat(assets): creature artwork from the shard's own client (Phase 3)
All checks were successful
PR Checks / client-build (pull_request) Successful in 18s
PR Checks / server-tests (pull_request) Successful in 24s
PR Checks / frozen-manifest (pull_request) Successful in 49s
Until now the only way a creature got a picture on this site was for an
operator to open UOFiddler on a desktop, export sprites by hand, copy them to
the web host and write a spawnAtlas.art.json naming each one. Almost nobody
did, so shard_spawn_creatures.art was NULL on every install.

The shard has had those files the whole time. Admin -> Shard -> Import now
walks its asset manifest, fetches only the sprites whose hash changed, writes
them under uploads/atlas/, asks the shard for a body id per atlas creature
(§8: it CONSTRUCTS the creature and reads Body.BodyID, which is the only thing
that is right for a shard's own custom creatures) and points each creature at
its picture. On a stock client that is 787 portraits, about a megabyte.

**The one thing v8.md §12 got wrong, and it is not cosmetic.** It says
`shard_spawn_creatures.art` "starts being filled by the import". That table is
emptied and refilled by replaceAtlas on EVERY atlas refresh, and a refresh runs
on every boot -- so a filename stored there would be destroyed by an ordinary
re-parse of the ServUO tree, with the next Update finding the client files
unchanged, reporting "nothing to do", and never restoring it. Nothing would
report a fault; the pictures would just be gone.

So the assets and the body map live in their own tables outside that blast
radius, and applyAtlas re-derives `art` on the way past as
`{ ...derived, ...operatorMap }` -- which is also the one place "the operator's
own artwork wins" is enforced, on every rebuild rather than only at import.

Smaller decisions worth not rediscovering:

- The derivation joins on the catalogue KEY, not on the body id. The simpler
  join is correct today and stops being correct the moment phase 6 adds
  body/400/a2/f0, at which point one slug matches dozens of rows.
- Filenames are content-addressed. A stable name overwritten in place leaves
  every browser and CDN serving the previous client's sprite, with the database
  row perfectly correct.
- An unchanged key whose FILE is missing is fetched again. The row and the disk
  can disagree (a wiped uploads volume, a restore from a dump), and a broken
  image on a creature page is worse than one re-fetched sprite.
- A key the shard cannot render is not a failure. Two thirds of the playable
  ghost and gargoyle bodies have no art on a stock client, and an import that
  reported eight failures every time would teach an operator to ignore the panel.
- A key that VANISHED from the manifest needs review before anything changes:
  an unmounted client volume and a deliberate downgrade look identical here.

23 new tests; 674 server and 42 client tests pass. The SQL was also run against
a real MariaDB, which is what proved the CONCAT join and the singleton CHECK.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-10 18:40:46 -05:00
55df03496d Merge pull request 'feat(cliloc): import the table from the shard, not from a file someone converted (Phase 2)' (#36) from feat/asset-bridge-p2 into edge
Reviewed-on: #36
2026-09-10 16:20:12 +00:00
893a36618b feat(cliloc): import the table from the shard, not from a file someone converted (Phase 2)
All checks were successful
PR Checks / client-build (pull_request) Successful in 34s
PR Checks / frozen-manifest (pull_request) Successful in 53s
PR Checks / server-tests (pull_request) Successful in 8m18s
The base cliloc table now comes over the bridge. `clilocBridge.js` walks
`GET /cliloc` page by page and the model merges the `custom/` overlays over it —
overlays stay on disk because ServUO has no server-side notion of a custom
cliloc, so there is nothing on the shard to ask for.

**The shard wins whenever uo-link is configured and enabled**, with no mode
setting: there is no version of "which source?" an operator benefits from
answering. A file on disk remains the source only where there is no shard link,
plus a one-off explicit `path` — deprecated, not removed, and unchanged.

**Boot no longer imports on the bridge.** The file path could hash 5 MB locally
and skip in 14 ms; a shard round trip in the boot sequence would be spent
answering "no" on every restart but the one after a client patch — and patching a
client is an operator action, so importing became one. Admin → Shard → Import.
Whatever table is loaded keeps serving until then.

Three checks in the walk, each for a way a shard can hand back a table that looks
complete:

  * only `cut: 'end'` finishes it — a short page can equally be a spent budget,
    and a truncated table renders some items named and some not, which is exactly
    what NO table looks like;
  * the cursor must advance, or the walk stops rather than spinning;
  * every page echoes the source's size and mtime, so a client patched mid-import
    is refused outright rather than stitched from two files.

**The base is exempt from the vanished-source rule**, which is an upgrade detail
rather than a preference: an install that used the file pipeline carries its base
file's label in the stored fingerprint, and on the bridge that label is *supposed*
to disappear. Counting it as vanished would demand an approval for a change the
upgrade itself made. Overlays keep the rule in full.

**The protocol pin moves 7 → 8** — the third declaration site, and the one
nothing enforces. Phase 1 moved the sidecar and the overlay together because the
installer refuses a mismatched bundle; this one has to be moved by hand, in the
phase that first calls a protocol-8 route. The schema block above it is the
record of what forgetting costs: two phases of every REST call answered 409.

Verified against a live shard, sidecar and site: 12 pages, 67,496 rows imported
in 1.68 s, the operator's three-row overlay overriding stock strings on top of
it, and the next import correctly `unchanged`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-10 11:13:24 -05:00
50f84b5ea3 Merge pull request 'fix(atlas): keep the UniqueId, and make a landmark value name one landmark' (#35) from fix/atlas-unique-id-and-landmark-values into main
Some checks failed
Release / release (push) Successful in 27s
SonarQube / analysis (push) Failing after -59s
Reviewed-on: #35
2026-09-10 02:14:49 +00:00
d6346996d3 fix(atlas): keep the UniqueId, and make a landmark value name one landmark
All checks were successful
PR Checks / client-build (pull_request) Successful in 23s
PR Checks / server-tests (pull_request) Successful in 28s
PR Checks / frozen-manifest (pull_request) Successful in -21s
Two defects the Phase 16b re-verify found in the released v1.2.1 bundle, both
of which make a shipped feature unusable and neither of which any test saw.

## The aggregator discarded the UniqueId

`shard_spawn_points.unique_id` was NULL on all 6,455 rows of a stock 57.4 tree.
`listSpawners` filters `unique_id IS NOT NULL`, so `uo.options.spawners` was an
empty dropdown -- and it is the ONLY option source for the Phase 12b
object-property leases, so no `Spawner.MaxCount` / `MinDelay` / `MaxDelay` lease
could be authored at all, with nothing on the form to say why.

Every part of the path was already right except one line. The spawn files carry
`<UniqueId>` (~6,374 of them), `parsePoints` returns it, the column exists and
the insert passes `p.uniqueId || null`. `buildAtlas` rebuilds each point from an
explicit field list and `uniqueId` was not on it -- the word appears nowhere in
that file. `PARSER_VERSION = 4`'s own note says "a spawn point keeps its
UniqueId, which is what a property lease targets", so the intent shipped as a
comment while the code dropped the field one function later.

`PARSER_VERSION` goes to 5 because the bump is the only thing that re-reads an
already-imported tree: `sameSources` compares the tree's hashes, which have not
changed -- only what is kept from them. Confirmed on the rig, where the boot
after the fix logged `spawn atlas refreshed` on an unchanged tree and the manual
import then correctly answered `unchanged`.

## A landmark option value named 23 places at once

A stock tree has 558 landmarks under 320 distinct `facet/name` pairs.
`Trammel/Entrance` is 23 different dungeons -- Blighted Grove, Covetous, Deceit,
Despise, Destard and so on -- and `landmarkPoint` resolved with `.find()`, so 22
of the 23 were unreachable. An author who picked "Entrance - Destard" got
Blighted Grove, and the run succeeded with no warning. The group was already the
disambiguator: it was shown in the dropdown and left out of the value.

The value is now `facet/group/name`, which is distinct across all 558.
`landmarkPoint` tries that form first and keeps the two-part read as a fallback,
because every event published before this fix stores `facet/name` and a
published version is immutable -- refusing to parse those would break runs
rather than correct them. The fallback keeps the old first-match behaviour
deliberately: it is imprecise in exactly the way it always was, and silently
relocating a live event's spawn point is worse than repeating a known
imprecision. A three-part value whose group is gone REFUSES rather than falling
back to the name, because it asked for one particular place.

## Verification

On the released-artefact rig (installer -> bundle 2026.09.10 -> stock 57.4 tree
-> protocol-7 sidecar -> core at main with this module):

  spawn points     6455 rows, 6364 with a unique_id   (was 0)
  uo.options.spawners   100 options, and `?q=orc` searches them   (was 0)
  uo.options.landmarks  558 options, 558 distinct values          (was 320)
  suite            625 pass, 0 fail

Each new test was confirmed to FAIL without its fix. The atlas one asserts the
field on the AGGREGATOR's output rather than the parser's, which is the whole
point of it -- and the test fixture had no `<UniqueId>` at all until now, which
is exactly why a green suite said nothing. The landmark one asserts an
INEQUALITY between two resolved points rather than a literal value string, so it
survives another change of format as long as two options still address two
places.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-09 20:59:49 -05:00
bbaf08f67c Merge pull request 'feat(events): the UO half of the Event System, and the core pin comes home (Phase 16b cutover, 3 of 6)' (#34) from chore/events-cutover-repin into main
Some checks failed
Release / release (push) Failing after -45s
SonarQube / analysis (push) Successful in 2m30s
Reviewed-on: #34
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
2026-09-10 01:23:30 +00:00
ea63ad019c chore(ci): the core pin comes home to main (Phase 16b cutover)
All checks were successful
PR Checks / client-build (pull_request) Successful in 18s
PR Checks / server-tests (pull_request) Successful in 23s
PR Checks / frozen-manifest (pull_request) Successful in 49s
`ci/core-ref.json` pointed at a website `edge` sha for the length of the Event
System window (org lead, 2026-09-04), because `api.registerEventActions` exists
only from MODULE_API 1.10.0: under the old `main` pin the frozen-manifest job's
`register()` threw and this module did not load at all, so the job would have
been red by construction for eight phases while a real regression hid behind it.

The cutover put 1.10.0 on `main` (website#199, 655fbf3f), so the pin returns to
a `main` sha -- and this is the same move that turns the Integration kit green,
since `checkCoreApi` asserts equality against whatever core this pin names.

`routes.manifest.json` needed NO regeneration. The frozen-manifest job's own
steps were run against this exact ref -- core's manifest alone, the module
installed, core's manifest again, then `frozenManifest.js --check` -- and it
answered `routes.manifest.json is current, 73 routes, all documented`. So the
file's own "commit both together" instruction had nothing to pair with this
time. website's `main` and `edge` are the identical tree (930422ff), which is
why the measurement taken on the branch holds for the merge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-09 19:48:29 -05:00
c73d62e93a Merge pull request 'fix(events): the atlas import, and a teardown that was a no-op (Phase 16a)' (#33) from fix/events-p16a-walk into edge
Reviewed-on: #33
2026-09-09 13:47:56 +00:00
8def6e19f4 fix(events): the atlas import, and a teardown that was a no-op (Phase 16a)
All checks were successful
PR Checks / client-build (pull_request) Successful in 24s
PR Checks / frozen-manifest (pull_request) Successful in 58s
PR Checks / server-tests (pull_request) Successful in 8m27s
Two defects the acceptance walk found in shipped code, both invisible to the
suites that were green on either side of them.

**The spawn atlas cannot import on a stock ServUO tree.** `spawnAtlasSource.js`
dedupes decoration types with a case-SENSITIVE `Map`, but `shard_decor_types.type`
is a PRIMARY KEY under MariaDB's default `..._ai_ci` collation, which folds case.
Stock 57.4's own `Data/Decoration/` names four types under two spellings each
(CheckerBoard/Checkerboard, ChessBoard/Chessboard, MetalChest/Metalchest,
SpinningWheelEastAddon/SpinningwheelEastAddon), and in every pair exactly one is a
real class. The second row raised `1062 Duplicate entry` and took the WHOLE import
transaction down. The blast radius is not decoration: with no atlas, EVERY option
source answers empty and no Phase 12 world verb can be authored at all.

The shard end already knew — `BridgeWorld.cs` resolves a decor type with
`FindTypeByName(name, ignoreCase: true)` and its comment says the atlas and the
decoration files disagree about casing. Folding here is the two ends agreeing.

**Teardown of every world verb was a no-op that reported success.** `revertOwned`
forwarded core's `idempotencyKey` as the despawn's OWN key — and core's key is the
step's, the one `placeOwned` spawned under. `BridgeIdempotency` keys on the key
alone, so the despawn was taken for a repeat and answered with the SPAWN's stored
reply; `OnDespawn` never ran. Core read `ok` with no `refused` and marked every
row `reverted` while the shard still held every object.

Measured on the rig: ledger `world | reverted | 21`, shard `world.owned` 21 alive
with `pruned: 0`, and the identical despawn re-sent with a fresh key removed all
21. It affected all five world verbs, so an invasion's creatures, boss, oracle,
gate and decoration stayed in the world for ever while the console reported a
clean teardown.

`MODULE_API.md` says what that key is for and it is not this: it identifies a
dispatch core never learned the outcome of, so the module can ask about it. No key
is needed on a despawn — a repeat answers `gone`, which both ends already treat as
success — and dropping it also makes the documented empty-`resources` case work,
since no serials means "everything this run owns". The parameter is removed from
`despawnWorld`'s signature rather than left optional.

Both fixes are verified end to end against a real ServUO + sidecar + website rig:
the import now yields 309 decor types (was failing at 313 with 4 collisions),
6,455 spawn points, 800 creatures, 558 landmarks; and a full four-phase run's
teardown left the shard owning 0 objects.

Each new test was confirmed to FAIL without its fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-09 08:27:50 -05:00
c289586a3d Merge pull request 'feat(events): what an author borrows, and two one-shots (Phase 12b)' (#32) from feature/events-p12b-borrowed-and-oneshots into edge
Reviewed-on: #32
2026-09-07 16:23:32 +00:00
10fde87724 feat(events): what an author borrows, and two one-shots (Phase 12b)
All checks were successful
PR Checks / server-tests (pull_request) Successful in 38s
PR Checks / client-build (pull_request) Successful in 24s
PR Checks / frozen-manifest (pull_request) Successful in 53s
Five targeted leases over two planes, the item grant, the world save, and the
atlas work the spawner dropdown needed.

FIVE LEASES, ONE FACTORY

`uo.spawner.maxcount`, `.mindelay`, `.maxdelay`, `.running` and
`uo.seasonal.status`. The four callables differ only in which key they name, so
they are built rather than repeated: five copies would be five chances for one of
them to forget the drift check, which is the one thing §F says a lease must not
be allowed to skip.

It is `MaxCount`, not the `Amount` EVENTS_PLAN.md named -- there is no such
property on ServUO 57.4. `MinDelay`/`MaxDelay` are TimeSpans, so the wire carries
SECONDS: the spawn files' own `DelayInSec` flag proves both units are in use on a
real tree, and a unit that cannot express five seconds cannot express this
shard's own data.

The seasonal lease is a THREE-value enum over EIGHT events. §G called
`GetEntry(type).Status` "a nine-value enum" and had it backwards: `EventStatus`
has three values and it is `EventType` that has nine entries. Eight rather than
nine because `TreasuresOfTokuno` is excluded -- `IsActive()` reads its own
`DropEra` rather than `Status`, so leasing it would apply cleanly, read back,
restore cleanly and do nothing at all.

Two behaviours worth the review. `inForce()` reads the frame's `holds` rather
than a row's `held` flag, because a catalog walk can enumerate the keys but never
the holds on a targeted one. And a target that VANISHED mid-run is a SUCCESSFUL
restore: there is nothing to give back, and reporting it failed would leave a
ledger row unresolved for ever over an object that is gone -- 12a's `gone` in the
lease plane's vocabulary.

THE GRANT NAMES A RUN, NEVER A RECIPIENT LIST

Core has the participants in `event_run_participants`, but a module cannot read
core's tables -- so the alternative was a new core surface handing them over. Not
needed: the shard has held the run's ledger since it opened, keyed by the same
serials core stores as `member_key`.

And the grant is RETRYABLE. §G called it un-retryable because a lost
acknowledgement and a grant that never applied were the same event, which is
exactly the argument that made `uo.broadcast` answer `retry: false` in Phase 9.
Protocol 6's idempotency key closes it. `uo.rewards` counts ITEMS rather than
grants: 500 gold to forty people and a candle to forty people are not the same
imposition.

THE ATLAS KEEPS UniqueId AGAIN, AND THE SPAWNER SOURCE SEARCHES

The parser has read `<UniqueId>` and thrown it away since the atlas shipped, on a
line citing a committed artifact -- there is no committed artifact, as
`spawnAtlasSource.js` says in its own header. It is the ONLY name for one
particular spawner that exists off the shard, so a property lease could not have
had a dropdown without it. `PARSER_VERSION` -> 4 so an unchanged tree is re-read.

`uo.options.spawners` is the first searchable source and the first that had to
be: 6,707 spawn points against `MAX_OPTIONS`' 2,000, so a flat list would drop
two thirds of the world and say nothing about which two thirds.

ONE DEFECT IN ALREADY-MERGED CODE, AND IT WOULD HAVE BROKEN EVERYTHING

The protocol pin never left 5. `uo_link_config.protocol` reaches the sidecar as
`X-UOLink-Version` on every REST call and an exact mismatch is a 409, so from
Phase 11a onward every sidecar call on a real deployment would have been refused
-- the whole event plane dead, loudly, for a reason nobody would look here for.
11a took the wire to 6 and 12a to 7; neither moved the pin, in either of the two
places this repo declares it. It survived both because both live walks set the
column by hand while standing the rig up, which is exactly what makes a migration
nobody runs invisible. All three sites go to 7.

The test that guards them is worth understanding before trusting it:
`schemaFragment.test.js` asserts the three declarations agree WITH EACH OTHER --
a real check they once failed -- but all three being equally stale passes it, and
nothing in this repo can anchor it to the wire. Recorded in the model's own
header so the next reader knows.

CHECKS

`npm test`: 620 pass, 0 fail (was 605). `check:imports` and `check:externals`
clean; the client builds and its 42 tests pass. `check:swagger` reports the
fragment stale -- it is ALREADY stale on `edge` (verified by stashing this
branch's changes and re-running) and this phase adds no route, so it is left
alone rather than regenerated inside an unrelated change.

Two bugs the new tests caught in this branch's own code before it left: `counted()`
returns `.count` and the grant read `.value`, so every grant went out with
`amount: undefined` and the non-stackable guard never fired; and `optionalInt`'s
`ok` was ignored, so a bad hue passed silently instead of refusing.

Refs: docs/link/v7.md §11-§14, docs/website/EVENTS_PLAN.md Phase 12b

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-07 08:08:08 -05:00
0b142adb81 Merge pull request 'feat(events): the five world verbs an author sees (Phase 12a)' (#31) from feature/events-p12a-world-verbs into edge
Reviewed-on: #31
2026-09-07 06:58:04 +00:00
89be9d6a4e feat(events): the five world verbs an author sees (Phase 12a)
All checks were successful
PR Checks / client-build (pull_request) Successful in 28s
PR Checks / server-tests (pull_request) Successful in 30s
PR Checks / frozen-manifest (pull_request) Successful in 43s
`uo.creature.spawn`, `uo.boss.spawn`, `uo.npc.place`, `uo.gate.open` and
`uo.decor.place`, over protocol 7's one command family. Five actions because
five is what an author has; one `perform`/`revert`/`reconcile` because on the
wire they are one thing.

Five new budget dimensions -- `uo.creatures`, `uo.bosses`, `uo.npcs`,
`uo.decor`, `uo.gate.minutes` -- all declared by THIS MODULE (org lead,
2026-09-07). Core meters whatever dimensions a module declares and holds no UO
knowledge, which is the whole of what MODULE_API means by game-agnostic. A gate
is priced in minutes rather than in gates: one standing all day and twelve
standing five minutes each are not the same imposition on a world.

`reconcile()` ASKS the shard, and is the one place in this file that must not
use `reconcileByBootId`. A crier line lives in shard memory, so a changed
`bootId` IS proof it is gone; a spawned creature is in the world SAVE and
survives the restart the stamp would report it lost by. Anything `world.owned`
does not list is gone -- safe only because the shard's registry and the objects
it describes are written by the same save.

Teardown reports `gone` as success and `refused` as failed. A creature a player
killed is the point of having spawned it, and a run that ended `incomplete`
because its event worked would be a report nobody could read. `refused` means
the shard denies this run ever owned the serial, so nothing will delete it
through this path and the row must land unresolved with a reason.

The atlas gains a decoration index, parsed from the shard's own
`Data/Decoration/**/*.cfg` -- 120 files, read RECURSIVELY because the real tree
nests two deep and a flat read would index a fraction of it while looking like
it worked. 313 distinct types. The decor verb resolves through it rather than
passing a type name through, which keeps the verb to this shard's own decoration
vocabulary AND fetches the item id: `Static` alone accounts for 5031 placements
under 1992 different graphics, so a bare type name places the wrong thing.
`PARSER_VERSION` -> 3, so an already-imported tree is re-read.

Two things the build found in code that had already shipped:

`uo.options.creatures` answered with the atlas SLUG -- unique, stable, and not
something the shard can build, because a creature is constructed from a ServUO
class name and `orc-brute` is not one. The atlas's `name` is the raw type token
from the spawn files, so the fix was to stop discarding the half that works.
Safe to change because Phase 12a is the source's first consumer; the file said
so when it shipped.

`uo.npc.place` could not be performed from its own required params. Both ends
refuse an oracle with neither a greeting nor a line, but both fields were
optional -- so a cross-field rule sat where no authoring form could render it.
The greeting is now `required`, which says the same thing in the contract
itself. Caught by the existing dry-run sweep, which is a better argument for
that test than anything written about it when it shipped.

605 tests pass. `swagger-fragment.json` is stale on `edge` already and this
phase adds no route, so it is left alone.

Refs: docs/link/v7.md, docs/website/EVENTS_PLAN.md Phase 12a

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-07 01:52:15 -05:00
c11c130438 Merge pull request 'feat(events): one lease and the participation verbs (Phase 11b)' (#30) from feature/events-p11b-leases-participation into edge
Reviewed-on: #30
2026-09-05 04:10:14 +00:00
88bfe9310e feat(events): one lease and the participation verbs (Phase 11b)
All checks were successful
PR Checks / client-build (pull_request) Successful in 20s
PR Checks / server-tests (pull_request) Successful in 26s
PR Checks / frozen-manifest (pull_request) Successful in 40s
The UO half of protocol 6 part b. No route added, no schema change, no
MODULE_API bump.

`uo.playercaps.skillcap` is the one lease, and the catalog is short because
ServUO made it short: of the 158 non-Bridge `Config.Get` call sites in
`Scripts/`, roughly eight are read live. This one is read inside
`CharacterCreation.cs`'s per-character path, so it is both live and observable --
which is what "proven" has to mean, since the failure an allowlist exists to
prevent is a key that applies cleanly and changes nothing.

Its `apply()` sends a DURATION rather than the deadline: an absolute time
computed here and honoured there is measured against two clocks, and a shard
running ten minutes fast would restore a ten-minute lease the instant it took it.
Its `restore()` turns `lease.drifted` into `{ drifted: true, current }` rather
than an error, because core records drift as a distinct successful outcome and an
error would put the row on the retry ladder. Its `inForce()` asks whether the
shard still HOLDS the lease, never whether the value still matches -- see the
core PR.

`uo.participation.open` / `.collect` count who took part and file them on the
success envelope. `open` is the one resource in this module that must NOT
reconcile by boot stamp: every other resource here lives in shard memory, so a
changed bootId IS the proof it is gone, while the participation ledger is written
into the world save precisely so it survives that restart. It asks instead.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-04 19:31:57 -05:00
bf9a702cfa Merge pull request 'feat(events): send the idempotency key, declare champ.boss.killed (Phase 11a)' (#29) from feature/protocol-v6-idempotency into edge
Reviewed-on: #29
2026-09-04 23:07:02 +00:00
dc13515927 feat(events): send the idempotency key, and declare champ.boss.killed (Phase 11a)
All checks were successful
PR Checks / client-build (pull_request) Successful in 20s
PR Checks / server-tests (pull_request) Successful in 26s
PR Checks / frozen-manifest (pull_request) Successful in 39s
The website's half of protocol 6.

Every event-driven write now carries the step's idempotency key, and `uo.broadcast`
stops being un-retryable. Phase 9 shipped it answering `retry: false` to everything
including a 503 from a shard that was merely restarting, with a comment naming the
line that would change when the wire could refuse a repeat. This is that line: it
defers to `sidecarFailure`, the same helper its two siblings already used, so the
hand-rolled variant that forced every outcome terminal is gone rather than re-tuned.

One verb was less idempotent than its own id made it look. Both keyed verbs post
under a run-scoped id and a repeat replaces — but `news.add` with `announce: true`
makes the criers proclaim the title on every post, so a retry replaced the article
silently and proclaimed it again. The key stops the second proclamation.

`champ.boss.killed` is mapped to the `champs` feature (rule 2 would otherwise fail
it closed to admin), with `damagers` a nested `staff` field rule: the kill is public
because a champion falling is what the board is for, the ranked roll of who was
strong enough to fell it is not. `uo.champ.boss_killed` is declared as a trigger —
which is what makes it usable as an event PHASE CONDITION, since a condition is
written over a trigger firing — and it carries `damagerCount`, never a damager name,
because a trigger variable reaches mail an operator may address to every subscriber.

Its seeded rule is its own group, `champ-boss-killed-v1`: `triggers-v1` is stamped
once under a settings guard, so appending a 27th entry would have reached fresh
installs and nothing else. It also ships email+inapp and NOT push, and the comment
says why — no trigger in this module is also a registered stream, so no engagement
rule here can push. That is pre-existing in twenty rules and flagged rather than
fixed; this one declines to be the twenty-first.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-04 14:57:26 -05:00
cf60932c85 Merge pull request 'feat(events): UO wave 1 — the verbs that need no protocol change (Phase 9)' (#28) from feature/events-phase-9 into edge
Reviewed-on: #28
2026-09-04 12:56:13 +00:00
021f191f65 fix(events): three defects the live rig found, two of them data loss
All checks were successful
PR Checks / client-build (pull_request) Successful in 20s
PR Checks / frozen-manifest (pull_request) Successful in 41s
PR Checks / server-tests (pull_request) Successful in 8m33s
The whole-rig walk (ServUO + sidecar + website) against a real two-phase event.

- **A WS reconnect would have orphaned every live resource.** The backfill
  replays the last several `server.hello` frames in order — this rig saw three,
  each with a different `bootId` — so every replayed frame reads as a restart,
  and the intermediate ones compare a resource stamped with the CURRENT boot
  against a boot that ended hours ago. The row is then `orphaned`: a live crier
  line core will never take down again, lost to nothing worse than the website
  reconnecting. Gated on `!fromBackfill`, the rule the engagement fan-out and
  the SSE broadcast beside it already state. The website-was-down case is not
  missed — core asks every module at its own boot.
- **The shard explains its refusals and the run log dropped the explanation.**
  A 403 body reads `{"reason":"admin write plane disabled"}`; `legError` looks
  for `data.message`, finds nothing, and reports "sidecar responded 403". For a
  staff member clicking a button that is survivable. For an event that ran at
  four in the morning the run log is the only place anyone will learn why.
- **The "not retried" clause explained the wrong thing on a permanent status.**
  A 403 will not succeed on any attempt, so telling an operator it was not
  retried "because a repeat would announce twice" points them at a policy
  decision instead of at the switch they have to flip. The clause is now added
  only where a retry was genuinely given up, and 403/404 join the statuses the
  keyed verbs treat as terminal.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-04 07:36:03 -05:00
57419111e6 feat(events): UO wave 1 — the verbs that need no protocol change (Phase 9)
module-uo registers its first event actions: `uo.broadcast`,
`uo.towncrier.post` and `uo.news.post`, plus the `uo.broadcasts` budget
dimension and the three spawn-atlas option sources. The write plane they use
has existed since protocol 2.1; what is new is the declaration that lets the
event engine drive it unattended.

Three things the tree corrected about the plan:

- The plan's `on_failure: 'skip'` for `uo.broadcast` is already the default for
  `risk: 'notify'`, and `on_failure` is what happens AFTER the retries. The
  lever a module actually has is the failure envelope, so the action answers
  `retry: false` to everything — and every action declares `budgetMs: 15000`,
  because core's 10s default deadline fires before `uoLinkClient`'s 12s timeout
  and `classify()` answers `retry` for a timeout without asking the module.
  Without the budget the retry refusal is unreachable.
- `reconcile()` needs no protocol work. A shard restart wipes both the crier
  lines and an event's news article, so `perform()` stamps the shard `bootId`
  into the resource payload and `reconcile()` reports in force exactly the rows
  whose stamp still matches — correct for the module's own trigger and for
  core's boot sweep alike. `shardIngest` fires `ctx.events.reconcile()` on a
  changed `bootId`, after `recordStatus` so the comparison reads the new boot.
- Event articles post under `evt-<idempotencyKey>`, because `newsGump.js` uses
  the bare website post id and re-pushes that set on every reconnect.

`ci/core-ref.json` moves to a website `edge` sha for the length of this
workstream: `registerEventActions` exists only from MODULE_API 1.10.0, so under
the old `main` pin the module does not load at all. Verified locally — the
frozen-manifest rig passes against the new pin.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-04 07:23:03 -05:00
144242fe8f Merge pull request 'ci(core-ref): pin the core on main, now that the cutover has landed' (#27) from ci/core-ref-main into main
All checks were successful
Release / release (push) Successful in 11s
SonarQube / analysis (push) Successful in 2m13s
Reviewed-on: #27
2026-09-01 18:03:20 +00:00
c679944181 ci(core-ref): pin the core on main, now that the cutover has landed
All checks were successful
PR Checks / client-build (pull_request) Successful in 33s
PR Checks / frozen-manifest (pull_request) Successful in 47s
PR Checks / server-tests (pull_request) Successful in 8m24s
Module-uo#25 moved this pin onto website `edge` (52eac24) to unbreak
frozen-manifest during the engagement window, with its own note saying it
reverts to a `main` sha at the cutover. The cutover is step 4 of 7, merged as
#26, and website#180 landed the same code on `main` -- so the pin now names a
branch that no longer exists.

No regeneration, and the reason is checkable rather than asserted: website's
tree at 66bb3b9a (main, the cutover merge) and at 52eac24d (the edge head it
merged) are the SAME tree, e7a7240. `main` was zero commits ahead, so the merge
carried edge's tree unchanged. The frozen-manifest job clones a different commit
and reads identical bytes; routes.manifest.json cannot move.

What changes is what a reader learns from the file: which core this module was
last proved against, named by a ref they can still resolve.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-09-01 12:48:08 -05:00
68 changed files with 15557 additions and 213 deletions

View File

@@ -13,6 +13,7 @@ module that follows.
│ module-uo (>>> HERE <<<) │
│ shard status · spawn atlas · marketplace │
│ governors · cliloc · town crier · uo-link│
│ client files: portraits, item art, names │
└───────────────────────────────────────────┘
│ server half: routers, models, schema fragment
│ client half: prebuilt ESM chunk, SPA routes + nav
@@ -182,7 +183,7 @@ reaches the container.
|---|---|---|
| `UOLINK_BASE_URL` | — | Default sidecar base URL for a site with nothing saved yet. The admin panel's stored value wins. |
| `UOLINK_WS_URL` | — | Same, for the WebSocket URL. |
| `UOLINK_PROTOCOL` | `3` | Wire protocol this build speaks. Again only a fallback — set it lower only if you deliberately run an older sidecar. |
| `UOLINK_PROTOCOL` | `8` | Wire protocol this build speaks. Again only a fallback — set it lower only if you deliberately run an older sidecar. |
| `TOWNCRIER_DURATION_SEC` | `3600` | How long a published news post's in-game town-crier message stays up (≤ `86400`). |
**The sidecar's auth token is deliberately not here.** It is entered in Admin → Shard, encrypted at

View File

@@ -1,6 +1,6 @@
{
"$comment": "The core this module is proved against. MODULE_API.md §5.3: the frozen-manifest job clones RunicGateway/website at this exact ref, drops this module in as modules/uo and runs CORE's own routeManifest.js — nothing else can answer whether the URLs the module claims are the URLs it actually serves. Pinned rather than tracking `edge` on purpose: core moves for reasons that have nothing to do with this module, and a bump is then a deliberate commit saying which core the module was last proved against, instead of an unexplained red X on someone else's PR. Bump it, regenerate routes.manifest.json, and commit both together.",
"$comment": "The core this module is proved against. MODULE_API.md §5.3: the frozen-manifest job clones RunicGateway/website at this exact ref, drops this module in as modules/uo and runs CORE's own routeManifest.js — nothing else can answer whether the URLs the module claims are the URLs it actually serves. Pinned rather than tracking a branch on purpose: core moves for reasons that have nothing to do with this module, and a bump is then a deliberate commit saying which core the module was last proved against, instead of an unexplained red X on someone else's PR. Bump it, regenerate routes.manifest.json, and commit both together. **It pointed at `edge` for the length of the Event System window** (org lead, 2026-09-04), and this commit ends that: `api.registerEventActions` exists only from MODULE_API 1.10.0, so under the previous `main` pin `register()` threw and the module did not load at all — the job would have been red by construction for eight phases and would have proved nothing while a real regression hid behind it. The Phase 16b cutover put 1.10.0 on `main`, so the pin comes home, and this is the same move that turns the Integration kit green again. **routes.manifest.json needed NO regeneration**: the job's own steps were run against this exact ref and answered `routes.manifest.json is current — 73 routes, all documented`, so the \"commit both together\" instruction above had nothing to pair with this time.",
"repo": "https://gitea.whitlocktech.com/RunicGateway/website.git",
"ref": "52eac24d170adbeb7cfb06486bc7512da1173ef9",
"refName": "edge @ MODULE_API 1.9.0, engagement Phase 11b (website#179)"
"ref": "655fbf3f69a6a1fd650ecbc81afd6cf9c2ad9f66",
"refName": "main @ MODULE_API 1.10.0, the Event System cutover (website#199)"
}

View File

@@ -164,6 +164,38 @@ export const admin = {
setPath: (path) => req('/admin/shard/atlas/path', { method: 'PUT', body: { path } }),
},
// The Asset Bridge (docs/link/v8.md §6, §14 — protocol 8 phase 8). Client
// artwork and the cliloc table both come off the operator's own UO client, over
// the same bridge, and boot deliberately never asks the shard for either — so
// these calls are the only thing that imports them, and the panel that makes
// them is where an operator goes after patching their client.
//
// `update` and `reimport` are §6's two stages rather than one call with a flag,
// because they cost wildly different things: an Update that finds the client
// files unchanged transfers nothing, and a re-import fetches every sprite in
// the catalogue. A checkbox spells that difference the same size as the button.
assets: {
status: () => req('/admin/shard/assets'),
update: (approve = false) =>
req('/admin/shard/assets/import', { method: 'POST', body: { approve } }),
reimport: (approve = false) =>
req('/admin/shard/assets/import', { method: 'POST', body: { force: true, approve } }),
// Item and land pictures, which arrive one at a time because a page asked for
// one. The pass runs on its own timer; this is for the operator who has just
// patched a client and would rather not wait for the interval.
warm: (force = false) => req('/admin/shard/assets/warm', { method: 'POST', body: { force } }),
},
clilocs: {
status: () => req('/admin/shard/clilocs'),
import: (opts = {}) =>
req('/admin/shard/clilocs/import', {
method: 'POST',
body: { force: !!opts.force, approve: !!opts.approve },
}),
setPath: (path) => req('/admin/shard/clilocs/path', { method: 'PUT', body: { path } }),
},
// In-game staff operations: write plane + support queue (admin/moderator).
// `actor` is stamped server-side from the session — never sent from here.
shardOps: {

View File

@@ -7,6 +7,7 @@
// player can reach is safe.
import ShardAccountActions from './ShardAccountActions.jsx'
import ItemIcon from './ItemIcon'
const RESIST_LABELS = { phys: 'Physical', fire: 'Fire', cold: 'Cold', pois: 'Poison', energy: 'Energy' }
@@ -270,7 +271,16 @@ export default function CharacterSheet({ char, moderation = false }) {
const detail = [label === layer ? null : layer, `id ${it.itemId}`, it.hue ? `hue ${it.hue}` : null]
return (
<div key={it.serial} style={{ display: 'flex', alignItems: 'center', gap: 12, padding: '10px 14px', border: '1px solid var(--line)', borderRadius: 8 }}>
<span style={{ flex: 'none', width: 22, height: 22, borderRadius: 5, border: '1px solid var(--line)', background: 'rgba(255,255,255,0.05)' }} />
{/* The sheet has always drawn an empty swatch here to hold the
row's alignment. As of phase 5 the shard can hand over the
item's real picture, hued the way the client would draw it —
so the swatch becomes the fallback rather than the only
state, and a row with no picture looks exactly as it did. */}
{it.art ? (
<ItemIcon art={it.art} name={label} size={22} />
) : (
<span style={{ flex: 'none', width: 22, height: 22, borderRadius: 5, border: '1px solid var(--line)', background: 'rgba(255,255,255,0.05)' }} />
)}
<div style={{ flex: 1, minWidth: 0 }}>
<div className="sans" style={{ color: 'var(--head)', fontSize: '0.88rem' }}>{label}</div>
<div className="sans dim" style={{ fontSize: '0.74rem' }}>{detail.filter(Boolean).join(' · ')}</div>

View File

@@ -0,0 +1,31 @@
// ── A label/value line in an admin detail panel ────────────────────────────
//
// Extracted from `SpawnAtlas.jsx` in phase 8, when the Client Files panel needed
// the same thing for the third time. Two copies of twenty lines is a coincidence;
// three is a component, and the reason to make it one here rather than later is
// that these lines are read side by side — an operator moves between Spawn Atlas
// and Client Files doing one job, and a panel whose rows are a few pixels off
// from its neighbour's looks like a different part of the product.
//
// Deliberately not styled through a class: this module ships as a prebuilt chunk
// into core's SPA and owns no stylesheet there (MODULE_API.md §3.2), so its own
// layout is inline and only core's theme VARIABLES are borrowed.
export default function DetailRow({ label, children }) {
return (
<div
className="sans"
style={{
display: 'flex',
alignItems: 'baseline',
justifyContent: 'space-between',
gap: 16,
padding: '7px 0',
borderBottom: '1px solid var(--line)',
fontSize: '0.86rem',
}}
>
<span className="dim">{label}</span>
<span style={{ color: 'var(--head)', textAlign: 'right', wordBreak: 'break-all' }}>{children}</span>
</div>
)
}

View File

@@ -0,0 +1,47 @@
// One item's picture, when this site holds one (docs/link/v8.md §5, §11 — phase 5).
//
// `art` is a FILENAME under uploads/items/, never a path or a URL — the same
// shape `CreaturePortrait` takes, so there is one place in this module that knows
// where uploads are mounted rather than one per surface.
//
// **NULL is ordinary and permanent for some items, and this renders nothing for
// it.** Three separate reasons an item has no picture, and none of them is a
// fault: the site has no shard link and never fetched one; the warm pass has not
// reached this key yet (pictures are fetched behind the page, never by it, so a
// new listing shows text first and gains its icon a few minutes later); or the
// operator's own client simply has no art at that id — 9,963 of a stock client's
// static ids have an empty index entry. Every layout using this is written to sit
// correctly with the icon absent, because that is the state all of them were
// built in.
//
// A hued item is a DIFFERENT picture, not a tinted one: the shard applies the hue
// out of `hues.mul` before it sends anything, because whether a hue repaints the
// whole sprite or only its grey pixels is decided by a flag in `tiledata.mul`
// that this browser has no way to read. So there is nothing to style here — the
// bytes already are the right colour.
//
// `imageRendering: 'pixelated'` for the same reason the creature portraits use
// it: UO art is pixel art, and a browser's default smoothing turns a 22×26
// item into a smear at any size above its own.
export default function ItemIcon({ art, name, size = 32 }) {
if (!art) return null
return (
<img
src={`/uploads/items/${encodeURIComponent(art)}`}
alt=""
// Decorative: the item's name is already beside it as text, and an alt
// repeating it would make a screen reader say it twice.
aria-hidden="true"
loading="lazy"
style={{
width: size,
height: size,
flex: 'none',
objectFit: 'contain',
imageRendering: 'pixelated',
}}
title={name}
/>
)
}

View File

@@ -41,6 +41,7 @@ import ShardAdmin from './routes/admin/ShardAdmin.jsx'
import ShardOps from './routes/admin/ShardOps.jsx'
import ShardVisibility from './routes/admin/ShardVisibility.jsx'
import SpawnAtlas from './routes/admin/SpawnAtlas.jsx'
import ClientFiles from './routes/admin/ClientFiles.jsx'
import HousesAdmin from './routes/admin/HousesAdmin.jsx'
import AdminCharacters from './routes/admin/AdminCharacters.jsx'
import AdminCharacter from './routes/admin/AdminCharacter.jsx'
@@ -93,12 +94,14 @@ registry.registerRoutes(ID, {
{ path: 'market/vendors/:serial', element: <MarketVendor /> },
],
admin: [
// Admin-only: the sidecar's configuration, who may see which surface, and
// the atlas import. No `gate` on the other three because AdminLayout already
// requires staff and these carry their own role rows below.
// Admin-only: the sidecar's configuration, who may see which surface, the
// atlas import and the client-file imports. No `gate` on these four because
// AdminLayout already requires staff and they carry their own role rows
// below.
{ path: 'link', element: <ShardAdmin /> },
{ path: 'visibility', element: <ShardVisibility /> },
{ path: 'atlas', element: <SpawnAtlas /> },
{ path: 'files', element: <ClientFiles /> },
{ path: 'ops', element: <ShardOps />, gate: STAFF },
{ path: 'houses', element: <HousesAdmin />, gate: STAFF },
// Self-service, and deliberately ungated: a staff member's own characters
@@ -150,6 +153,7 @@ registry.registerNav(ID, {
{ label: 'Shard (uo-link)', to: '/admin/uo/link', icon: IconShard, group: 'System', order: 8, roles: ['admin'] },
{ label: 'Shard Visibility', to: '/admin/uo/visibility', icon: IconShard, group: 'System', order: 8, roles: ['admin'] },
{ label: 'Spawn Atlas', to: '/admin/uo/atlas', icon: IconShard, group: 'System', order: 8, roles: ['admin'] },
{ label: 'Client Files', to: '/admin/uo/files', icon: IconShard, group: 'System', order: 8, roles: ['admin'] },
// No group: a trailing untitled group of its own, below core's Account row
// rather than beside it (§3.3). One position lower than it sits today, and
// the alternative — letting a module into core's furniture groups — is worse.

View File

@@ -0,0 +1,663 @@
import { useCallback, useEffect, useRef, useState } from 'react'
import api from '../../api.js'
import { ErrorState, Loading } from '../../core.js'
import Row from '../../components/DetailRow.jsx'
import { CreaturePortrait } from '../public/Atlas.jsx'
// ── Admin · Client files ────────────────────────────────────────────────────
//
// Everything on this site that comes out of the operator's own UO client, and
// the buttons that bring it in (docs/link/v8.md §6, §14 — the Asset Bridge,
// phase 8).
//
// Three things, one page, because they are one job. Creature portraits, item and
// land pictures, and the cliloc table all live in files inside a UO client
// install; the shard decodes them and hands them over the bridge; and every one
// of them changes at the same moment, when the operator patches that client. An
// operator who has just done that has exactly one place to come.
//
// **Boot never asks the shard for any of it** (org lead, phase 2 and again in
// phase 7). A client patch is an event the operator knows about and the website
// does not, and a site that re-read 343 MB of client files on every restart to
// discover nothing had changed would be paying for the rare case forever. The
// consequence is the reason this panel exists at all: these buttons are the ONLY
// thing that imports. Nothing here happens on its own except the item-art warm
// pass, which is lazy by design and only fetches what a page has already asked
// for.
//
// **Nothing on this page throws for an operator-visible problem.** A shard that
// is down, an asset plane switched off, a Linux host with no libgdiplus, a client
// with no cliloc file — each is a reported state with a reason naming what to
// fix. A red box that says "500" would be the one thing an operator cannot act
// on, and every one of these states is ordinary.
// ── outcomes ───────────────────────────────────────────────────────────────
//
// An import reports its result rather than throwing, so these are answers, not
// errors. They are written in the operator's terms — what happened to their
// site — rather than in the protocol's.
const ASSET_OUTCOME = {
imported: (r) =>
`Imported — ${r.written?.toLocaleString() ?? 0} picture(s) written, ` +
`${r.assets?.toLocaleString() ?? 0} in the catalogue, ` +
`${r.bodies?.resolved?.toLocaleString() ?? 0} creature(s) matched to a body.`,
unchanged: () =>
'Unchanged — the shards client files match what was imported, so nothing was transferred.',
needsReview: (r) =>
`Waiting for you: ${r.vanishedCount?.toLocaleString() ?? 0} picture(s) this site holds are no` +
' longer offered by the shard.',
unavailable: (r) => `The shard could not serve this: ${r.reason || 'unknown reason'}`,
skipped: () => 'No shard is linked, so there are no client files to read.',
failed: (r) => `The import failed: ${r.reason || 'unknown reason'}`,
}
// The warm pass speaks the same vocabulary as the body import deliberately
// (`skipped` / `unavailable` / `unchanged` / `imported` / `failed`), but its
// numbers mean something different: it is bounded, so "imported" routinely
// leaves work behind and saying so is the difference between a button that looks
// broken and one that is doing what it promised.
const WARM_OUTCOME = {
imported: (r) =>
`Fetched ${r.written?.toLocaleString() ?? 0} picture(s)` +
(r.remaining ? `; ${r.remaining.toLocaleString()} still waiting — press again.` : '.'),
unchanged: () => 'Nothing waiting — every picture a page has asked for is already here.',
unavailable: (r) => `The shard could not serve this: ${r.reason || 'unknown reason'}`,
skipped: () => 'No shard is linked, so there is nothing to fetch.',
failed: (r) => `That did not work: ${r.reason || 'unknown reason'}`,
}
const CLILOC_OUTCOME = {
imported: (r) => `Imported — ${r.count?.toLocaleString() ?? 0} names loaded.`,
unchanged: () => 'Unchanged — the source matches the table that is already loaded.',
needsReview: (r) =>
`Waiting for you: ${r.missingSources?.length ?? 0} overlay file(s) that were loaded last time` +
' are missing.',
unavailable: (r) => `The source could not be read: ${r.reason || 'unknown reason'}`,
skipped: (r) => r.reason || 'There is no cliloc source configured.',
failed: (r) => `The import failed: ${r.reason || 'unknown reason'}`,
}
const describe = (table, result) =>
(table[result?.status] || (() => `Result: ${result?.status}`))(result || {})
const num = (n) => (n == null ? '—' : Number(n).toLocaleString())
const when = (v) => (v ? new Date(v).toLocaleString() : 'Never')
// ── the vanished-key review (§6) ───────────────────────────────────────────
//
// A key the site holds that the shard no longer offers is refused rather than
// applied, because an unmounted client volume and a deliberate client downgrade
// are the same thing from the server and the wrong guess deletes artwork.
//
// It is held in this component's state and not in a table, deliberately (org
// lead, 2026-09-14). The atlas persists its equivalent because BOOT re-parses the
// tree and would otherwise re-prompt on every restart forever; an asset import
// only ever happens because somebody pressed a button on this page, so the
// review is in front of the person who caused it, by construction. Declining is
// therefore not a decision to remember — it is simply not pressing the other
// button.
//
// The pictures matter. `body/820/a23` names nothing a human recognises; the horse
// it is a picture of does, and "is it right that these disappear?" is not a
// question anyone can answer from a list of keys.
function VanishedReview({ review, busy, onApprove, onDismiss }) {
const rows = review.result.vanished || []
const total = review.result.vanishedCount ?? rows.length
return (
<section
style={{
border: '1px solid #c58f4a',
borderRadius: 10,
padding: 16,
background: 'rgba(197,143,74,0.08)',
}}
>
<h3 className="display" style={{ margin: 0, fontSize: '1rem', color: 'var(--head)' }}>
An import is waiting for you
</h3>
<p className="sans" style={{ margin: '6px 0 12px', fontSize: '0.86rem', color: 'var(--muted)', lineHeight: 1.6 }}>
The shard no longer offers <strong>{num(total)}</strong> picture{total === 1 ? '' : 's'} this
site is currently serving, so nothing was changed. That is what a client volume that failed
to mount looks like as well as a deliberate client downgrade, and only you can tell them
apart. Approving re-reads the shard as it is right now if the mount was the problem and you
have since fixed it, what lands is the corrected import, not a deletion.
</p>
<div
style={{
display: 'flex',
flexWrap: 'wrap',
gap: 10,
maxHeight: 260,
overflowY: 'auto',
padding: '4px 0',
}}
>
{rows.map((row) => (
<div key={row.key} style={{ width: 96, textAlign: 'center' }}>
<CreaturePortrait art={row.file} name={row.key} size={48} />
<div
className="sans dim"
style={{ fontSize: '0.7rem', wordBreak: 'break-all', marginTop: 2 }}
title={row.key}
>
{row.key}
</div>
</div>
))}
</div>
{total > rows.length && (
<p className="sans dim" style={{ margin: '10px 0 0', fontSize: '0.8rem' }}>
Showing the first {num(rows.length)} of {num(total)}.
</p>
)}
<div style={{ display: 'flex', gap: 10, marginTop: 14, flexWrap: 'wrap' }}>
<button type="button" className="btn btn-primary btn-sq" disabled={busy} onClick={onApprove}>
Approve and import
</button>
<button type="button" className="btn btn-sq" disabled={busy} onClick={onDismiss}>
Keep the pictures I have
</button>
</div>
</section>
)
}
// What the last import did. Core's activity log records the same action, but it
// is one unfiltered list of every admin action on the site — so the answer to
// "did last week's import actually do anything" is here, beside the button that
// caused it, rather than twenty pages into a log.
function LastImport({ last, at }) {
if (!last) {
return <Row label="Last import">{at ? when(at) : 'No import recorded yet'}</Row>
}
const tally = last.bodies || {}
const unmatched = [
tally.unknown ? `${num(tally.unknown)} unknown to the shard` : '',
tally.notCreature ? `${num(tally.notCreature)} not a creature` : '',
tally.failed ? `${num(tally.failed)} failed` : '',
].filter(Boolean)
return (
<>
<Row label="Last import">
{`${when(last.at || at)}${last.by ? ` · ${last.by}` : ''}${last.force ? ' · full re-import' : ''}`}
</Row>
<Row label="Pictures written">
{`${num(last.written)} written, ${num(last.fetched)} fetched`}
{last.removed ? `, ${num(last.removed)} removed` : ''}
</Row>
{unmatched.length > 0 && (
// Only the creatures that did NOT match, because how many did is the row
// above this block and a number that means "now" should not also appear
// as a number that means "at that import". What is left is the part an
// operator can act on: `unknown` is a spawn file naming a type this
// shard's scripts do not define, which is real drift.
<Row label="Could not be matched">{unmatched.join(', ')}</Row>
)}
</>
)
}
export default function ClientFiles() {
const [assets, setAssets] = useState(null)
const [clilocs, setClilocs] = useState(null)
const [clilocPath, setClilocPath] = useState('')
const [loading, setLoading] = useState(true)
const [error, setError] = useState('')
const [busy, setBusy] = useState(false)
// One message per section: three panels that can each speak means an operator
// must never have to work out which button a sentence belongs to.
const [msg, setMsg] = useState({})
// The in-session reviews, keyed by which plane raised them.
const [review, setReview] = useState({})
// `quiet` re-reads without flipping `loading`, and that distinction is the
// whole difference between a usable panel and a maddening one: `loading`
// replaces the page with a spinner, so refreshing that way after an action
// unmounts everything, throws the operator back to the top of a long page, and
// takes the sentence saying what just happened with it — at the bottom of the
// cliloc section, that means pressing Update appears to do nothing at all.
const load = useCallback(async ({ quiet = false } = {}) => {
if (!quiet) setLoading(true)
setError('')
try {
// Both statuses call the shard, and neither one failing should cost the
// other its panel: an operator whose cliloc file is missing still needs to
// see what the asset import says.
const [a, c] = await Promise.all([
api.admin.assets.status().catch((err) => ({ error: err.message })),
api.admin.clilocs.status().catch((err) => ({ error: err.message })),
])
setAssets(a)
setClilocs(c)
setClilocPath(c?.path || '')
} catch (err) {
setError(err.message || 'Could not load the client-file status.')
} finally {
if (!quiet) setLoading(false)
}
}, [])
useEffect(() => {
load()
}, [load])
// One automatic re-read when the shard answered BUSY (§3.2's single slot),
// and exactly one per mount.
//
// BUSY is not a fault and it is not sticky on the shard — it means something
// else held the asset slot for longer than the client's own 425 backoff, and
// the two things that hold it are both ordinary: an import the operator
// started, and the item-art warm pass refilling itself after a client patch.
// The panel does not poll, so without this the operator is left reading a
// refusal about a shard that was free again seconds later, until they think to
// reload. A second read clears the common case; if it is still busy, the
// sentence says to come back, because a page that retried forever would be
// holding the slot it is waiting for.
const retried = useRef(false)
useEffect(() => {
if (retried.current || busy) return
const stillBusy = assets?.code === 'BUSY' || clilocs?.code === 'BUSY'
if (!stillBusy) return
retried.current = true
const t = setTimeout(() => load({ quiet: true }), 4000)
return () => clearTimeout(t)
}, [assets, clilocs, busy, load])
// Every action shares this: run it, say what it said, then re-read status so
// the panel reflects the world rather than what we assumed happened.
async function run(section, table, fn) {
setBusy(true)
setMsg((m) => ({ ...m, [section]: '' }))
setError('')
try {
const result = await fn()
setMsg((m) => ({ ...m, [section]: describe(table, result) }))
// Set or cleared from the SAME answer, in one place. Clearing separately
// left the review standing after an approve that had already applied — a
// banner asking for a decision that was made ten seconds ago, on pictures
// that are already gone.
setReview((r) => ({
...r,
[section]: result?.status === 'needsReview' ? { result, run: fn } : null,
}))
await load({ quiet: true })
return result
} catch (err) {
setError(err.message || 'That did not work.')
return null
} finally {
setBusy(false)
}
}
async function saveClilocPath() {
setBusy(true)
setMsg((m) => ({ ...m, clilocs: '' }))
setError('')
try {
const fresh = await api.admin.clilocs.setPath(clilocPath.trim())
setClilocs(fresh)
setClilocPath(fresh.path || '')
setMsg((m) => ({
...m,
clilocs:
fresh.source === 'bridge'
? 'Saved. The base table still comes from the shard — this selects where custom/ overlay' +
' files are read from.'
: fresh.path === ''
? 'Path cleared. The loaded table keeps serving; nothing new will be read.'
: fresh.fileReadable
? 'Saved. The file is readable — import when you are ready.'
: 'Saved, but the file could not be read from here. Check the mount and permissions.',
}))
} catch (err) {
setError(err.message || 'Could not save the path.')
} finally {
setBusy(false)
}
}
if (loading) return <Loading />
if (error && !assets && !clilocs) return <ErrorState message={error} />
const loaded = assets?.loaded || null
const shard = assets?.shard || null
const families = shard?.families || []
// Reported by the server rather than inferred from `shard` being null — which
// is also what a linked shard that is simply DOWN looks like, and those two
// want opposite things from this page: one needs its buttons disabled, the
// other needs them available so the operator can retry.
const linked = Boolean(assets?.linked)
const imagingBroken = shard?.imaging && shard.imaging.ok === false
return (
<div style={{ display: 'flex', flexDirection: 'column', gap: 20 }}>
<header>
<h2 className="display" style={{ margin: 0, fontSize: '1.3rem', color: 'var(--head)' }}>
Client files
</h2>
<p className="sans" style={{ margin: '6px 0 0', color: 'var(--muted)', fontSize: '0.88rem', lineHeight: 1.6, maxWidth: 760 }}>
Creature portraits, item pictures and the names your shards items and titles are stored
under all come out of the UO client on the shard host. The shard reads and decodes them
itself and hands them over uo-link nothing is converted on a desktop and nothing is
uploaded. They change when you patch that client, which is something only you know about,
so <strong>these buttons are the only thing that imports them</strong>: nothing here
happens on a restart.
</p>
</header>
{(assets?.error || clilocs?.error) && (
<section
style={{ border: '1px solid #d98b84', borderRadius: 10, padding: 16 }}
className="sans"
>
<strong style={{ color: 'var(--head)' }}>Part of this page could not be read.</strong>
<p style={{ margin: '6px 0 0', color: 'var(--muted)', fontSize: '0.86rem', lineHeight: 1.6 }}>
{assets?.error || clilocs?.error} the counts below may be missing. Both status calls
are written never to fail for an ordinary problem (a shard that is down is an ANSWER
here), so this one is worth the server log.
</p>
</section>
)}
{assets?.reason && !shard && (
<section
style={{ border: '1px solid var(--line)', borderRadius: 10, padding: 16 }}
className="sans"
>
{/* BUSY is the one code here that is not a fault, and saying "the shard
is not answering" about it sends an operator to check a shard that is
working. The slot is held by something ordinary — an import running,
or the warm pass — and it frees itself. */}
<strong style={{ color: 'var(--head)' }}>
{assets.code === 'BUSY'
? 'The shard is busy with another client-file request.'
: 'The shard is not answering for client files.'}
</strong>
<p style={{ margin: '6px 0 0', color: 'var(--muted)', fontSize: '0.86rem', lineHeight: 1.6 }}>
{assets.code === 'BUSY'
? 'The shard serves one of these at a time, so an import running now — or the' +
' item-picture pass refilling itself after a client patch — holds it until it is' +
' done. This page re-reads once on its own; if the counts below are still missing' +
' after that, reload in a moment.'
: assets.reason}
{assets.code === 'DISABLED' &&
' — set Bridge.AssetsEnabled on the shard to allow it to read its own client files.'}
</p>
<p style={{ margin: '6px 0 0', color: 'var(--muted)', fontSize: '0.86rem', lineHeight: 1.6 }}>
What is already imported keeps serving; only new imports are affected.
</p>
</section>
)}
{imagingBroken && (
<section
style={{ border: '1px solid #c58f4a', borderRadius: 10, padding: 16, background: 'rgba(197,143,74,0.08)' }}
className="sans"
>
<strong style={{ color: 'var(--head)' }}>The shard host cannot render images.</strong>
<p style={{ margin: '6px 0 0', color: 'var(--muted)', fontSize: '0.86rem', lineHeight: 1.6 }}>
{shard.imaging.reason ||
'A Linux shard host needs libgdiplus before it can decode a single sprite.'}{' '}
Names (the cliloc table) are unaffected and can still be imported they have no pixels
in them.
</p>
</section>
)}
{review.assets && (
<VanishedReview
review={review.assets}
busy={busy}
onApprove={() => run('assets', ASSET_OUTCOME, () => review.assets.run(true))}
onDismiss={() => setReview((r) => ({ ...r, assets: null }))}
/>
)}
{/* ── creature portraits ── */}
<section style={{ border: '1px solid var(--line)', borderRadius: 10, padding: 16 }}>
<h3 className="display" style={{ margin: '0 0 4px', fontSize: '1rem', color: 'var(--head)' }}>
Creature portraits
</h3>
<p className="sans" style={{ margin: '0 0 12px', fontSize: '0.84rem', color: 'var(--muted)', lineHeight: 1.6 }}>
One picture per creature body, imported as a set and shown on the bestiary. Creatures the
client has no artwork for are normal and stay as text a stock client has none for most
ghost and gargoyle bodies. Portraits you drew yourself and named in{' '}
<code>spawnAtlas.art.json</code> always win over an imported one.
</p>
<Row label="Pictures held">{`${num(loaded?.stored)} of ${num(loaded?.assets)} catalogued`}</Row>
<Row label="Creatures matched">{`${num(loaded?.resolved)} of ${num(loaded?.creatures)}`}</Row>
<LastImport last={loaded?.last} at={loaded?.importedAt} />
<Row label="Client files changed since">
{assets?.drift == null
? '—'
: assets.drift
? 'Yes — an update would pick it up'
: 'No'}
</Row>
{shard?.hashing && (
<Row label="Shard is hashing">
Yes it is still fingerprinting its client files in the background. Drift may read as
yes until it finishes.
</Row>
)}
<Row label="Extractor version">
{/* "—" for a version nobody has imported yet reads as a missing value;
it is an answer, and the shard's own version is the useful half of
the sentence on exactly that install. */}
{(loaded?.extractorVersion == null ? 'None' : num(loaded.extractorVersion)) +
' imported' +
(shard?.extractorVersion == null ? '' : ` · ${num(shard.extractorVersion)} on the shard`)}
</Row>
<div style={{ display: 'flex', gap: 12, flexWrap: 'wrap', alignItems: 'center', marginTop: 14 }}>
<button
type="button"
className="btn btn-primary btn-sq"
disabled={busy || !linked}
onClick={() => run('assets', ASSET_OUTCOME, (approve = false) => api.admin.assets.update(approve))}
>
{busy ? 'Working…' : 'Update'}
</button>
<button
type="button"
className="btn btn-sq"
disabled={busy || !linked}
onClick={() => run('assets', ASSET_OUTCOME, (approve = false) => api.admin.assets.reimport(approve))}
>
Re-import everything
</button>
</div>
<p className="sans dim" style={{ margin: '10px 0 0', fontSize: '0.8rem', lineHeight: 1.6 }}>
<strong>Update</strong> checks the shards client files first and transfers only the
pictures that actually changed when nothing has, it costs one small round trip.{' '}
<strong>Re-import everything</strong> fetches the whole catalogue again; use it after
restoring a backup or losing the uploads volume, where the database still remembers
pictures that are no longer on disk.
</p>
{msg.assets && (
<p className="sans" style={{ margin: '10px 0 0', fontSize: '0.85rem', color: '#7fd0a4' }}>{msg.assets}</p>
)}
</section>
{/* ── item and land pictures ── */}
<section style={{ border: '1px solid var(--line)', borderRadius: 10, padding: 16 }}>
<h3 className="display" style={{ margin: '0 0 4px', fontSize: '1rem', color: 'var(--head)' }}>
Item and land pictures
</h3>
<p className="sans" style={{ margin: '0 0 12px', fontSize: '0.84rem', color: 'var(--muted)', lineHeight: 1.6 }}>
The pictures beside marketplace listings and on character sheets. These are never imported
as a set there are tens of thousands of item graphics, times every dye colour so they
arrive one at a time, shortly after a page asks for one, and refresh themselves after a
client patch. This is here for the two moments waiting is the wrong answer: you have just
linked a shard, or you have just patched a client and would rather not wait.
</p>
<Row label="Item pictures held">{num(loaded?.items)}</Row>
<Row label="Land pictures held">{num(loaded?.land)}</Row>
<Row label="Shard serves">
{families.length > 0 ? families.join(', ') : '—'}
{shard && !families.includes('static')
? ' — this shards plugin predates item pictures; update the overlay to get them'
: ''}
</Row>
<div style={{ display: 'flex', gap: 12, flexWrap: 'wrap', alignItems: 'center', marginTop: 14 }}>
<button
type="button"
className="btn btn-sq"
disabled={busy || !linked}
onClick={() => run('warm', WARM_OUTCOME, () => api.admin.assets.warm(false))}
>
Fetch waiting pictures
</button>
<button
type="button"
className="btn btn-sq"
disabled={busy || !linked}
onClick={() => run('warm', WARM_OUTCOME, () => api.admin.assets.warm(true))}
>
Refresh the ones I have
</button>
</div>
{msg.warm && (
<p className="sans" style={{ margin: '10px 0 0', fontSize: '0.85rem', color: '#7fd0a4' }}>{msg.warm}</p>
)}
</section>
{/* ── the cliloc table ── */}
<section style={{ border: '1px solid var(--line)', borderRadius: 10, padding: 16 }}>
<h3 className="display" style={{ margin: '0 0 4px', fontSize: '1rem', color: 'var(--head)' }}>
Item and title names (clilocs)
</h3>
<p className="sans" style={{ margin: '0 0 12px', fontSize: '0.84rem', color: 'var(--muted)', lineHeight: 1.6 }}>
UO stores most item, title and reward names as numbers, and the words live in the clients
cliloc file. Without this table the marketplace and character sheets show numbers. With a
shard linked the shard decompresses and serves it; otherwise the site reads a file you
point it at below.
</p>
<Row label="Names loaded">{num(clilocs?.count)}</Row>
<Row label="Imported">{when(clilocs?.importedAt)}</Row>
<Row label="Source">
{clilocs?.source === 'bridge'
? 'The shard, over uo-link'
: clilocs?.configured
? clilocs.path
: 'None configured'}
</Row>
<Row label="Overlays">
{clilocs?.sources?.length ? clilocs.sources.join(', ') : 'None'}
</Row>
<Row label="Changed since import">
{clilocs?.drift == null ? '—' : clilocs.drift ? 'Yes — an import would pick it up' : 'No'}
</Row>
{clilocs?.problem && (
<Row label="Problem">
<span style={{ color: '#d98b84' }}>{clilocs.problem}</span>
</Row>
)}
{clilocs?.missingSources?.length > 0 && (
<Row label="Missing since last import">
<span style={{ color: '#d98b84' }}>{clilocs.missingSources.join(', ')}</span>
</Row>
)}
<div style={{ display: 'flex', gap: 12, flexWrap: 'wrap', alignItems: 'center', marginTop: 14 }}>
<button
type="button"
className="btn btn-primary btn-sq"
disabled={busy}
onClick={() =>
run('clilocs', CLILOC_OUTCOME, (approve = false) =>
api.admin.clilocs.import({ approve }),
)
}
>
{busy ? 'Working…' : 'Update'}
</button>
<button
type="button"
className="btn btn-sq"
disabled={busy}
onClick={() =>
run('clilocs', CLILOC_OUTCOME, (approve = false) =>
api.admin.clilocs.import({ force: true, approve }),
)
}
>
Re-import everything
</button>
</div>
{review.clilocs && (
<div
style={{
marginTop: 14,
border: '1px solid #c58f4a',
borderRadius: 10,
padding: 14,
background: 'rgba(197,143,74,0.08)',
}}
>
<strong className="sans" style={{ color: 'var(--head)', fontSize: '0.9rem' }}>
An overlay file that was loaded last time is missing
</strong>
<p className="sans" style={{ margin: '6px 0 10px', fontSize: '0.85rem', color: 'var(--muted)', lineHeight: 1.6 }}>
{(review.clilocs.result.missingSources || []).join(', ') || 'One or more overlays'}
the table was left exactly as it is. If you deleted those files on purpose, import
anyway; if this is a mount that did not come back, fix it first and the next import
picks the names up again.
</p>
<div style={{ display: 'flex', gap: 10, flexWrap: 'wrap' }}>
<button
type="button"
className="btn btn-primary btn-sq"
disabled={busy}
onClick={() =>
run('clilocs', CLILOC_OUTCOME, () => review.clilocs.run(true))
}
>
Import without them
</button>
<button
type="button"
className="btn btn-sq"
disabled={busy}
onClick={() => setReview((r) => ({ ...r, clilocs: null }))}
>
Keep the names I have
</button>
</div>
</div>
)}
<div style={{ marginTop: 16 }}>
<p className="sans dim" style={{ margin: '0 0 8px', fontSize: '0.8rem', lineHeight: 1.6 }}>
{clilocs?.source === 'bridge'
? 'Where custom/ overlay files are read from. The base table comes from the shard' +
' either way; leave this blank if you have no overlays.'
: 'The directory holding the cliloc file. Blank turns cliloc resolution off — the' +
' table that is already loaded keeps serving.'}
</p>
<div style={{ display: 'flex', gap: 10, flexWrap: 'wrap', alignItems: 'center' }}>
<input
className="input"
value={clilocPath}
onChange={(e) => setClilocPath(e.target.value)}
placeholder="/srv/uo-client"
style={{ flex: '1 1 320px', minWidth: 0 }}
/>
<button type="button" className="btn btn-sq" disabled={busy} onClick={saveClilocPath}>
Save path
</button>
</div>
</div>
{msg.clilocs && (
<p className="sans" style={{ margin: '10px 0 0', fontSize: '0.85rem', color: '#7fd0a4' }}>{msg.clilocs}</p>
)}
</section>
{error && (
<span className="sans" style={{ color: '#d98b84', fontSize: '0.85rem' }}>{error}</span>
)}
</div>
)
}

View File

@@ -1,6 +1,7 @@
import { useCallback, useEffect, useState } from 'react'
import api from '../../api.js'
import { ErrorState, Loading } from '../../core.js'
import Row from '../../components/DetailRow.jsx'
// ── Admin · Spawn atlas ─────────────────────────────────────────────────────
//
@@ -36,26 +37,6 @@ const OUTCOME = {
const describe = (result) => (OUTCOME[result?.status] || (() => `Result: ${result?.status}`))(result)
function Row({ label, children }) {
return (
<div
className="sans"
style={{
display: 'flex',
alignItems: 'baseline',
justifyContent: 'space-between',
gap: 16,
padding: '7px 0',
borderBottom: '1px solid var(--line)',
fontSize: '0.86rem',
}}
>
<span className="dim">{label}</span>
<span style={{ color: 'var(--head)', textAlign: 'right', wordBreak: 'break-all' }}>{children}</span>
</div>
)
}
function PendingReview({ pending, busy, onApprove, onReject }) {
const declined = pending.status === 'rejected'
return (
@@ -159,11 +140,14 @@ export default function SpawnAtlas() {
setStatus(fresh)
setPath(fresh.path || '')
setMsg(
fresh.path === ''
? 'Path cleared. The atlas will be skipped on the next boot; what is loaded keeps serving.'
: fresh.treeReadable
? 'Saved. The tree is readable — import when you are ready.'
: 'Saved, but the tree could not be read from here. Check the mount and permissions.',
fresh.source === 'bridge'
? 'Saved, but not in use: this site reads the atlas from the linked shard. The path takes'
+ ' over only if uo-link is disabled.'
: fresh.path === ''
? 'Path cleared. The atlas will be skipped on the next boot; what is loaded keeps serving.'
: fresh.treeReadable
? 'Saved. The tree is readable — import when you are ready.'
: 'Saved, but the tree could not be read from here. Check the mount and permissions.',
)
} catch (err) {
setError(err.message || 'Could not save the path.')
@@ -185,9 +169,11 @@ export default function SpawnAtlas() {
</h2>
<p className="sans" style={{ margin: '6px 0 0', color: 'var(--muted)', fontSize: '0.88rem', lineHeight: 1.6, maxWidth: 760 }}>
The bestiary and spawn map on the public site, parsed from the shards own ServUO files.
It refreshes itself on every server start; everything here is for the times you dont want
to wait for one. Nothing on this page touches the sidecar the atlas is shard content, not
shard state, and stays complete while the shard is down.
Where those files come from depends on whether a shard is linked: with uo-link configured
the shard serves them over the bridge and importing is something you do here, when a map
changes. Without one, the site reads a local tree and re-imports itself on every server
start. Either way the atlas is shard <em>content</em> rather than shard state, so what is
loaded keeps serving in full while the shard is down.
</p>
</header>
@@ -218,10 +204,23 @@ export default function SpawnAtlas() {
<Row label="Champion altars">{counts.champions?.toLocaleString() ?? '—'}</Row>
</>
)}
<Row label="Tree readable">
{!status?.configured ? 'No path set' : status.treeReadable ? 'Yes' : 'No'}
<Row label="Source">
{status?.source === 'bridge'
? 'The shard, over uo-link'
: status?.configured
? status.path
: 'None — no shard linked and no path set'}
</Row>
<Row label="Tree changed since import">
<Row label="Source readable">
{!status?.configured
? 'No source'
: status.treeReadable
? 'Yes'
: status.source === 'bridge'
? 'No — the shard did not answer, or Bridge.TreeEnabled is off'
: 'No'}
</Row>
<Row label="Changed since import">
{status?.drift == null ? '—' : status.drift ? 'Yes — an import would pick it up' : 'No'}
</Row>
</section>
@@ -231,9 +230,13 @@ export default function SpawnAtlas() {
ServUO tree
</h3>
<p className="sans" style={{ margin: '0 0 12px', fontSize: '0.84rem', color: 'var(--muted)', lineHeight: 1.6 }}>
Where the website reads the shards spawn files from the same host, a bind mount or a
A local ServUO tree the website can read directly the same host, a bind mount or a
shared volume. This setting wins over the <code>SERVUO_PATH</code> deploy default, so the
mount can move without a redeploy. Leave it blank to turn the atlas off.
mount can move without a redeploy.
{status?.source === 'bridge'
? ' It is not in use right now: this site has a shard linked, and the shard serves its' +
' own files over the bridge. Unlink or disable uo-link to fall back to a path.'
: ' Leave it blank to turn the atlas off.'}
</p>
<div style={{ display: 'flex', gap: 10, flexWrap: 'wrap', alignItems: 'center' }}>
<input
@@ -254,9 +257,11 @@ export default function SpawnAtlas() {
Re-import
</h3>
<p className="sans" style={{ margin: '0 0 12px', fontSize: '0.84rem', color: 'var(--muted)', lineHeight: 1.6 }}>
Applies a map change without restarting. An unchanged tree costs nothing the source files
are hashed first and skipped when they match. A refresh that would remove a facet still
comes back here for approval rather than being applied.
Applies a map change without restarting and on a linked shard it is the only thing that
does, because boot deliberately never calls the shard for this. An unchanged source costs
almost nothing: the file list and its hashes are read first (about 32 KB over the bridge)
and no file is transferred when they match. A refresh that would remove a facet still comes
back here for approval rather than being applied.
</p>
<div style={{ display: 'flex', gap: 12, flexWrap: 'wrap', alignItems: 'center' }}>
<button

View File

@@ -45,6 +45,46 @@ function Chip({ active, onClick, children }) {
)
}
// One creature's portrait, when there is one.
//
// `art` is a FILENAME under uploads/atlas/, never a path or a URL: it is either a
// sprite the shard extracted from the operator's own UO client (docs/link/v8.md
// §12) or a picture the operator drew and named in `spawnAtlas.art.json`, and the
// two are indistinguishable here on purpose.
//
// **NULL is the ordinary case and always will be.** An install with no shard link
// has never imported one; a shard whose host cannot render images has none; and
// even on a complete import, two thirds of the playable ghost and gargoyle bodies
// have no art in the client at all (§5.2). So this renders nothing rather than a
// placeholder, and every layout around it is written to sit correctly with the
// picture absent — which is the state the whole atlas was designed in.
//
// Sprites are small (a couple of dozen pixels square) and UO's art is pixel art,
// so `imageRendering: 'pixelated'` matters: a browser's default smoothing turns a
// 24×63 wolf into a smear at any size above its own.
export function CreaturePortrait({ art, name, size = 40 }) {
if (!art) return null
return (
<img
src={`/uploads/atlas/${encodeURIComponent(art)}`}
alt=""
// Decorative: the creature's name is already beside it as text, so an alt
// repeating it would make a screen reader say it twice.
aria-hidden="true"
loading="lazy"
style={{
width: size,
height: size,
flex: 'none',
objectFit: 'contain',
imageRendering: 'pixelated',
}}
title={name}
/>
)
}
function CreatureCard({ creature }) {
const facets = Object.entries(creature.facets || {}).sort((a, b) => b[1] - a[1])
return (
@@ -60,6 +100,7 @@ function CreatureCard({ creature }) {
color: 'inherit',
}}
>
<CreaturePortrait art={creature.art} name={creature.name} />
<div style={{ minWidth: 0, flex: 1 }}>
<div
className="display"

View File

@@ -2,6 +2,7 @@ import { useMemo, useState } from 'react'
import { Link, useParams } from 'react-router-dom'
import api from '../../api.js'
import { EmptyState, ErrorState, Loading, PageHeader, PublicLayout, useAsync } from '../../core.js'
import { CreaturePortrait } from './Atlas.jsx'
// One creature: where it spawns, and what spawns alongside it.
//
@@ -146,11 +147,22 @@ export default function AtlasCreature() {
{!loading && !error && data && (
<>
<PageHeader
eyebrow="Bestiary"
title={data.name}
lead={`Up to ${num(data.total)} alive at once across ${num(data.points)} spawner${data.points === 1 ? '' : 's'}.`}
/>
{/* The portrait sits BESIDE the header rather than inside it: `art`
is NULL for most creatures on most installs — no shard link, a
host that cannot render images, or simply a body this client has
no art for — and a header component that had to lay out around an
absent picture would be carrying that case forever. Here the row
collapses to exactly the header, which is what it was before. */}
<div style={{ display: 'flex', alignItems: 'flex-start', gap: 16 }}>
<CreaturePortrait art={data.art} name={data.name} size={96} />
<div style={{ minWidth: 0, flex: 1 }}>
<PageHeader
eyebrow="Bestiary"
title={data.name}
lead={`Up to ${num(data.total)} alive at once across ${num(data.points)} spawner${data.points === 1 ? '' : 's'}.`}
/>
</div>
</div>
<div style={{ display: 'flex', flexDirection: 'column', gap: 12 }}>
<Panel

View File

@@ -2,6 +2,7 @@ import { useCallback, useEffect, useState } from 'react'
import { Link } from 'react-router-dom'
import api from '../../api.js'
import { EmptyState, ErrorState, Loading, PageHeader, PublicLayout, useAsync } from '../../core.js'
import ItemIcon from '../../components/ItemIcon'
// ── The player-vendor marketplace ───────────────────────────────────────────
//
@@ -86,6 +87,7 @@ function ListingRow({ listing }) {
return (
<div className="panel" style={{ padding: '13px 15px', display: 'flex', gap: 14, alignItems: 'center' }}>
<ItemIcon art={listing.art} name={itemLabel(listing)} />
<div style={{ minWidth: 0, flex: 1 }}>
<div
className="display"

View File

@@ -1,6 +1,7 @@
import { Link, useParams } from 'react-router-dom'
import api from '../../api.js'
import { EmptyState, ErrorState, Loading, PageHeader, PublicLayout, useAsync } from '../../core.js'
import ItemIcon from '../../components/ItemIcon'
// One player vendor: where to find it and everything it is selling.
//
@@ -75,8 +76,9 @@ export default function MarketVendor() {
<div
key={i.serial}
className="panel"
style={{ padding: '10px 14px', display: 'flex', gap: 12, alignItems: 'baseline' }}
style={{ padding: '10px 14px', display: 'flex', gap: 12, alignItems: 'center' }}
>
<ItemIcon art={i.art} name={itemLabel(i)} size={28} />
<span className="sans" style={{ flex: 1, minWidth: 0, color: 'var(--head)', fontSize: '0.88rem' }}>
{i.amount > 1 ? `${num(i.amount)} × ` : ''}
{itemLabel(i)}

View File

@@ -102,6 +102,42 @@ test('admin atlas actions use the right methods and bodies', async () => {
assert.deepEqual(calls[1].opts.body, { path: '/srv/servuo' })
})
// ── the Asset Bridge's two stages (docs/link/v8.md §6) ──────────────────────
// Update and Re-import are one route and differ only by `force`, and the
// difference is not cosmetic: one transfers nothing when the client files are
// unchanged, the other fetches the whole catalogue. A binding that sent `force`
// on both would make the cheap button the expensive one, and nothing visible
// would change — the pictures would be correct either way.
test('assets.update asks for the diff and assets.reimport asks for everything', async () => {
await admin.assets.update()
assert.equal(calls[0].url, '/api/v1/admin/shard/assets/import')
assert.equal(calls[0].opts.method, 'POST')
assert.deepEqual(calls[0].opts.body, { approve: false })
await admin.assets.reimport()
assert.deepEqual(calls[1].opts.body, { force: true, approve: false })
})
// Approving a vanished key re-runs the SAME operation the operator pressed, so
// `approve` has to ride on both. Sending the update's approval as a re-import
// would quietly turn "yes, accept those deletions" into a full re-download.
test('approve rides on whichever import the operator ran', async () => {
await admin.assets.update(true)
await admin.assets.reimport(true)
assert.deepEqual(calls[0].opts.body, { approve: true })
assert.deepEqual(calls[1].opts.body, { force: true, approve: true })
})
test('cliloc admin actions use the right methods and bodies', async () => {
await admin.clilocs.import({ force: true })
assert.equal(calls[0].url, '/api/v1/admin/shard/clilocs/import')
assert.deepEqual(calls[0].opts.body, { force: true, approve: false })
await admin.clilocs.setPath('/srv/uo-client')
assert.equal(calls[1].opts.method, 'PUT')
assert.deepEqual(calls[1].opts.body, { path: '/srv/uo-client' })
})
// ── path encoding ───────────────────────────────────────────────────────────
// A city name with an apostrophe and a space is the real case: "Serpent's Hold"
// is a governor city, and an unencoded one would break the route match rather

View File

@@ -120,7 +120,7 @@ const it = (name, fn) => test(name, { skip: skip && 'no dist/entry.js — run np
it('registers routes in all three areas, namespaced under the module id', () => {
const { routes } = registered
assert.equal(routes.public.length, 13)
assert.equal(routes.admin.length, 7)
assert.equal(routes.admin.length, 8)
assert.equal(routes.player.length, 2)
for (const area of ['public', 'admin', 'player']) {
for (const r of routes[area]) {

View File

@@ -1,8 +1,8 @@
{
"id": "uo",
"name": "Ultima Online",
"version": "0.5.0",
"coreApi": "^1.9.0",
"version": "0.6.0",
"coreApi": "^1.10.0",
"server": "server/index.js",
"client": { "entry": "client/dist/entry.js" },
"schema": "server/db/schema.sql",

View File

@@ -16,6 +16,11 @@
"path": "/api/v1/admin/shard/accounts",
"tier": "public"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/assets",
"tier": "public"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/atlas",
@@ -271,6 +276,16 @@
"path": "/api/v1/admin/shard/account",
"tier": "public"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/assets/import",
"tier": "public"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/assets/warm",
"tier": "public"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/atlas/approve",

View File

@@ -33,6 +33,7 @@ const shardBroadcast = require('./utils/shardBroadcast')
const shardAtlas = require('./model/shardAtlas/shardAtlas.model')
const shardClilocs = require('./model/shardClilocs/shardClilocs.model')
const shardMarket = require('./model/shardMarket/shardMarket.model')
const shardItemArt = require('./model/shardAssets/shardItemArt.model')
/**
* Best-effort startup probe of the uo-link sidecar.
@@ -80,11 +81,18 @@ async function onBoot() {
// REMOVE a facet is staged for admin approval instead of being applied.
await shardAtlas.refreshOnBoot()
// Refresh the cliloc table (UO's id → display-string map) from the file the
// operator converted out of their own client. Same contract as the atlas:
// hash-gated so an unchanged file costs one read, and best-effort so a missing
// or wrong-format file never stops the site coming up — it just means item
// names render as ids, which is what they did before the table existed.
// Refresh the cliloc table (UO's id → display-string map).
//
// **On an install with uo-link configured this imports nothing** — protocol 8
// moved the base table to the shard, and asking for it would put a sidecar
// round trip in the boot sequence to answer a question whose answer is "no"
// except after a client patch. That is an operator action, so importing is an
// operator action: Admin → Shard (docs/link/v8.md §9).
//
// Without a shard link it is the old file pipeline, unchanged: hash-gated so an
// unchanged file costs one read, and best-effort so a missing or wrong-format
// file never stops the site coming up — it just means item names render as ids,
// which is what they did before the table existed.
const clilocResult = await shardClilocs.refreshOnBoot()
// A cliloc import changes what item names RESOLVE to, and the marketplace
@@ -107,6 +115,15 @@ async function onBoot() {
// Deliberately not awaited — see the header. An unreachable sidecar would
// otherwise hold the listener closed for the length of an HTTP timeout.
checkUoLink().catch((err) => log.warn('uo-link startup probe failed', { error: err.message }))
// Item and land pictures for the keys this site's own rows name (§11, phase 5).
//
// A timer rather than a boot pass, and it is the same rule §9.2 set for clilocs:
// **boot does not call the shard.** The first pass is one interval away, so an
// unreachable sidecar costs a log line rather than a startup delay, and an
// operator who has just configured the bridge does not have to restart to get
// pictures. `unref`ed, so it never holds shutdown open.
shardItemArt.startWarming()
}
async function onShutdown() {
@@ -114,6 +131,7 @@ async function onShutdown() {
// still works — the pool is open, the push dispatcher is up, the SSE fan-out
// is live. It is the only chance to close cleanly, and it is budgeted, so a
// hook that will not let go costs five seconds rather than the whole shutdown.
shardItemArt.stopWarming() // stop the item-art warm pass
uoLinkSocket.stop() // close the uo-link WS ingest client
shardBroadcast.closeAll() // end any open shard live-feed SSE streams
}

View File

@@ -467,6 +467,38 @@ const TEMPLATES = [
'{{champsUrl}}',
),
// ── The champion falls (Protocol 6) ─────────────────────────────────────
//
// The other half of the pair above, and the half the wire could not report
// until protocol 6 gave the shard a kind for it. Written as the crier's own
// follow-up: the same voice that announced the champion walking is the one
// that reports it did not walk far.
//
// `{{damagerNote}}` is a single-token block, so an unattributed kill renders
// the paragraph without it rather than as a sentence with a hole in it.
email(
'uo.champ.boss-killed',
'Champion spawn — the champion falls (town crier)',
'uo.champ.boss_killed',
'Hear ye — {{bossName}} has fallen',
[
heading('h', 'Hear ye, hear ye'),
text('p1',
'{{bossName}} has fallen{{atPlace}}.{{damagerNote}} The altar is quiet again, and it '
+ 'will not stay quiet.'),
button('cta', 'See the altars', '{{champsUrl}}'),
],
),
inapp(
'uo.champ.boss-killed-inapp',
'Champion spawn — the champion falls (in-app)',
'uo.champ.boss_killed',
'{{bossName}} has fallen',
'{{bossName}} has fallen{{atPlace}}.{{damagerNote}}',
'See the altars',
'{{champsUrl}}',
),
// ── A guildmaster of the craft ──────────────────────────────────────────
email(
'uo.skill.capped',
@@ -624,6 +656,14 @@ const TEMPLATES = [
const CHANNELS_OWNER = ['email', 'inapp']
const CHANNELS_BROADCAST = ['email', 'inapp', 'push']
// The same two channels as CHANNELS_OWNER and a different reason for them: a
// rule that goes to every subscriber but cannot be PUSHED, because push is
// keyed on a subscription id and no trigger in this module is also a registered
// stream. Same value, different fact — folding them into one constant would lose
// the distinction the moment somebody added push to whichever one they read as
// "the broadcast-ish list". See `uo.champ.boss_killed`.
const CHANNELS_CONTENT = ['email', 'inapp']
/** In-universe: both bodies are this module's, the digest is core's. */
const bodies = (key) => ({
email: `uo.${key}`,
@@ -863,6 +903,32 @@ const RULES = [
cooldown_seconds: 1800,
max_sends_per_hour: 1000,
},
{
trigger_id: 'uo.champ.boss_killed',
name: 'Champion spawn — the champion falls',
audience: 'subscribers',
// **`CHANNELS_CONTENT`, not `CHANNELS_BROADCAST`** — this is the one rule in
// the file that leaves push out, and it is not an oversight.
//
// Push delivery is keyed on the SUBSCRIPTION id, and a subscription row only
// ever exists for an id the preferences screen offered a push toggle for —
// which core's catalog grants to registered STREAMS and nothing else. This
// module's stream ids (`champ.start`, `idoc.warning`, …) and its trigger ids
// (`uo.champ.started`, …) are disjoint sets, so no trigger here can be pushed
// through the engagement path at all: the tickle resolves to zero endpoints
// while the send log records it delivered.
//
// That is true of every sibling rule above and is a pre-existing defect, not
// one this rule introduces. What this rule declines to do is add a
// twenty-first instance of it. See EVENTS_PLAN.md Phase 11a.
channels: CHANNELS_CONTENT,
template_keys: bodies('champ.boss-killed'),
// The same half-hour as its `boss_up` twin, and on the SAME subject — the
// spawn — so an altar that pops and is cleared inside the window produces the
// walk or the fall, not both.
cooldown_seconds: 1800,
max_sends_per_hour: 1000,
},
{
trigger_id: 'uo.server.up',
name: 'Shard — came online',
@@ -960,10 +1026,29 @@ const RULES = [
},
]
const RULE_GROUPS = [{
key: 'triggers-v1',
note: 'UO notifications stay off until an operator enables one',
rules: RULES,
}]
// A group is seeded ONCE, under its own settings guard. So a rule appended to an
// existing group reaches fresh installs and nothing else: every deployment that
// has already stamped `triggers-v1` is done with it forever, and the new rule
// would silently never arrive. That is Engagement Phase 11's seed-key finding,
// and core applied the same remedy in Events Phase 10 — a NEW key per addition,
// never an edit to an old one.
//
// So protocol 6's `uo.champ.boss_killed` rule ships as its own group rather than
// as a twenty-seventh entry above. `RULES` remains the whole declared set, which
// is what the "every declared trigger has exactly one rule" invariant reads.
const BOSS_KILLED = RULES.filter((r) => r.trigger_id === 'uo.champ.boss_killed')
const RULE_GROUPS = [
{
key: 'triggers-v1',
note: 'UO notifications stay off until an operator enables one',
rules: RULES.filter((r) => !BOSS_KILLED.includes(r)),
},
{
key: 'champ-boss-killed-v1',
note: 'The champion-falls notice, added with protocol 6; off like every other',
rules: BOSS_KILLED,
},
]
module.exports = { TEMPLATES, RULES, RULE_GROUPS }

View File

@@ -600,6 +600,42 @@ const COME_ONLINE = [
description: 'A trailing fragment, LEADING SPACE included, or empty when the frame carries no location.' },
],
},
{
// Protocol 6, and the reason the kind exists at all. Its first consumer is not
// a mail rule but an EVENT PHASE CONDITION: `{ on: 'uo.champ.boss_killed',
// where: [...], count: 1 }` is how an author says "move to the next phase when
// the boss falls", and a condition is expressed over a trigger firing. That is
// also why it is declared here rather than only ingested — a kind nothing
// declares is a kind no event can wait on.
id: 'uo.champ.boss_killed',
label: 'A champion boss was defeated',
description: 'Players brought down a champion spawn boss.',
kind: 'event',
subjectKey: 'spawnSerial',
audience: 'subscribers',
ceiling: 'authenticated',
version: V1,
variables: [
{ name: 'spawnSerial', type: 'string', required: true, example: '0x40012345',
description: 'The spawn controller, or the boss itself where the shard could not name an altar. Also the cooldown subject.' },
{ name: 'bossName', type: 'string', required: true, example: 'Semidar',
description: 'The boss that fell.' },
{ name: 'category', type: 'string', required: false, example: 'champion',
description: 'champion or sea.' },
{ name: 'location', type: 'string', required: false, example: 'Felucca 5187, 570 (Destard)',
description: 'Where, already formatted for reading.' },
{ name: 'killerName', type: 'string', required: false, example: 'Aldric',
description: 'Who struck the last blow, when the shard names one.' },
{ name: 'damagerCount', type: 'int', required: false, example: 14,
description: 'How many players did damage to it. The names themselves are staff-only and are deliberately not offered here.' },
{ name: 'damagerNote', type: 'string', required: false, example: ' 14 players fought it.',
description: 'A trailing sentence, LEADING SPACE included, or empty when nobody is credited.' },
{ name: 'champsUrl', type: 'url', required: false, example: '/uo/champs',
description: 'Site-relative path to the champions page.' },
{ name: 'atPlace', type: 'string', required: false, example: ' at Felucca 1480, 1600 (Destard)',
description: 'A trailing fragment, LEADING SPACE included, or empty when the frame carries no location.' },
],
},
{
id: 'uo.server.up',
label: 'The shard came online',

File diff suppressed because it is too large Load Diff

View File

@@ -104,7 +104,16 @@ module.exports = {
// do with a storage failure of core's. `inbox.push` additionally does not report
// "the user has this switched off", because a module that could see that would
// be a module that could enumerate people's preferences one write at a time.
events: { emit: (...args) => need().events.emit(...args) },
events: {
emit: (...args) => need().events.emit(...args),
// MODULE_API 1.10.0 (EVENTS.md F, Phase 8). "Ask every action of mine which
// of its ledgered resources the game still has." Core cannot know when to
// ask -- it has no concept of the game being up -- so the module says when,
// and `shardIngest` says it on a changed `bootId`. Fire-and-forget like
// `emit`, and for the same reason: core owns what happens next and there is
// nothing a game-event handler could correctly do with the answer.
reconcile: (...args) => need().events.reconcile(...args),
},
inbox: { push: (...args) => need().inbox.push(...args) },
secretBox: {
encrypt: (...args) => need().secretBox.encrypt(...args),

View File

@@ -27,6 +27,19 @@
-- marker, and deleting it would re-arm a protocol bump against tables this
-- file has just dropped.
-- The Asset Bridge's three (phase 3). No foreign keys of their own, so they lead:
-- `shard_creature_bodies.slug` mirrors an atlas slug and `shard_assets.body` a body
-- id, but neither is declared as a constraint — the atlas tables are rebuilt from
-- scratch on every refresh, and an FK into a table that is emptied and refilled
-- would make an ordinary re-parse fail on rows that are about to be re-inserted.
--
-- The uploaded PNGs are NOT removed here. They live under the uploads directory
-- alongside the operator's own artwork, this file drops tables rather than files,
-- and a purge that deleted an operator's hand-drawn creature portraits because
-- they shared a directory with imported ones would be unrecoverable.
DROP TABLE IF EXISTS `shard_asset_meta`;
DROP TABLE IF EXISTS `shard_creature_bodies`;
DROP TABLE IF EXISTS `shard_assets`;
DROP TABLE IF EXISTS `shard_atlas_pending`;
DROP TABLE IF EXISTS `shard_atlas_meta`;
DROP TABLE IF EXISTS `shard_cliloc_meta`;

View File

@@ -47,7 +47,7 @@ CREATE TABLE IF NOT EXISTS uo_link_config (
base_url VARCHAR(255) NULL,
ws_url VARCHAR(255) NULL,
auth_token_enc TEXT NULL,
protocol INT NOT NULL DEFAULT 5,
protocol INT NOT NULL DEFAULT 8,
enabled TINYINT(1) NOT NULL DEFAULT 0,
status VARCHAR(20) NOT NULL DEFAULT 'disconnected',
status_detail VARCHAR(500) NULL,
@@ -485,6 +485,13 @@ CREATE TABLE IF NOT EXISTS shard_spawn_points (
id INT AUTO_INCREMENT PRIMARY KEY,
facet VARCHAR(40) NOT NULL,
name VARCHAR(120) NULL, -- the ServUO spawner's own name
-- `XmlSpawner.UniqueId` (Phase 12b): the only name for one particular spawner
-- that exists OFF the shard. A property lease is targeted by it, because a
-- serial is assigned when the world is built and nothing here could know one --
-- so without this column the lease's target field could have no dropdown at
-- all. NULLable: a shard's own spawners, added in-world rather than from the
-- spawn files, carry none, and they are addressed by serial instead.
unique_id VARCHAR(64) NULL,
x INT NOT NULL,
y INT NOT NULL,
width INT NOT NULL DEFAULT 0,
@@ -500,7 +507,10 @@ CREATE TABLE IF NOT EXISTS shard_spawn_points (
landmark VARCHAR(120) NULL,
label VARCHAR(120) NOT NULL DEFAULT 'Wilderness',
INDEX idx_shard_spawn_points_facet (facet),
INDEX idx_shard_spawn_points_label (label)
INDEX idx_shard_spawn_points_label (label),
-- The spawner target's dropdown searches by name, and 6,707 rows is more than
-- a dropdown holds, so the search is the read rather than a filter over one.
INDEX idx_shard_spawn_points_name (name)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- The many-to-many between the two above: one spawner commonly carries several
@@ -544,6 +554,26 @@ CREATE TABLE IF NOT EXISTS shard_landmarks (
INDEX idx_shard_landmarks_name (name)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Item types this shard uses as decoration, from Data/Decoration/**/*.cfg.
--
-- Import-owned like every other shard_* atlas table. It exists so the events
-- decoration verb can offer an author a dropdown of what THIS shard already
-- calls scenery, rather than a list of item types curated by us: a shard with
-- custom decoration gets its own, and the list resolves with the shard offline
-- because it came out of the tree at import time.
--
-- `item_id` is a preview, not an identity. A type appears under as many item
-- ids as it has facings or variants (a BarredMetalDoor under eight), and the
-- first one seen is kept; the plugin constructs from the TYPE NAME and picks
-- its own graphic. `uses` is how many times the shard's own decoration reaches
-- for the type, which is the only ordering signal available that means anything.
CREATE TABLE IF NOT EXISTS shard_decor_types (
type VARCHAR(120) NOT NULL PRIMARY KEY,
item_id INT NOT NULL DEFAULT 0,
uses INT NOT NULL DEFAULT 0,
INDEX idx_shard_decor_types_uses (uses)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Configured champion altars from Config/ChampionSpawns.xml. This is static
-- roster data ("there is an Unholy Terror altar in Deceit") and is distinct from
-- the live champ.update feed in shard_champs ("it is on level 3 right now").
@@ -630,6 +660,108 @@ CREATE TABLE IF NOT EXISTS shard_atlas_pending (
CONSTRAINT chk_shard_atlas_pending_singleton CHECK (id = 1)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- ── The Asset Bridge (docs/link/v8.md, protocol 8 phase 3) ─────────────────
--
-- One row per imported asset: the manifest side of §6, and what makes an Update
-- a diff rather than a re-download. `sha256` is of the PNG the shard produced, so
-- a re-import fetches only the keys whose hash moved.
--
-- **`file` is a filename under the uploads directory, never a path.** Images are
-- written through `ctx.uploads`, the same door the operator's own atlas art comes
-- in by, and storing a path here would let a row decide where the server reads
-- from.
--
-- `bytes`/`width`/`height` are carried from the manifest rather than re-derived,
-- because the manifest reports them before the pixels are fetched and a screen
-- that lists what WOULD be imported needs them then.
CREATE TABLE IF NOT EXISTS shard_assets (
asset_key VARCHAR(191) NOT NULL PRIMARY KEY, -- §5's key: `body/34/a0`, `body/820/a23`
family VARCHAR(24) NOT NULL DEFAULT 'body',
sha256 CHAR(64) NOT NULL,
bytes INT NOT NULL DEFAULT 0,
width INT NOT NULL DEFAULT 0,
height INT NOT NULL DEFAULT 0,
body INT NULL, -- the body id, for the atlas join
direction TINYINT NULL,
file VARCHAR(191) NULL, -- filename under uploads/, NULL until fetched
imported_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
-- The atlas art derivation joins creature → body → asset on every atlas refresh,
-- so the body lookup is the read that has to be fast, not the key.
INDEX idx_shard_assets_body (body)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Which of the shard's catalogues a row was fetched under (§7, phase 5).
--
-- The body catalogue can answer "is this stale?" from `shard_asset_meta`, because
-- it is imported as a SET: one manifest walk covers every key, so one stored
-- fingerprint describes all of them. Item and land art has no manifest and never
-- will — 49,152 static ids times three thousand hues is not a set anyone
-- enumerates — so staleness has to be recorded per row, and this is it.
--
-- The shard derives the id from the files that decide the bytes (its art data
-- file, hues.mul, tiledata.mul, verdata.mul, and its own extractor version), so a
-- client patch changes it and a restart does not. A row whose `catalog` is not the
-- shard's current one is stale: the warm pass re-fetches it the next time
-- something asks for that key, and pictures nobody looks at any more are simply
-- never re-fetched, which is the whole reason this is per-row and lazy rather than
-- a sweep. NULL means "written before this column existed", which is stale by the
-- same test and costs one re-fetch.
ALTER TABLE shard_assets ADD COLUMN IF NOT EXISTS catalog VARCHAR(32) NULL;
-- Which action a body's thumbnail came from (§11.2, phase 6).
--
-- The catalogue is still one row per body and still a first frame; what changed
-- is that a body with no art at action 0 is catalogued at the first action that
-- has any, and the key says so — `body/820/a23` is a horse whose action 0 is
-- empty. 73 of a stock client's bodies are in that state, and they rendered as
-- text on the bestiary until this phase looked one action further.
--
-- It is stored rather than parsed back out of the key because the atlas join
-- needs it in SQL, and re-deriving it there with SUBSTRING_INDEX would put a
-- second, weaker parser of §5's key scheme in the schema. NULL means a row
-- written before this column existed, which is action 0 by definition — every
-- key the catalogue had then ended in `a0`.
ALTER TABLE shard_assets ADD COLUMN IF NOT EXISTS action TINYINT NULL;
-- Slug → body id, as the shard itself answered it (§8).
--
-- **Deliberately NOT a column on `shard_spawn_creatures`.** That table is
-- IMPORT-OWNED: `replaceAtlas` empties and refills it inside one transaction on
-- every atlas refresh. A body id living there would be destroyed by a routine
-- re-parse of the ServUO tree — and the next asset Update would find the source
-- hashes unchanged, report "nothing to do", and never put it back. The portrait
-- would simply vanish from every creature page until somebody thought to force a
-- re-import.
--
-- So the resolution lives here, outside the atlas's blast radius, and
-- `replaceAtlas` READS it to derive `shard_spawn_creatures.art` on the way past.
--
-- `status` is the shard's own verdict and each value is a different thing an
-- operator can act on: `ok`, `unknown` (the spawn file names a type this shard's
-- scripts do not define — real drift), `notCreature` (a spawn file legitimately
-- naming an item or decoration, a permanent answer), `failed` (its constructor
-- threw). A row is kept for every one of them, because "asked and answered no" is
-- what stops the next pass asking again.
CREATE TABLE IF NOT EXISTS shard_creature_bodies (
slug VARCHAR(120) NOT NULL PRIMARY KEY, -- → shard_spawn_creatures.slug (no FK)
type_name VARCHAR(120) NOT NULL, -- the ServUO class name that was asked
body INT NULL, -- NULL unless status = 'ok'
status VARCHAR(16) NOT NULL DEFAULT 'ok',
resolved_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX idx_shard_creature_bodies_body (body)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Singleton (id = 1) describing the asset import currently applied: the shard's
-- catalogue id, its extractor version, the counts and when it ran. Same shape and
-- same job as `shard_cliloc_meta` — it is what an Update compares against to
-- decide there is nothing to do.
CREATE TABLE IF NOT EXISTS shard_asset_meta (
id TINYINT NOT NULL PRIMARY KEY DEFAULT 1,
payload JSON NOT NULL,
imported_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
CONSTRAINT chk_shard_asset_meta_singleton CHECK (id = 1)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- House registry (Protocol 2.0). The house.update full-state feed carries richer
-- fields than the house.decay transition feed shard_houses was built for. Rather
-- than a second table for one entity, extend shard_houses: house.update writes the
@@ -786,3 +918,59 @@ UPDATE uo_link_config SET protocol = 5
WHERE id = 1 AND protocol < 5
AND NOT EXISTS (SELECT 1 FROM settings WHERE `key` = 'uo_link_protocol_5_migrated');
INSERT IGNORE INTO settings (`key`, value) VALUES ('uo_link_protocol_5_migrated', '1');
-- 4. The protocol pin again, at 7 -- and this block is a FIX to already-merged
-- code rather than ordinary Phase 12b work.
--
-- Phase 11a took the wire to 6 and Phase 12a took it to 7, and neither moved
-- this. `uoLinkClient` sends `X-UOLink-Version: <this column>` on every call and
-- the sidecar answers an exact mismatch with a 409, so a deployment that installed
-- this module at any point since Phase 10 would have had EVERY sidecar call
-- refused against a protocol-7 sidecar -- the whole event plane dead, loudly but
-- for a reason nobody would look here for.
--
-- It survived two phases because both live walks set the column by hand while
-- standing the rig up, which is exactly the shape of a migration nobody runs.
-- One block carries an install the whole way rather than one per missed version:
-- `protocol < 7` is deliberate, and it is why the 4 and 5 blocks above wrote
-- `< n` rather than `= n-1`.
ALTER TABLE uo_link_config MODIFY COLUMN protocol INT NOT NULL DEFAULT 7;
UPDATE uo_link_config SET protocol = 7
WHERE id = 1 AND protocol < 7
AND NOT EXISTS (SELECT 1 FROM settings WHERE `key` = 'uo_link_protocol_7_migrated');
INSERT IGNORE INTO settings (`key`, value) VALUES ('uo_link_protocol_7_migrated', '1');
-- The protocol pin at 8 -- the Asset Bridge (docs/link/v8.md), and the first bump this
-- module takes IN the phase that consumes it rather than a phase or two later.
--
-- Phase 1 of that work moved `link`'s PROTOCOL_VERSION and the overlay's `overlay.toml`
-- together, because the installer refuses to pair a sidecar and an overlay that disagree.
-- Nothing enforces the third declaration -- this one -- and the block above is the record
-- of what that costs: two phases of every REST call answered `409 protocol version
-- mismatch`, invisible because both live walks had set the column by hand.
--
-- Phase 2 is where this module first calls a protocol-8 route (`GET /cliloc`), so it is
-- where the pin moves. Same one-shot shape and the same `protocol < 8`, so an install
-- that missed an earlier bump is carried the whole way rather than one step.
ALTER TABLE uo_link_config MODIFY COLUMN protocol INT NOT NULL DEFAULT 8;
UPDATE uo_link_config SET protocol = 8
WHERE id = 1 AND protocol < 8
AND NOT EXISTS (SELECT 1 FROM settings WHERE `key` = 'uo_link_protocol_8_migrated');
INSERT IGNORE INTO settings (`key`, value) VALUES ('uo_link_protocol_8_migrated', '1');
-- `shard_spawn_points.unique_id` for an install that already had the table
-- (Asset Bridge phase 9; the column itself is Events phase 12b).
--
-- The column was added to the CREATE TABLE above and nowhere else, so it reached
-- fresh installs and no existing one -- `CREATE TABLE IF NOT EXISTS` does not add
-- a column to a table that is already there, which is what every ALTER in this
-- file exists to do. `replaceAtlas` inserts `unique_id` unconditionally, so on an
-- upgraded install EVERY spawn-atlas import since v1.2.0 has failed outright with
-- `Unknown column 'unique_id' in 'INSERT INTO'` -- the bestiary, the spawn map and
-- the champion altars all frozen at whatever was last imported.
--
-- Found by the phase 9 acceptance walk, on a rig whose tables predate 12b: a fresh
-- install cannot reproduce it, and neither can a test whose schema is this file
-- applied to an empty database. That is the same blind spot the protocol-pin block
-- above records, two phases running.
ALTER TABLE shard_spawn_points ADD COLUMN IF NOT EXISTS unique_id VARCHAR(64) NULL;

View File

@@ -47,6 +47,7 @@ module.exports = function register(ctx, api) {
const shardTriggers = require('./config/shardTriggers')
const shardAudiences = require('./config/shardAudiences')
const engagementSeeds = require('./config/engagementSeeds')
const uoEventActions = require('./config/uoEventActions')
const townCrierLeg = require('./utils/shardAnnounce')
const teamProvider = require('./model/teamProvider/teamProvider.model')
const guildCommand = require('./commands/guild.command')
@@ -157,6 +158,29 @@ const engagementSeeds = require('./config/engagementSeeds')
// either.
api.registerSlashCommands([guildCommand])
// The event contract (MODULE_API 1.10.0, EVENTS.md F, EVENTS_PLAN.md Phase 9).
// Three verbs an event author can put in a step, the one budget dimension that
// bounds a broadcast, and the three option sources the spawn atlas answers.
//
// **All of it is optional, by the contract's own posture.** A deployment
// without this module still has an event engine that can announce, wait, cue a
// human and publish results; what these add is the ability for an event to
// reach the GAME. Nothing here is a precondition for anything of core's.
//
// The wave is deliberately the verbs that need no protocol change: the write
// plane they use has existed since protocol 2.1 and the admin screens have
// driven it by hand for months. The world verbs -- creatures, gates, leases --
// wait for Phase 11 to put an idempotency key and a lease deadline on the wire,
// because a world write core cannot prove ran exactly once is not one this
// module is willing to make unattended.
api.registerEventBudgets(uoEventActions.BUDGETS)
api.registerEventActions(uoEventActions.ACTIONS)
// Phase 11b. One live-read config key, and the module never writes it: an author
// puts `core.lease` in a step and core owns the duration bound, the
// two-events-one-target check and the teardown restore.
api.registerEventLeases(uoEventActions.LEASES)
api.registerEventOptionSources(uoEventActions.OPTION_SOURCES)
api.onBoot(boot.onBoot)
api.onShutdown(boot.onShutdown)
@@ -166,5 +190,6 @@ const engagementSeeds = require('./config/engagementSeeds')
streams: shardStreams.STREAMS.length,
triggers: shardTriggers.TRIGGERS.length,
audiences: shardAudiences.AUDIENCES.length,
eventActions: uoEventActions.ACTIONS.length,
})
}

View File

@@ -0,0 +1,380 @@
const core = require('../../core')
const { query } = core
// Raw SQL for the Asset Bridge's three tables (docs/link/v8.md §6, §8, §12).
//
// Unlike `shard_clilocs` and the atlas tables, these are NOT import-owned in the
// empty-and-refill sense, and the difference is the whole reason phase 3 put them
// in their own tables rather than in columns on `shard_spawn_creatures`.
//
// An asset row is expensive to obtain — a decode on the shard, a PNG across the
// wire, a file written under uploads/ — and it is valid until the operator
// patches their client. An atlas refresh, by contrast, happens on every boot and
// destroys everything it owns. Putting the two in one table would mean a routine
// re-parse of the ServUO tree silently deleting every imported portrait, with the
// next Update reporting "nothing changed" and never restoring them.
//
// So these are upserted per key, and the only thing that ever deletes from them
// is an explicit removal of a key the shard no longer offers — which is staged
// for review, never applied silently (§6).
const BATCH = 500
async function batched(conn, sql, rows) {
for (let i = 0; i < rows.length; i += BATCH) {
await conn.batch(sql, rows.slice(i, i + BATCH))
}
return rows.length
}
// ── the manifest side ──────────────────────────────────────────────────────
/**
* The asset rows we hold in one family, as a Map of key → row.
*
* **The family is required, and the reason is a deletion.** The import diffs what
* this returns against a manifest, and a manifest is always of ONE family (§14 —
* the reply carries a single catalogue id, so it could not be otherwise). Phase 5
* put item and land art in this table beside the body catalogue; read whole, the
* body import then sees every item picture as a key the shard has stopped
* offering and stages all of them for deletion. On a real install that is a few
* hundred pictures the operator is asked to approve the loss of, with a sentence
* that is entirely wrong about what happened.
*
* `null` reads every family, which nothing in the import path should ever want.
*/
async function allAssets(family = null) {
const rows = await query(
'SELECT asset_key, family, sha256, bytes, width, height, body, action, direction, file, catalog ' +
'FROM shard_assets' +
(family ? ' WHERE family = ?' : ''),
family ? [family] : [],
)
const map = new Map()
for (const row of rows) {
map.set(row.asset_key, {
key: row.asset_key,
family: row.family,
sha256: row.sha256,
bytes: Number(row.bytes) || 0,
width: Number(row.width) || 0,
height: Number(row.height) || 0,
body: row.body === null ? null : Number(row.body),
action: row.action === null ? null : Number(row.action),
direction: row.direction === null ? null : Number(row.direction),
file: row.file || null,
catalog: row.catalog || null,
})
}
return map
}
/**
* Write the assets an import produced, and record what the import was.
*
* One transaction for the rows and the meta together: the meta row is what an
* Update compares against to decide there is nothing to do, so a meta written
* without its rows would make the site believe it holds a catalogue it does not.
*
* `ON DUPLICATE KEY UPDATE` rather than delete-and-insert, because an unchanged
* key must keep the file it already points at — re-writing the file for every
* asset on every Update is exactly the cost the manifest diff exists to avoid.
*
* `remove` is the keys an operator has APPROVED the loss of (§6). They are
* deleted here, inside the same transaction, because a half-applied removal is
* the worst of the three outcomes: until phase 8 the import unlinked the sprite
* and left the row, so the catalogue still counted a picture that was gone, the
* atlas could point a creature at a deleted file, and the very next forced
* import staged the same key for review again — telling the operator nothing had
* changed, about a file it had already deleted.
*/
async function saveAssets(rows, meta, remove = []) {
const conn = await core.pool.getConnection()
try {
await conn.beginTransaction()
const values = rows.map((r) => [
r.key,
r.family || 'body',
r.sha256,
r.bytes ?? 0,
r.width ?? 0,
r.height ?? 0,
r.body ?? null,
r.action ?? null,
r.direction ?? null,
r.file ?? null,
r.catalog ?? meta?.catalog ?? null,
])
await batched(
conn,
'INSERT INTO shard_assets ' +
'(asset_key, family, sha256, bytes, width, height, body, action, direction, file, catalog) ' +
'VALUES (?,?,?,?,?,?,?,?,?,?,?) ' +
'ON DUPLICATE KEY UPDATE family = VALUES(family), sha256 = VALUES(sha256), ' +
'bytes = VALUES(bytes), width = VALUES(width), height = VALUES(height), ' +
'body = VALUES(body), action = VALUES(action), direction = VALUES(direction), ' +
'file = VALUES(file), catalog = VALUES(catalog), imported_at = CURRENT_TIMESTAMP',
values,
)
if (remove.length > 0) {
for (let i = 0; i < remove.length; i += BATCH) {
const slice = remove.slice(i, i + BATCH)
await conn.query(
`DELETE FROM shard_assets WHERE asset_key IN (${slice.map(() => '?').join(',')})`,
slice,
)
}
}
if (meta) {
await conn.query(
'INSERT INTO shard_asset_meta (id, payload) VALUES (1, ?) ' +
'ON DUPLICATE KEY UPDATE payload = VALUES(payload), imported_at = CURRENT_TIMESTAMP',
[JSON.stringify(meta)],
)
}
await conn.commit()
return values.length
} catch (err) {
await conn.rollback().catch(() => {})
throw err
} finally {
conn.release()
}
}
// ── the on-demand side (§11, phase 5) ──────────────────────────────────────
/**
* The pictures we hold for an explicit list of keys, as a Map of key → filename.
*
* This is the read on the hot path — every marketplace page and every character
* sheet runs it — so it is one statement over the primary key and it returns only
* what it is asked for. It deliberately does NOT check staleness: a page renders
* the picture it has, and deciding whether that picture is out of date is the warm
* pass's job, off the request.
*/
async function filesForKeys(keys) {
const list = [...new Set(keys.filter((k) => typeof k === 'string' && k !== ''))]
if (list.length === 0) return new Map()
const rows = await query(
`SELECT asset_key, file FROM shard_assets WHERE file IS NOT NULL AND asset_key IN (${list
.map(() => '?')
.join(',')})`,
list,
)
const map = new Map()
for (const row of rows) map.set(row.asset_key, row.file)
return map
}
/**
* Which of these keys we already hold under the shard's CURRENT catalogue.
*
* The warm pass subtracts this from what it wants, so everything it does not
* return gets fetched: a key we have never seen, and a key whose row was written
* against a catalogue the shard has since moved past (§7 — an operator patched
* their client). A row with no file is not held either, because the database and
* the uploads volume can disagree and a broken image is worse than a re-fetch.
*/
async function freshKeys(keys, catalog) {
const list = [...new Set(keys.filter((k) => typeof k === 'string' && k !== ''))]
if (list.length === 0) return new Set()
const rows = await query(
`SELECT asset_key FROM shard_assets WHERE file IS NOT NULL AND catalog <=> ? ` +
`AND asset_key IN (${list.map(() => '?').join(',')})`,
[catalog ?? null, ...list],
)
return new Set(rows.map((r) => r.asset_key))
}
/** Counts for the admin surface, split by family. */
async function countByFamily() {
const rows = await query(
'SELECT family, COUNT(*) AS total, SUM(file IS NOT NULL) AS stored FROM shard_assets GROUP BY family',
)
const out = {}
for (const row of rows) {
out[row.family] = { total: Number(row.total) || 0, stored: Number(row.stored) || 0 }
}
return out
}
/**
* Record what the import that just finished actually did (§6, phase 8).
*
* **A second write, deliberately.** The interesting half of that summary — how
* many atlas creatures resolved to a body id, how many portraits were applied —
* does not exist when `saveAssets` commits: producing it takes another round trip
* to the shard, and widening the rows-and-meta transaction to cover a network
* call is how an import ends up holding a write lock for the length of a timeout.
*
* `JSON_SET` rather than a read-modify-write for the same reason the rest of this
* file is one statement per operation: the payload is the gate an Update compares
* against, and re-serialising it from the outside is how a concurrent import
* loses a field nobody notices for a month.
*
* It is cosmetic by design — nothing reads `last` to make a decision, the panel
* only renders it — so a failure here is logged and swallowed by the caller
* rather than failing an import that has already applied.
*/
async function recordLastImport(last) {
await query('UPDATE shard_asset_meta SET payload = JSON_SET(payload, ?, JSON_COMPACT(?)) WHERE id = 1', [
'$.last',
JSON.stringify(last),
])
}
async function getMeta() {
const rows = await query('SELECT payload, imported_at FROM shard_asset_meta WHERE id = 1')
if (rows.length === 0) return null
const payload = typeof rows[0].payload === 'string' ? JSON.parse(rows[0].payload) : rows[0].payload
return { ...payload, importedAt: rows[0].imported_at }
}
/**
* How many assets we hold, optionally in one family.
*
* **The family argument is not optional in spirit.** Phase 5 put item and land
* art in this table beside the body catalogue, and they are counted differently
* by nature: the catalogue is a SET with a known size, while item art is however
* much of an unbounded space the site has happened to ask for. A whole-table
* count answers neither question — it reported the creature catalogue as 1,408
* rows on an install holding 1,095 portraits and 313 item pictures, which is a
* confident wrong number in the one place an operator checks whether the import
* worked.
*/
async function countAssets(family = null) {
const rows = await query(
'SELECT COUNT(*) AS n, SUM(file IS NOT NULL) AS stored FROM shard_assets' +
(family ? ' WHERE family = ?' : ''),
family ? [family] : [],
)
return { total: Number(rows[0]?.n) || 0, stored: Number(rows[0]?.stored) || 0 }
}
// ── the body resolution side (§8) ──────────────────────────────────────────
/**
* Replace the whole slug → body map.
*
* This one IS a replace, and for the opposite reason to the assets above: it is
* derived from the atlas's creature list, so a slug that has left the atlas has
* no meaning any more and keeping its row would leave the map growing forever
* across map changes. The pass that produces it is cheap to redo — a shard round
* trip, no files — which is what makes replacing safe here and not there.
*/
async function replaceBodies(rows) {
const conn = await core.pool.getConnection()
try {
await conn.beginTransaction()
await conn.query('DELETE FROM shard_creature_bodies')
const values = rows.map((r) => [r.slug, r.typeName, r.body ?? null, r.status || 'ok'])
await batched(
conn,
'INSERT INTO shard_creature_bodies (slug, type_name, body, status) VALUES (?,?,?,?)',
values,
)
await conn.commit()
return values.length
} catch (err) {
await conn.rollback().catch(() => {})
throw err
} finally {
conn.release()
}
}
async function allBodies() {
return query(
'SELECT slug, type_name, body, status, resolved_at FROM shard_creature_bodies ORDER BY slug',
)
}
async function countBodies() {
const rows = await query(
"SELECT COUNT(*) AS n, SUM(status = 'ok') AS resolved FROM shard_creature_bodies",
)
return { total: Number(rows[0]?.n) || 0, resolved: Number(rows[0]?.resolved) || 0 }
}
/**
* The derivation `replaceAtlas` applies on the way past: slug → uploaded filename.
*
* One join rather than two reads, because it runs inside the atlas transaction —
* the atlas rows are being inserted at that moment and every extra round trip is
* time the site's creature list does not exist.
*
* Rows with no body, no asset or an asset whose bytes were never fetched are
* simply absent from the result, which is what leaves `art` NULL. That is a
* first-class state everywhere it is consumed and the expected one for two thirds
* of the player bodies (§5.2).
*
* **The join is pinned to the catalogue key, not merely to the body id** — and as
* of phase 6 that key is no longer always `a0`. 73 of this client's bodies have
* no art at action 0 and are catalogued at the first action that does (§11.2), so
* a join hardcoding `a0` would silently drop exactly the creatures this phase
* added — a horse among them. It reads the row's own `action` instead, which
* still excludes any deeper key a later phase adds (`body/400/a2/f0` does not
* equal `body/400/a2`), so one slug still matches at most one row.
*
* `COALESCE(a.action, 0)` because a row written before this column existed has
* NULL there and a NULL inside `CONCAT` makes the whole comparison NULL — which
* would have dropped every portrait on the site until the next import, with the
* database perfectly correct.
*/
async function artBySlug() {
const rows = await query(
'SELECT b.slug, a.file FROM shard_creature_bodies b ' +
"JOIN shard_assets a ON a.body = b.body AND a.family = 'body' " +
"AND a.asset_key = CONCAT('body/', b.body, '/a', COALESCE(a.action, 0)) " +
"WHERE b.status = 'ok' AND b.body IS NOT NULL AND a.file IS NOT NULL",
)
const map = {}
for (const row of rows) map[row.slug] = row.file
return map
}
module.exports = {
allAssets,
saveAssets,
recordLastImport,
getMeta,
countAssets,
replaceBodies,
allBodies,
countBodies,
artBySlug,
filesForKeys,
freshKeys,
countByFamily,
}

View File

@@ -0,0 +1,588 @@
const fs = require('fs')
const path = require('path')
const db = require('./shardAssets.db')
const atlasDb = require('../shardAtlas/shardAtlas.db')
const core = require('../../core')
const bridge = require('../../utils/assetBridge')
const uoLinkConfig = require('../uoLinkConfig/uoLinkConfig.model')
const log = require('../../core').logger('shardAssets')
// Client artwork, over the bridge (docs/link/v8.md — protocol 8, phase 3).
//
// What this replaces: until now the only way a creature got a picture on this
// site was for an operator to open UOFiddler on a desktop, export sprites by
// hand, copy them to the web host and write a `spawnAtlas.art.json` naming each
// one. Almost nobody did, so `shard_spawn_creatures.art` was NULL on every
// install and the atlas rendered as text.
//
// The shard has had those files the whole time — a ServUO server cannot boot
// without a UO client — so as of protocol 8 it decodes them itself and hands the
// pictures over the same request/reply path as every other shard read.
//
// ── Two passes, and they answer different questions ───────────────────
//
// **The catalogue** (§4.8, §11) is one thumbnail per creature body: the shard
// walks bodies 02047, validates each index entry, decodes the ones that are real
// and hands back `{ key, sha256 }` first and the PNG second. On a stock client
// that is **1,095 sprites** — 787 out of the legacy anim files, 235 more out of
// the UOP packages (phase 4), and 73 more since phase 6, which have no art at
// action 0 and real art at a later one. Never the 1,144 the decoder claims.
//
// A key therefore names its action — `body/820/a23` is a horse whose action 0 is
// empty — and the key is still one per body. Nothing here treats `a0` as the
// shape of a body key; the atlas join reads the row's own action (§11.2).
//
// **Body resolution** (§8) is the join. The atlas knows a creature by the class
// name in `Spawns/*.xml`; the client knows it by a body id; nothing in the ServUO
// tree declares the mapping. Only code inside ServUO can answer it, by
// constructing the creature and reading `Body.BodyID`, and that is the whole
// reason this could not be done off the shard.
//
// ── The 357, and why nothing here trusts a success ────────────────────
//
// 357 of the bodies ServUO's decoder returns a bitmap for **have no art**. Their
// index entry reads `length 0`, the library's stream buffer still holds the
// previous creature, and what comes back is whichever body was decoded before —
// a real, plausible, correctly-sized picture of the wrong animal. The shard now
// validates every index entry before it decodes, which is what cut the catalogue
// from 1,144 to 787, and the count going down is the point.
//
// The consequence for this file is a rule: **a missing asset is a normal
// outcome, never an error.** Two thirds of the player bodies have no art on a
// stock client (§5.2), so an import that reported eight failures every time would
// teach an operator to ignore the panel.
//
// ── Where the pictures go, and what still wins ────────────────────────
//
// Into `<uploads>/atlas/`, through the same door the operator's own artwork uses,
// and `shard_spawn_creatures.art` is DERIVED from them rather than written by
// them. **The operator's `spawnAtlas.art.json` still wins outright**: someone who
// has drawn their own creature portraits must not have them replaced by a sprite
// rip on the next Update.
//
// ── Why the resolution does not live on the atlas row ─────────────────
//
// `shard_spawn_creatures` is emptied and refilled on every atlas refresh. A body
// id or a filename stored there would be destroyed by an ordinary re-parse of the
// ServUO tree, and the next asset Update would find the client files unchanged,
// report "nothing to do" and never restore it. So both live in their own tables
// and the atlas import reads them on the way past.
/** Where imported sprites land, under core's upload directory. */
const ART_SUBDIR = 'atlas'
// ── configuration ──────────────────────────────────────────────────────────
/**
* Is there a shard to ask?
*
* Both halves matter, exactly as in `shardClilocs.model`: a `baseUrl` on a
* disabled config is an install that was set up and then switched off, and
* calling it would spend a 12 s timeout to learn what the row already says.
*/
async function shardLinked() {
try {
const config = await uoLinkConfig.getSafe()
return Boolean(config?.enabled && config?.baseUrl)
} catch {
return false
}
}
function artDir() {
return path.join(core.uploads.UPLOAD_DIR, ART_SUBDIR)
}
/**
* The operator's own art map, which wins over anything imported.
*
* Read through the atlas model rather than re-implemented, so there is one
* definition of where that file lives and what an absent one means.
*/
function operatorArt() {
// eslint-disable-next-line global-require
return require('../shardAtlas/shardAtlas.model').loadArtMap()
}
// ── writing a sprite ───────────────────────────────────────────────────────
/**
* The filename one asset gets on disk.
*
* **Content-addressed on purpose.** A stable name per key (`uo-body-34.png`)
* would be overwritten in place by an Update, and every browser and CDN that had
* already cached it would keep serving last month's client's sprite — with
* nothing anywhere to notice, because the database row would be correct. Putting
* eight bytes of the hash in the name makes a changed sprite a changed URL.
*
* The old file is removed when a key's hash moves, so the directory tracks the
* catalogue rather than accumulating one file per import forever.
*/
function fileNameFor(key, sha256) {
const stem = key.replace(/[^a-zA-Z0-9]+/g, '-').replace(/^-+|-+$/g, '')
return `uo-${stem}-${String(sha256).slice(0, 8)}.png`
}
/**
* Write one sprite and return its filename, or null if it could not be written.
*
* Never throws. A full disk or a read-only volume must degrade to "this creature
* has no picture" — which the whole site already renders correctly, because it is
* the state every install was in until this phase — rather than failing an import
* that has already fetched hundreds of others.
*/
function writeSprite(key, sha256, png) {
const name = fileNameFor(key, sha256)
try {
fs.mkdirSync(artDir(), { recursive: true })
fs.writeFileSync(path.join(artDir(), name), png)
return name
} catch (err) {
log.warn('could not write an imported sprite', { key, error: err.message })
return null
}
}
/** Best-effort removal of a sprite a key no longer points at. */
function removeSprite(name) {
if (!name) return
try {
fs.unlinkSync(path.join(artDir(), name))
} catch {
// Already gone, or never written. Either way there is nothing to do, and an
// import must not fail because a file it was tidying up was tidied already.
}
}
// ── the import ─────────────────────────────────────────────────────────────
/**
* Import (or update) the body catalogue and the slug → body map.
*
* Returns a result rather than throwing, so a controller can render it and an
* operator can read it:
*
* `skipped` no shard configured — the file era had no equivalent here
* `unavailable` the shard could not answer (down, plane off, no libgdiplus)
* `unchanged` the client files match what was imported; nothing fetched
* `imported` fetched and applied
* `needsReview` a key we hold has vanished from the shard's manifest
* `failed` something went wrong mid-import
*
* `force` re-imports even when the client files are unchanged (which is also how
* an operator recovers from a deleted uploads directory — the database still
* holds the hashes, but the files behind them are gone). `approve` accepts a
* catalogue that no longer offers keys we hold.
*
* `by` is who pressed the button, carried through only so the panel can say what
* the last import did and who ran it without reading the audit log (phase 8). It
* decides nothing.
*/
async function importAssets({ force = false, approve = false, by = null } = {}) {
if (!(await shardLinked())) {
return {
status: 'skipped',
reason: 'uo-link is not configured, so there is no shard to read client files from',
}
}
let sources
try {
sources = await bridge.sourceFingerprint()
} catch (err) {
return failure(err, 'client file manifest')
}
// §4.4: a Linux shard host without libgdiplus cannot render a sprite at all.
// It is reported on the source gate precisely so an operator meets it while
// setting the shard up rather than from an empty bestiary weeks later.
if (sources.imaging && sources.imaging.ok === false) {
return {
status: 'unavailable',
code: 'NO_IMAGING',
reason: sources.imaging.reason || 'the shard host cannot render images',
}
}
const meta = await db.getMeta().catch(() => null)
if (!force && bridge.sameSources(sources, meta?.sources)) {
const counts = await db.countAssets(bridge.FAMILY)
const bodies = await db.countBodies()
return {
status: 'unchanged',
assets: counts.total,
stored: counts.stored,
bodies: bodies.resolved,
hashing: sources.hashing,
importedAt: meta?.importedAt ?? null,
}
}
let manifest
try {
manifest = await bridge.readManifest({ family: bridge.FAMILY })
} catch (err) {
return failure(err, 'asset manifest')
}
// The body family only. This diff decides what gets DELETED, and the manifest
// it is diffed against is of one family by construction — so reading the whole
// table here stages every item picture phase 5 warmed as a vanished key.
const held = await db.allAssets(bridge.FAMILY)
const offered = new Set(manifest.rows.map((r) => r.key))
// A key we hold that the shard no longer offers. An unmounted client volume and
// a deliberate downgrade look identical from here, and the wrong guess deletes
// artwork, so it is staged rather than applied — the same rule, and the same
// reasoning, as a vanished cliloc overlay or a disappearing atlas facet.
const vanished = [...held.keys()].filter((key) => !offered.has(key))
if (vanished.length > 0 && !approve) {
return {
status: 'needsReview',
reason:
`${vanished.length} asset(s) this site holds are no longer offered by the shard; ` +
'nothing was changed',
// Each one carries the picture it currently has, because the decision the
// operator is being asked for is "is it right that these disappear?" and a
// list of keys cannot be looked at. `body/820/a23` names nothing a human
// recognises; the horse it is a picture of does.
vanished: vanished.slice(0, 50).map((key) => ({ key, file: held.get(key)?.file ?? null })),
vanishedCount: vanished.length,
}
}
// The diff, and the whole reason stage 2 carries hashes and not pixels. An
// unchanged key is skipped ONLY if its file is actually still on disk: the row
// and the file can disagree (a wiped uploads volume, a restore from a database
// dump), and re-fetching a sprite is far cheaper than a creature page with a
// broken image on it.
const wanted = manifest.rows.filter((row) => {
const existing = held.get(row.key)
if (!existing || existing.sha256 !== row.sha256) return true
if (!existing.file) return true
return !fs.existsSync(path.join(artDir(), existing.file))
})
let fetched = { assets: new Map(), missing: { absent: 0, unsupported: 0 } }
if (wanted.length > 0) {
try {
fetched = await bridge.fetchAssets({
keys: wanted.map((r) => r.key),
catalog: manifest.catalog,
})
} catch (err) {
return failure(err, 'asset content')
}
}
const rows = []
let written = 0
for (const row of manifest.rows) {
const existing = held.get(row.key)
const got = fetched.assets.get(row.key)
if (!got) {
// Either it was unchanged and skipped, or the shard could not serve it. The
// row is kept either way, with whatever file it already had — a key the
// shard suddenly cannot render must not lose the picture we already hold.
rows.push({ ...row, file: existing?.file ?? null })
continue
}
const name = writeSprite(row.key, got.sha256, got.png)
if (name) {
written++
if (existing?.file && existing.file !== name) removeSprite(existing.file)
}
rows.push({
...row,
sha256: got.sha256 || row.sha256,
bytes: got.bytes || row.bytes,
width: got.width || row.width,
height: got.height || row.height,
body: got.body ?? row.body,
action: got.action ?? row.action ?? 0,
direction: got.direction ?? row.direction,
file: name ?? existing?.file ?? null,
})
}
const removed = []
if (vanished.length > 0) {
for (const key of vanished) {
removeSprite(held.get(key)?.file)
removed.push(key)
}
}
try {
await db.saveAssets(
rows,
{
catalog: manifest.catalog,
extractorVersion: manifest.extractorVersion,
family: bridge.FAMILY,
playerBodies: manifest.playerBodies,
sources: { files: sources.files, extractorVersion: sources.extractorVersion },
count: rows.length,
},
// The approved removals go in with the write. The sprite is already
// unlinked above; leaving the row behind would keep counting a picture
// that is gone and re-offer the same key for review on every import.
removed,
)
} catch (err) {
return { status: 'failed', reason: err.message }
}
const bodies = await resolveAtlasBodies()
const art = await applyArt()
// What this run did, kept beside the catalogue it produced (phase 8). The admin
// panel renders it as "the last import", which is the question an operator has
// straight after pressing a button that takes a minute and prints nothing:
// what changed, and did the body pass find drift. Core's activity log records
// the same action, but it is one unfiltered list of every admin action on the
// site, so an import from three client patches ago is not findable there.
//
// Best-effort on purpose: the import has already applied, and losing a cosmetic
// summary must not turn a successful import into a failure.
const last = {
at: new Date().toISOString(),
by,
force,
approve,
assets: rows.length,
fetched: fetched.assets.size,
written,
removed: removed.length,
absent: fetched.missing.absent,
unsupported: fetched.missing.unsupported,
bodies: bodies.tally ?? null,
art: art.applied ?? 0,
}
try {
await db.recordLastImport(last)
} catch (err) {
log.warn('could not record the import summary', { error: err.message })
}
log.info('asset import applied', {
assets: rows.length,
fetched: fetched.assets.size,
written,
absent: fetched.missing.absent,
bodies: bodies.resolved,
art: art.applied,
})
return {
status: 'imported',
catalog: manifest.catalog,
extractorVersion: manifest.extractorVersion,
assets: rows.length,
fetched: fetched.assets.size,
written,
absent: fetched.missing.absent,
unsupported: fetched.missing.unsupported,
removed: removed.length,
scanned: manifest.scanned,
pages: manifest.pages,
playerBodies: manifest.playerBodies,
bodies,
art,
}
}
function failure(err, what) {
if (err instanceof bridge.AssetBridgeError) {
return { status: 'unavailable', code: err.code, reason: err.message }
}
log.warn(`asset import failed reading the ${what}`, { error: err.message })
return { status: 'failed', reason: err.message }
}
// ── the body pass (§8) ─────────────────────────────────────────────────────
/**
* Ask the shard for a body id for every creature the atlas knows.
*
* `shard_spawn_creatures.name` is the ServUO class name — the atlas build picks
* the winning spelling of the spawn TYPE token rather than inventing a display
* name — so this needs no new column to ask its question.
*
* Never throws: a shard that goes down between the asset fetch and this pass
* leaves the assets imported and the map as it was, which is a strictly better
* state than failing the whole import back to nothing.
*/
async function resolveAtlasBodies() {
let creatures = []
try {
creatures = await atlasDb.allCreatureTypes()
} catch (err) {
return { resolved: 0, asked: 0, reason: err.message }
}
if (creatures.length === 0) {
return { resolved: 0, asked: 0, reason: 'the spawn atlas has no creatures loaded' }
}
let rows
try {
rows = await bridge.resolveBodies({ creatures })
} catch (err) {
return { resolved: 0, asked: creatures.length, reason: err.message }
}
try {
await db.replaceBodies(rows)
} catch (err) {
return { resolved: 0, asked: creatures.length, reason: err.message }
}
const tally = { ok: 0, unknown: 0, notCreature: 0, failed: 0 }
for (const row of rows) {
if (tally[row.status] === undefined) tally.failed++
else tally[row.status]++
}
return { asked: creatures.length, answered: rows.length, resolved: tally.ok, tally }
}
// ── the derivation (§12) ───────────────────────────────────────────────────
/**
* Point every atlas creature at its imported portrait.
*
* Two rules, and the second is the one worth stating:
*
* 1. The operator's `spawnAtlas.art.json` wins. Someone who drew their own
* creature portraits must not have them replaced by a sprite rip.
* 2. A slug with neither is set back to NULL rather than left alone. A creature
* whose body stopped resolving — the operator removed a script package, say
* — would otherwise keep pointing at a file that is about to be deleted, and
* a broken image is worse than no image.
*/
async function applyArt() {
const derived = await db.artBySlug()
const operator = operatorArt()
const map = { ...derived, ...operator }
try {
const applied = await atlasDb.setCreatureArt(map)
return { applied, derived: Object.keys(derived).length, operator: Object.keys(operator).length }
} catch (err) {
log.warn('could not apply imported creature art', { error: err.message })
return { applied: 0, derived: Object.keys(derived).length, error: err.message }
}
}
// ── status ─────────────────────────────────────────────────────────────────
/**
* What the admin panel renders: what is loaded, what the shard says, and whether
* the two agree.
*
* Never throws and never fails a page: every branch that could — no shard, a
* shard that is down, an asset plane the operator switched off — is a reported
* state with a reason an operator can act on.
*/
async function getStatus() {
// The BODY family, not the whole table: item and land art live here too and
// are reported separately below, because they are a working set rather than a
// catalogue with a size (§11).
const counts = await db.countAssets(bridge.FAMILY).catch(() => ({ total: 0, stored: 0 }))
const bodies = await db.countBodies().catch(() => ({ total: 0, resolved: 0 }))
const meta = await db.getMeta().catch(() => null)
const families = await db.countByFamily().catch(() => ({}))
const status = {
// Is there a shard to ask at all? Stated rather than left to be inferred:
// the panel disables its import buttons on it, and the alternative — reading
// it out of `reason`'s wording, or out of `shard` being null, which is also
// what a shard that is merely DOWN looks like — is a sentence that decides
// behaviour.
linked: await shardLinked(),
loaded: {
assets: counts.total,
stored: counts.stored,
creatures: bodies.total,
resolved: bodies.resolved,
catalog: meta?.catalog ?? null,
extractorVersion: meta?.extractorVersion ?? null,
importedAt: meta?.importedAt ?? null,
// Item and land pictures, counted separately because they are a different
// KIND of thing (§11, phase 5): no manifest, no set, and no "how many are
// there" to compare against. `items` is how many the site has been asked
// for and holds, which is the only number that means anything here.
items: families.static?.stored ?? 0,
land: families.land?.stored ?? 0,
// What the last import did, and who ran it (phase 8). Null on an install
// that has never imported, and on one whose last import predates this
// field — both of which render as "no import recorded" rather than as
// zeroes, because an import that fetched nothing is a real and different
// answer from one that never happened.
last: meta?.last ?? null,
},
shard: null,
drift: null,
}
if (!status.linked) {
status.reason = 'uo-link is not configured'
return status
}
try {
const sources = await bridge.sourceFingerprint()
status.shard = {
files: Object.keys(sources.files).length,
extractorVersion: sources.extractorVersion,
hashing: sources.hashing,
complete: sources.complete,
imaging: sources.imaging,
// Which §5 families this overlay serves. A phase-3 or phase-4 overlay says
// `['body']`, which is what an admin panel needs in order to say "update
// your plugin" rather than showing an item-art pipeline that cannot work.
families: sources.families,
}
status.drift = meta ? !bridge.sameSources(sources, meta.sources) : true
} catch (err) {
status.reason = err.message
status.code = err instanceof bridge.AssetBridgeError ? err.code : 'UNAVAILABLE'
}
return status
}
module.exports = {
ART_SUBDIR,
artDir,
fileNameFor,
importAssets,
resolveAtlasBodies,
applyArt,
getStatus,
}

View File

@@ -0,0 +1,510 @@
const fs = require('fs')
const path = require('path')
const db = require('./shardAssets.db')
const core = require('../../core')
const bridge = require('../../utils/assetBridge')
const uoLinkConfig = require('../uoLinkConfig/uoLinkConfig.model')
const log = require('../../core').logger('shardItemArt')
// Item and land pictures, fetched because something on this site names them
// (docs/link/v8.md §5, §11 — protocol 8, phase 5).
//
// ── Why this is not the body catalogue with a different prefix ─────────
//
// The bestiary wants every creature, so phase 3 imports a SET: walk a manifest,
// diff the hashes, fetch what moved. That works because the set is 1,095 rows
// and one megabyte.
//
// This side has no set. The shard's client addresses 49,152 item graphics and
// has art for 39,189 of them; multiply by three thousand hues and there is
// nothing to enumerate, no manifest worth building and nothing worth importing
// ahead of time. What there IS, at any moment, is a few hundred keys that the
// site's own rows actually name: the items on a vendor, the things a character
// is wearing. That is the working set, and it is what this fetches.
//
// ── Who is allowed to make the shard do work ──────────────────────────
//
// **Ingest warms; the route only serves** (org lead, 2026-09-11). A page never
// waits on the shard and never causes a fetch: it renders the pictures already
// on disk and leaves out the ones that are not, which is exactly the state every
// install was in before this phase and which every surface already handles.
// Fetching happens behind that, from the keys the site has stored.
//
// The alternative — fetch on the first request for a key — was rejected on one
// number. The shard's asset plane serves **one request at a time** by design
// (§3.2), so any anonymous visitor able to name a key could walk 49,152 ids
// times 3,000 hues through that single slot and keep an operator's own import
// waiting behind it, from a URL with nothing to authenticate. Warming from the
// site's own data has no such surface: the ceiling is the number of distinct
// (item, hue) pairs the shard itself has told us about.
//
// ── Why the wanted set is DERIVED and not a queue ─────────────────────
//
// A queue table would need writing on the ingest path, draining, retrying,
// pruning and reconciling after a restart. The same answer falls out of a
// `SELECT DISTINCT` over the rows that name the items — which is self-healing by
// construction: a key lost to a restart comes back the next time the pass runs,
// and a key for a vendor that has gone stops being wanted the moment its row is
// deleted. The in-memory set below is an optimisation on top of that, never the
// record: it exists so a picture seen on a LIVE character sheet — which is
// fetched from the shard per request and stored nowhere — is not forgotten.
//
// ── Staleness, without a manifest (§7) ────────────────────────────────
//
// Every fetched row records the shard's `catalog` id, which is a hash of the
// files that decide the bytes. A client patch changes it, a restart does not. So
// "is this picture out of date?" is a per-row comparison rather than a manifest
// diff, and the answer costs nothing for the pictures nobody is looking at any
// more: they are simply never re-fetched.
/** Where item and land pictures land, under core's upload directory. */
const ART_SUBDIR = 'items'
/**
* How many keys one warm pass will fetch.
*
* A bound rather than a target. The shard serves one asset request at a time, so
* a pass that asked for everything at once would hold that slot for as long as it
* took — against an operator who might be trying to run an import. Passes are
* cheap and repeat; a backlog drains over several of them and nothing waits.
*/
const WARM_BATCH = 400
/**
* How many live-observed keys are remembered between passes.
*
* Bounded because this is a set fed by page views. It is an optimisation over the
* derived set, so dropping from it costs a picture appearing one pass later, and
* never a picture that is lost.
*/
const SEEN_CAP = 5000
const seen = new Set()
// ── keys (§5) ──────────────────────────────────────────────────────────────
/**
* The one place an item key is spelled.
*
* Hue 0 means "not hued" on the wire, so it produces the plain key rather than a
* `/h0` one — the shard refuses `/h0` outright for the same reason, and the two
* agreeing is what stops the same PNG being stored twice under two names.
*/
function staticKey(itemId, hue = 0) {
const id = Number(itemId)
if (!Number.isInteger(id) || id < 0) return null
const h = Number(hue)
return Number.isInteger(h) && h > 0 ? `static/${id}/h${h}` : `static/${id}`
}
function landKey(tileId) {
const id = Number(tileId)
return Number.isInteger(id) && id >= 0 && id < 0x4000 ? `land/${id}` : null
}
function artDir() {
return path.join(core.uploads.UPLOAD_DIR, ART_SUBDIR)
}
/**
* Content-addressed, exactly as the body catalogue's names are and for the same
* reason: a stable name overwritten in place leaves every browser and CDN serving
* last month's client's sprite while the database row stays perfectly correct.
*/
function fileNameFor(key, sha256) {
const stem = key.replace(/[^a-zA-Z0-9]+/g, '-').replace(/^-+|-+$/g, '')
return `uo-${stem}-${String(sha256).slice(0, 8)}.png`
}
// ── noticing ───────────────────────────────────────────────────────────────
/**
* Remember that something on this site showed these (itemId, hue) pairs.
*
* Called from the market ingest and from the character sheet, and deliberately
* synchronous and allocation-light: it is on a request path and a page must never
* pay for a picture it is not going to get anyway.
*/
function notice(items) {
if (!Array.isArray(items)) return 0
let added = 0
for (const item of items) {
const key = staticKey(item?.itemId ?? item?.item_id, item?.hue)
if (!key || seen.has(key)) continue
// Oldest-first, and only when full. The derived set is the record; this is a
// cache of hints, so forgetting one costs a pass, not a picture.
if (seen.size >= SEEN_CAP) seen.delete(seen.values().next().value)
seen.add(key)
added++
}
return added
}
/** For tests and the admin surface: how many hints are waiting. */
function noticedCount() {
return seen.size
}
// ── serving ────────────────────────────────────────────────────────────────
/**
* Attach `art` to rows that name an item, in place, and notice what is missing.
*
* `art` is a FILENAME under `uploads/items/`, never a path or a URL — the same
* shape `shard_spawn_creatures.art` uses, so the client builds one URL the same
* way everywhere and the API never hard-codes a mount point.
*
* A row with no stored picture gets `art: null` rather than being changed in any
* other way. That is a first-class state: it is what every row looked like before
* this phase, every surface renders it, and it is what an item this client has no
* art for looks like permanently.
*/
async function decorate(rows, { itemIdField = 'itemId', hueField = 'hue' } = {}) {
const list = Array.isArray(rows) ? rows.filter((r) => r && typeof r === 'object') : []
if (list.length === 0) return list
const keys = list.map((row) => staticKey(row[itemIdField], row[hueField]))
let files = new Map()
try {
files = await db.filesForKeys(keys.filter(Boolean))
} catch (err) {
// Decoration, not the page. A picture lookup that fails must not fail a
// marketplace search.
log.warn('could not read item art', { error: err.message })
return list
}
for (let i = 0; i < list.length; i++) {
list[i].art = (keys[i] && files.get(keys[i])) || null
}
// Everything this page WANTED is worth warming, whether or not we had it: the
// ones we had may be stale, and the ones we did not are the point.
notice(list.map((row) => ({ itemId: row[itemIdField], hue: row[hueField] })))
return list
}
// ── warming ────────────────────────────────────────────────────────────────
async function shardLinked() {
try {
const config = await uoLinkConfig.getSafe()
return Boolean(config?.enabled && config?.baseUrl)
} catch {
return false
}
}
/**
* Every item key the site's own rows name, newest-priced first.
*
* `shard_vendor_items` is the only stored table that carries (item_id, hue)
* today. The character sheet's equipment is fetched live from the shard per
* request and stored nowhere, which is precisely what the in-memory hint set is
* for.
*/
async function wantedKeys() {
const keys = []
try {
const rows = await core.query(
'SELECT DISTINCT item_id, hue FROM shard_vendor_items WHERE item_id > 0 LIMIT 20000',
)
for (const row of rows) {
const key = staticKey(row.item_id, row.hue)
if (key) keys.push(key)
}
} catch (err) {
log.warn('could not read the marketplace for item art', { error: err.message })
}
// Hints last, so a backlog of stored rows is never starved by page traffic.
for (const key of seen) keys.push(key)
return [...new Set(keys)]
}
function writePicture(key, sha256, png) {
const name = fileNameFor(key, sha256)
try {
fs.mkdirSync(artDir(), { recursive: true })
fs.writeFileSync(path.join(artDir(), name), png)
return name
} catch (err) {
log.warn('could not write an item picture', { key, error: err.message })
return null
}
}
function removePicture(name) {
if (!name) return
try {
fs.unlinkSync(path.join(artDir(), name))
} catch {
// Already gone, or never written. A warm pass must not fail because a file it
// was tidying up was tidied already.
}
}
/**
* One warm pass: fetch the wanted keys we do not already hold, and store them.
*
* Returns a result rather than throwing, with the same vocabulary the body import
* uses — `skipped`, `unavailable`, `unchanged`, `imported`, `failed` — so the
* admin surface reports one set of words for both halves of this protocol.
*
* `limit` bounds one pass. `force` re-fetches keys we hold, which is how an
* operator recovers from a wiped uploads volume without waiting for a client
* patch to invalidate every row.
*/
async function warm({ limit = WARM_BATCH, force = false } = {}) {
if (!(await shardLinked())) {
return { status: 'skipped', reason: 'uo-link is not configured, so there is no shard to ask' }
}
let sources
try {
sources = await bridge.sourceFingerprint()
} catch (err) {
return failure(err, 'client file manifest')
}
if (sources.imaging && sources.imaging.ok === false) {
return {
status: 'unavailable',
code: 'NO_IMAGING',
reason: sources.imaging.reason || 'the shard host cannot render images',
}
}
// A phase-3 or phase-4 overlay serves bodies and nothing else. Asking it for a
// static is refused per request, which would be a warn on every pass forever —
// so it is checked once, here, and reported as the ordinary state it is.
// Defensive default rather than a trusted field: an older sidecar, an older
// overlay or a stubbed fingerprint can all leave it off, and `['body']` is the
// truthful reading of its absence (§6 — the families field arrived in phase 5).
const families = Array.isArray(sources.families) ? sources.families : ['body']
if (!families.includes('static')) {
return {
status: 'unavailable',
code: 'UNSUPPORTED',
reason:
"this shard's overlay does not serve item art; it offers " +
`${families.join(', ')}. Update the plugin overlay to get it.`,
}
}
const wanted = await wantedKeys()
if (wanted.length === 0) {
return { status: 'unchanged', wanted: 0, fetched: 0, written: 0 }
}
// The catalogue is learned from the first reply rather than asked for, so this
// pass cannot be the thing that decides what is stale. `catalog: null` on the
// request means "whatever you have"; the mid-walk guard in `fetchAssets` is what
// catches a client that moves underneath it.
let held = new Set()
if (!force) {
const current = await currentCatalog()
try {
held = await db.freshKeys(wanted, current)
} catch (err) {
return { status: 'failed', reason: err.message }
}
}
const todo = wanted.filter((key) => !held.has(key)).slice(0, Math.max(1, limit))
if (todo.length === 0) {
forget(wanted)
return { status: 'unchanged', wanted: wanted.length, held: held.size, fetched: 0, written: 0 }
}
let fetched
try {
fetched = await bridge.fetchAssets({ keys: todo })
} catch (err) {
return failure(err, 'item art')
}
// Only the filenames, and only for the keys in hand: the old file is removed
// when a key's hash moves, so `uploads/items/` tracks the working set instead of
// accumulating one file per client patch forever.
const existing = await db.filesForKeys(todo).catch(() => new Map())
const rows = []
let written = 0
for (const key of todo) {
const got = fetched.assets.get(key)
// A key the shard has no art for is not a failure and not a row: writing an
// empty row would make it "held" and stop it ever being asked again, which is
// wrong the moment an operator patches in the missing graphic.
if (!got) continue
const name = writePicture(key, got.sha256, got.png)
if (!name) continue
written++
const before = existing.get(key)
if (before && before !== name) removePicture(before)
rows.push({
key,
family: key.startsWith('land/') ? 'land' : 'static',
sha256: got.sha256,
bytes: got.bytes,
width: got.width,
height: got.height,
body: null,
direction: null,
file: name,
catalog: fetched.catalog,
})
}
if (rows.length > 0) {
try {
// No meta: `shard_asset_meta` is the BODY catalogue's singleton — what an
// Update compares a manifest against — and this family has no manifest. A
// warm pass writing there would tell the body import that a client it never
// looked at is unchanged.
await db.saveAssets(rows, null)
} catch (err) {
return { status: 'failed', reason: err.message }
}
}
forget(todo)
const result = {
status: 'imported',
catalog: fetched.catalog,
wanted: wanted.length,
held: held.size,
asked: todo.length,
fetched: fetched.assets.size,
written,
absent: fetched.missing.absent,
unsupported: fetched.missing.unsupported,
remaining: Math.max(0, wanted.length - held.size - todo.length),
}
log.info('item art warmed', result)
return result
}
/** Drop hints a pass has dealt with, so the set does not grow without bound. */
function forget(keys) {
for (const key of keys) seen.delete(key)
}
/**
* The catalogue id the shard would answer under right now.
*
* Read from a one-key probe rather than from a dedicated command: the shard puts
* `catalog` on every fetch reply, so the cheapest honest way to ask is to fetch
* something. `static/0` is the smallest such question and its answer is thrown
* away — what is wanted is the id beside it.
*
* A shard that cannot answer returns null, and null compares unequal to every
* stored catalogue, so the pass falls back to "everything is stale" — which costs
* a re-fetch and never serves a wrong picture. That is the right way round.
*/
async function currentCatalog() {
try {
const probe = await bridge.fetchAssets({ keys: ['static/0'] })
return probe.catalog ?? null
} catch (err) {
log.warn('could not read the shard art catalogue', { error: err.message })
return null
}
}
function failure(err, what) {
if (err instanceof bridge.AssetBridgeError) {
return { status: 'unavailable', code: err.code, reason: err.message }
}
log.warn(`item art failed reading the ${what}`, { error: err.message })
return { status: 'failed', reason: err.message }
}
// ── the background pass ────────────────────────────────────────────────────
let timer = null
/**
* Run a warm pass every few minutes, forever, while the process lives.
*
* Deliberately a plain interval and not a debounce on ingest. A market sweep
* delivers dozens of `vendor.listing` frames in a burst and debouncing each of
* them would either fire once per frame or need its own state machine; a pass is
* cheap when there is nothing to do (one `SELECT DISTINCT` and one probe) and the
* work it exists for is not urgent — a picture appearing a few minutes after the
* listing that wants it is invisible to everyone.
*
* `unref()` so this never holds the process open at shutdown.
*/
function startWarming({ everyMs = 5 * 60 * 1000 } = {}) {
if (timer) return
timer = setInterval(() => {
warm().catch((err) => log.warn('item art warm pass failed', { error: err.message }))
}, everyMs)
if (typeof timer.unref === 'function') timer.unref()
}
function stopWarming() {
if (!timer) return
clearInterval(timer)
timer = null
}
module.exports = {
ART_SUBDIR,
WARM_BATCH,
artDir,
fileNameFor,
staticKey,
landKey,
notice,
noticedCount,
decorate,
wantedKeys,
warm,
currentCatalog,
startWarming,
stopWarming,
}

View File

@@ -16,6 +16,7 @@ const ATLAS_TABLES = [
'shard_regions',
'shard_landmarks',
'shard_champion_spawns',
'shard_decor_types',
]
async function insertBatched(conn, sql, rows) {
@@ -103,6 +104,16 @@ async function replaceAtlas(atlas, art = {}) {
]),
)
// Optional: a tree with no Data/Decoration leaves this empty rather than
// failing the import, and the decoration verb then simply has nothing to
// offer. `?? []` rather than a guard, so an atlas built by an older parser
// (no `decor` key at all) reloads cleanly instead of throwing here.
counts.decor = await insertBatched(
conn,
'INSERT INTO shard_decor_types (type, item_id, uses) VALUES (?,?,?)',
(atlas.decor ?? []).map((d) => [d.type, d.itemId ?? 0, d.uses ?? 0]),
)
// Point ids are assigned explicitly rather than left to AUTO_INCREMENT: the
// join rows need to know them and `conn.batch()` reports no usable insertId
// for a multi-row insert. Safe because this transaction just emptied the
@@ -110,13 +121,14 @@ async function replaceAtlas(atlas, art = {}) {
counts.points = await insertBatched(
conn,
'INSERT INTO shard_spawn_points ' +
'(id, facet, name, x, y, width, height, spawn_range, max_count, min_delay, max_delay, ' +
'(id, facet, name, unique_id, x, y, width, height, spawn_range, max_count, min_delay, max_delay, ' +
'tod_start, tod_end, tod_mode, region, landmark, label) ' +
'VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)',
'VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)',
atlas.points.map((p, i) => [
i + 1,
p.facet,
p.name,
p.uniqueId || null,
p.x,
p.y,
p.width ?? 0,
@@ -357,6 +369,73 @@ function listLandmarks({ facet = '', q = '' } = {}) {
)
}
/**
* Every decoration type this shard uses, most-used first.
*
* Ordered by `uses` because a dropdown of 313 types needs the ones the shard
* actually reaches for at the top; the alphabetical tiebreak keeps the order
* stable across imports, which matters for a form an author scrolls.
*/
function listDecorTypes({ q = '' } = {}) {
const where = []
const params = []
if (q) {
where.push('type LIKE ?')
params.push(`%${q}%`)
}
return query(
`SELECT type, item_id, uses
FROM shard_decor_types
${where.length ? `WHERE ${where.join(' AND ')}` : ''}
ORDER BY uses DESC, type ASC`,
params,
)
}
/**
* Spawners an author can name, searched by name and bounded (Phase 12b).
*
* **A search rather than a list, and the numbers are why.** This tree has 6,707
* spawn points against a 2,000-entry dropdown bound, so a flat read would drop
* two thirds of the world and say nothing about which two thirds — the failure
* Phase 12a named for decoration, arriving for real. `resolveOptionSource` grew
* a `q` for this.
*
* Only rows with a `unique_id` are offered: that is the only name for a spawner
* that exists off the shard, and a row without one cannot be targeted from a
* form however it is labelled. A shard's own in-world spawners have none and are
* addressed by serial, which an author types rather than picks.
*
* Ordered by `max_count DESC` so the spawners worth an event's attention come
* first, with a stable alphabetical tiebreak for a form somebody scrolls.
*/
function listSpawners({ q = '', limit = 200 } = {}) {
const where = ['unique_id IS NOT NULL', "unique_id <> ''"]
const params = []
if (q) {
where.push('(name LIKE ? OR region LIKE ? OR landmark LIKE ?)')
params.push(`%${q}%`, `%${q}%`, `%${q}%`)
}
params.push(Number(limit) || 200)
return query(
`SELECT unique_id, name, facet, region, landmark, max_count
FROM shard_spawn_points
WHERE ${where.join(' AND ')}
ORDER BY max_count DESC, name ASC
LIMIT ?`,
params,
)
}
/** One decoration type, or nothing when this shard's files never name it. */
async function getDecorType(type) {
const rows = await query(
'SELECT type, item_id, uses FROM shard_decor_types WHERE type = ?',
[type],
)
return rows[0] || null
}
function listChampions({ facet = '' } = {}) {
const params = []
let where = ''
@@ -373,8 +452,64 @@ function listChampions({ facet = '' } = {}) {
)
}
/**
* Every creature the atlas knows, as `{ slug, name }` (docs/link/v8.md §8).
*
* `name` is the ServUO CLASS NAME, not a display string invented here: the atlas
* build picks the winning spelling of the spawn type token, so "GiantSpider" is
* what the column holds and what `ScriptCompiler.FindTypeByName` will resolve.
* That is the one property that lets the asset import ask its question without a
* new column, and it is worth knowing before anyone "tidies" this into a
* prettified label.
*/
async function allCreatureTypes() {
return query('SELECT slug, name FROM shard_spawn_creatures ORDER BY slug')
}
/**
* Point creatures at their artwork, from a `{ slug: filename }` map.
*
* Everything NOT in the map is set back to NULL, which is deliberate: a creature
* whose body stopped resolving must lose its portrait rather than keep pointing
* at a file that is about to be deleted. A broken image is worse than no image,
* and no image is the state the whole atlas UI was designed around.
*
* One transaction, and a single `CASE` update rather than a statement per slug —
* at ~800 creatures the round trips are the cost, not the work.
*/
async function setCreatureArt(map) {
const entries = Object.entries(map ?? {}).filter(
([slug, file]) => typeof slug === 'string' && slug !== '' && typeof file === 'string' && file !== '',
)
const conn = await core.pool.getConnection()
try {
await conn.beginTransaction()
await conn.query('UPDATE shard_spawn_creatures SET art = NULL WHERE art IS NOT NULL')
for (let i = 0; i < entries.length; i += BATCH) {
await conn.batch(
'UPDATE shard_spawn_creatures SET art = ? WHERE slug = ?',
entries.slice(i, i + BATCH).map(([slug, file]) => [file, slug]),
)
}
await conn.commit()
return entries.length
} catch (err) {
await conn.rollback().catch(() => {})
throw err
} finally {
conn.release()
}
}
module.exports = {
replaceAtlas,
allCreatureTypes,
setCreatureArt,
getMeta,
getFacets,
getPending,
@@ -388,5 +523,8 @@ module.exports = {
listCreatureCompanions,
listRegions,
listLandmarks,
listDecorTypes,
listSpawners,
getDecorType,
listChampions,
}

View File

@@ -5,13 +5,13 @@ const db = require('./shardAtlas.db')
const core = require('../../core')
const { settings } = core
const { slugify } = require('../../utils/spawnAtlasParse')
const {
AtlasSourceError,
PARSER_VERSION,
buildAtlas,
hashSources,
sameSources,
} = require('../../utils/spawnAtlasSource')
const { AtlasSourceError, PARSER_VERSION, sameSources } = require('../../utils/spawnAtlasSource')
// The two readers are reached through the namespace rather than destructured,
// because a test stubs them ON the module object and a binding taken at require
// time would keep calling the real one — quietly, and while reporting success.
const spawnAtlasSource = require('../../utils/spawnAtlasSource')
const { TreeBridgeError } = require('../../utils/treeBridge')
const uoLinkConfig = require('../uoLinkConfig/uoLinkConfig.model')
const log = require('../../core').logger('shardAtlas')
// The spawn atlas, refreshed from the shard's own ServUO tree.
@@ -37,6 +37,45 @@ const log = require('../../core').logger('shardAtlas')
const SETTING_KEY = 'spawn_atlas_servuo_path'
/**
* Is there a shard to ask?
*
* Both halves matter. `baseUrl` alone is an install that has been configured and
* then switched off, and calling it would spend a 12 s timeout to learn what the
* row already says. Never throws: an unreadable config means "no shard", and a
* local tree is a working answer.
*/
async function shardLinked() {
try {
const config = await uoLinkConfig.getSafe()
return Boolean(config?.enabled && config?.baseUrl)
} catch {
return false
}
}
/**
* Which end this atlas is built from (docs/link/v8.md §10, §17.7).
*
* **The bridge wins whenever uo-link is configured and enabled**, the same rule
* the cliloc table follows and for the same reason: there is no version of "which
* source?" an operator benefits from answering, so there is no setting asking it.
* A local tree remains the source where there is no shard link — development,
* same-host installs — plus the one-off explicit path an admin can type, which is
* an instruction rather than a default and therefore overrules this.
*/
async function sourceFor(pathOverride = '') {
const explicit = String(pathOverride || '').trim()
if (explicit !== '') return { kind: 'fs', root: explicit }
if (await shardLinked()) return { kind: 'bridge', root: '' }
return { kind: 'fs', root: await getServuoPath() }
}
/** How a source reads in a log line or an admin panel. */
const describe = (source) => (source.kind === 'bridge' ? 'the shard bridge' : source.root)
/**
* Where the ServUO tree lives.
*
@@ -107,8 +146,45 @@ function pointTypeRows(points) {
return rows
}
/**
* The art each creature gets when the atlas is rebuilt.
*
* **`replaceAtlas` empties `shard_spawn_creatures` and refills it**, so anything
* on that row is destroyed on every refresh — and a refresh happens on every
* boot. Before protocol 8 that cost nothing: `art` came from a file on disk and
* was simply re-read. As of phase 3 it can also come from an IMPORT, which is
* expensive to obtain and whose gate (the shard's client-file hashes) would say
* "unchanged" for weeks afterwards. So the imported values are re-derived here,
* on the way past, rather than being restored by an import that has no reason to
* run again.
*
* **The operator's map is spread last and therefore wins.** Someone who drew
* their own creature portraits must not have them replaced by a sprite rip on the
* next Update — the one property §12 states outright.
*
* Never throws: the asset tables are the newer half of this pair, and an atlas
* refresh must not start failing because an asset query did. Losing the imported
* art for one boot is recoverable by pressing Import; a boot that cannot rebuild
* the atlas is not.
*/
async function artForAtlas() {
const operator = loadArtMap()
try {
// eslint-disable-next-line global-require
const assetsDb = require('../shardAssets/shardAssets.db')
const derived = await assetsDb.artBySlug()
return { ...derived, ...operator }
} catch (err) {
log.warn('imported creature art could not be read; using the operator map alone', {
error: err.message,
})
return operator
}
}
async function applyAtlas(atlas) {
return db.replaceAtlas({ ...atlas, pointTypes: pointTypeRows(atlas.points) }, loadArtMap())
return db.replaceAtlas({ ...atlas, pointTypes: pointTypeRows(atlas.points) }, await artForAtlas())
}
/**
@@ -138,18 +214,29 @@ const currentParser = (meta) => meta?.parserVersion === PARSER_VERSION
async function refresh({ force = false, approve = false, path: pathOverride = '' } = {}) {
// An explicit override wins outright — it is a one-off "use this tree", and it
// must not be silently overruled by the configured path the way an env default
// would be.
const root = pathOverride.trim() !== '' ? pathOverride.trim() : await getServuoPath()
if (root === '') return { status: 'skipped', reason: 'no ServUO path configured' }
// would be, nor by the bridge.
const source = await sourceFor(pathOverride)
const root = source.root
const where = describe(source)
if (source.kind === 'fs' && root === '') {
return { status: 'skipped', reason: 'no ServUO path configured' }
}
let hashes
try {
hashes = hashSources(root)
hashes = await spawnAtlasSource.hashFrom(source)
} catch (err) {
if (err instanceof AtlasSourceError) {
return { status: 'unavailable', reason: err.message, code: err.code, path: root }
if (err instanceof AtlasSourceError || err instanceof TreeBridgeError) {
return {
status: 'unavailable',
source: source.kind,
reason: err.message,
code: err.code,
path: where,
}
}
return { status: 'failed', reason: err.message, path: root }
return { status: 'failed', source: source.kind, reason: err.message, path: where }
}
const meta = await db.getMeta().catch(() => null)
@@ -162,7 +249,7 @@ async function refresh({ force = false, approve = false, path: pathOverride = ''
// whatever an older build derived — a corrected parse would ship and never
// reach the data.
if (!force && sameSources(hashes, loaded) && currentParser(meta)) {
return { status: 'unchanged', path: root }
return { status: 'unchanged', source: source.kind, path: where }
}
// A rejected refresh must not re-prompt on every boot. It stays rejected until
@@ -170,14 +257,28 @@ async function refresh({ force = false, approve = false, path: pathOverride = ''
// decision.
const pending = await db.getPending().catch(() => null)
if (!approve && !force && pending?.status === 'rejected' && sameSources(hashes, pending.hashes)) {
return { status: 'unchanged', path: root, reason: 'refresh previously rejected' }
return {
status: 'unchanged',
source: source.kind,
path: where,
reason: 'refresh previously rejected',
}
}
let atlas
try {
atlas = buildAtlas(root)
atlas = await spawnAtlasSource.buildFrom(source)
} catch (err) {
return { status: 'failed', reason: err.message, path: root }
if (err instanceof AtlasSourceError || err instanceof TreeBridgeError) {
return {
status: 'unavailable',
source: source.kind,
reason: err.message,
code: err.code,
path: where,
}
}
return { status: 'failed', source: source.kind, reason: err.message, path: where }
}
const currentFacets = await db.getFacets().catch(() => [])
@@ -190,7 +291,8 @@ async function refresh({ force = false, approve = false, path: pathOverride = ''
if (removedFacets.length > 0 && !approve) {
const summary = {
hashes,
path: root,
source: source.kind,
path: where,
currentFacets,
incomingFacets,
removedFacets,
@@ -205,9 +307,16 @@ async function refresh({ force = false, approve = false, path: pathOverride = ''
try {
const counts = await applyAtlas(atlas)
return { status: 'imported', path: root, counts, addedFacets, removedFacets }
return {
status: 'imported',
source: source.kind,
path: where,
counts,
addedFacets,
removedFacets,
}
} catch (err) {
return { status: 'failed', reason: err.message, path: root }
return { status: 'failed', source: source.kind, reason: err.message, path: where }
}
}
@@ -230,7 +339,9 @@ async function rejectPending() {
/** Everything the admin panel needs to describe atlas state. */
async function status({ path: pathOverride = '' } = {}) {
const root = pathOverride.trim() !== '' ? pathOverride.trim() : await getServuoPath()
const source = await sourceFor(pathOverride)
const root = source.root
const configured = source.kind === 'bridge' || root !== ''
const [meta, pending, facets] = await Promise.all([
db.getMeta().catch(() => null),
db.getPending().catch(() => null),
@@ -239,9 +350,13 @@ async function status({ path: pathOverride = '' } = {}) {
let treeReadable = false
let drift = null
if (root !== '') {
if (configured) {
try {
const hashes = hashSources(root)
// On the bridge this is the MANIFEST, not the tree: 141 rows and ~32 KB,
// with no file bytes crossing the wire to answer "has anything changed".
// It is still a shard round trip on an admin page load, which is why it is
// here and not on the boot path (§17.7).
const hashes = await spawnAtlasSource.hashFrom(source)
treeReadable = true
const loaded = meta?.source
? Object.fromEntries(Object.entries(meta.source).map(([l, v]) => [l, v.sha256]))
@@ -255,8 +370,9 @@ async function status({ path: pathOverride = '' } = {}) {
}
return {
configured: root !== '',
path: root,
configured,
source: source.kind,
path: describe(source),
treeReadable,
drift,
facets,
@@ -272,6 +388,18 @@ async function status({ path: pathOverride = '' } = {}) {
*/
async function refreshOnBoot() {
try {
// **On the bridge it imports nothing**, deliberately, and by the same
// reasoning as the cliloc table (§17.7). A local tree hashes in ~120 ms and
// skips; asking the shard would put a sidecar round trip in the boot sequence
// to answer a question whose answer is "no" on every restart that did not
// follow a map edit — and editing spawn files is an operator action, so
// importing became one: Admin → Shard → Import. Whatever atlas is loaded
// keeps serving until then.
if ((await sourceFor()).kind === 'bridge') {
log.info('spawn atlas comes from the shard; import is admin-triggered (Admin → Shard)')
return { status: 'skipped', source: 'bridge', reason: 'the shard is the atlas source' }
}
const result = await refresh()
switch (result.status) {
case 'imported':
@@ -403,6 +531,59 @@ async function getCreature(slug, { facet = '', points = 200 } = {}) {
}
}
/**
* Decoration types, shaped for a dropdown.
*
* `type` is both the value and the label: it is the ServUO class name and it is
* what the plugin constructs from, so showing the author anything else would
* put a name on the screen that does not appear in the refusal if the shard
* declines it.
*/
async function listDecorTypes(opts = {}) {
const rows = await db.listDecorTypes(opts)
return rows.map((r) => ({
type: r.type,
itemId: Number(r.item_id) || 0,
uses: Number(r.uses) || 0,
}))
}
/**
* Spawners an author can name, searched (Phase 12b).
*
* The value is the `UniqueId` because that is what the shard resolves a target
* by; the label is the spawner's own name, which is what an author recognises
* ("fel bulbous putrification" is a place they know). A row with no name still
* answers, labelled by its id, rather than being dropped: a nameless spawner is
* still a spawner somebody may need to turn down.
*/
async function listSpawners(opts = {}) {
const rows = await db.listSpawners(opts)
return rows.map((r) => ({
uniqueId: r.unique_id,
name: r.name || null,
facet: r.facet,
region: r.region || null,
landmark: r.landmark || null,
maxCount: Number(r.max_count) || 0,
}))
}
/**
* One decoration type, or null.
*
* The events decoration verb resolves through this rather than passing a type
* name straight through, which does two things at once: it fetches the item id
* the graphic-holder classes need, and it keeps the verb to the vocabulary this
* shard's own decoration files use. A type the atlas has never seen is refused
* here rather than constructed there.
*/
async function getDecorType(type) {
const row = await db.getDecorType(String(type == null ? '' : type).trim())
if (!row) return null
return { type: row.type, itemId: Number(row.item_id) || 0, uses: Number(row.uses) || 0 }
}
async function listRegions(opts = {}) {
const rows = await db.listRegions(opts)
return rows.map((r) => ({
@@ -485,6 +666,9 @@ module.exports = {
getCreature,
listRegions,
listLandmarks,
listDecorTypes,
listSpawners,
getDecorType,
listChampions,
listFacets,
publicMeta,

View File

@@ -1,6 +1,6 @@
const db = require('./shardClilocs.db')
const { settings } = require('../../core')
const { displayText } = require('../../utils/clilocParse')
const { displayText, parseCliloc } = require('../../utils/clilocParse')
const {
ClilocFormatError,
ClilocSourceError,
@@ -8,8 +8,12 @@ const {
hashSources,
sameSources,
missingSources,
missingOverlays,
readOverlays,
readCliloc,
} = require('../../utils/clilocSource')
const bridge = require('../../utils/clilocBridge')
const uoLinkConfig = require('../uoLinkConfig/uoLinkConfig.model')
const log = require('../../core').logger('shardClilocs')
// The cliloc table — UO's id → display-string map, refreshed from a file the
@@ -29,11 +33,35 @@ const log = require('../../core').logger('shardClilocs')
// 2. **Nothing client-derived is committed.** The table is built from the
// operator's own file at a configured path. The repo ships no strings.
//
// The table is built from a SET of sources — the converted client table plus
// every operator-maintained overlay beside it — because shards edit items and
// add new ones, and those carry cliloc ids no stock client table has. All of
// them are re-read on every boot and hash-gated together, so adding one custom
// item never means re-exporting a 5 MB client file. Later sources win.
// The table is built from a SET of sources — a base table plus every
// operator-maintained overlay beside it — because shards edit items and add new
// ones, and those carry cliloc ids no stock client table has. Later sources win,
// so an overlay both adds ids the client never had and overrides stock ones.
//
// ── Where the base comes from (protocol 8, docs/link/v8.md §9) ─────────
//
// **The shard**, on any install with uo-link configured. It has the operator's
// client files already — a ServUO server cannot boot without them — and since
// phase 2 it has the decompressor too, so `GET /cliloc` returns the table and
// nobody installs UOFiddler or copies a 5 MB file anywhere.
//
// **A file on disk** otherwise. That is the pipeline this replaces, kept for
// installs with no shard link and for development, and deprecated rather than
// removed: an operator who has one keeps working, and an operator who has a shard
// never builds one. Passing an explicit `path` to `refresh()` still selects it,
// which is the escape hatch for "import from this file, this once".
//
// **Overlays are always the filesystem's**, either way. There is nothing on the
// shard to ask for: ServUO has no server-side notion of a custom cliloc, so the
// `custom/` directory is the only place those ids exist.
//
// ── What that changed about WHEN this runs ───────────────────────
//
// Boot no longer imports on the shard path. The file path could hash 5 MB locally
// on every restart and skip; the shard path would mean a sidecar round trip in the
// boot sequence, for a table that changes when an operator patches their client —
// an event they know about and we do not. So on the bridge, importing is an admin
// action (Admin → Shard), and boot leaves whatever is loaded serving.
//
// That set is also why this has the atlas's escalation, in a lighter form. A
// single corrupt file fails the parse loudly, but a source that has simply
@@ -96,8 +124,224 @@ const currentParser = (meta) => meta?.parserVersion === PARSER_VERSION
*
* `force` skips the hash check (an admin asking for a reimport). `approve`
* additionally accepts a vanished source.
*
* Which SOURCE it reads is decided here and nowhere else: the shard when uo-link
* is configured and enabled, a file otherwise, and always a file when the caller
* named one.
*/
async function refresh({ force = false, approve = false, path: pathOverride = '' } = {}) {
const override = String(pathOverride ?? '').trim()
if (override === '' && (await shardLinked())) {
return refreshFromShard({ force, approve })
}
return refreshFromFile({ force, approve, path: override })
}
/**
* Is there a shard to ask?
*
* Both halves matter. `baseUrl` alone is an install that has been configured and
* then switched off, and calling it would spend a 12 s timeout to learn what the
* row already says. Never throws: an unreadable config means "no shard", and the
* file path is a working answer.
*/
async function shardLinked() {
try {
const config = await uoLinkConfig.getSafe()
return Boolean(config?.enabled && config?.baseUrl)
} catch {
return false
}
}
/**
* Merge parsed sources in order, later winning.
*
* Shared by both paths, because the merge is the same question whichever end the
* base arrived from: what did each source contribute, and what did it override.
* The per-source breakdown is for the admin panel — an operator who adds an
* overlay wants to see it took effect, and `overrode: 0` on a file meant to
* re-label stock items says it did not.
*/
function mergeSources(groups) {
const merged = new Map()
const sources = []
for (const group of groups) {
let added = 0
let overrode = 0
for (const entry of group.entries) {
if (!Number.isInteger(entry.number)) continue
if (merged.has(entry.number)) overrode++
else added++
merged.set(entry.number, entry)
}
sources.push({
label: group.label,
kind: group.kind,
entries: group.entries.length,
added,
overrode,
})
}
return { entries: [...merged.values()], sources }
}
/** Overlay hashes as a `{ label: sha256 }` map, in merge order. */
function overlayHashes(files) {
const hashes = {}
for (const file of files) hashes[file.label] = file.sha256
return hashes
}
/** The overlay half of a stored fingerprint — everything under `custom/`. */
function onlyOverlays(hashes) {
if (!hashes) return null
const out = {}
for (const [label, sha] of Object.entries(hashes)) {
if (label.startsWith('custom/')) out[label] = sha
}
return out
}
/**
* Import with the shard as the base source.
*
* The gate is two-part, and neither part is something the shard can answer for
* us: has the client file changed (size/mtime/sha256, plus the shard's own
* `EXTRACTOR_VERSION`), and has any overlay beside the configured path changed.
* Either is drift; neither is the normal case.
*/
async function refreshFromShard({ force = false, approve = false } = {}) {
let fingerprint
try {
fingerprint = await bridge.fingerprint()
} catch (err) {
if (err instanceof bridge.ClilocBridgeError) {
return { status: 'unavailable', source: 'bridge', reason: err.message, code: err.code }
}
return { status: 'failed', source: 'bridge', reason: err.message }
}
const configured = await getClientPath()
const overlays = readOverlays(configured)
const hashes = overlayHashes(overlays.files)
const meta = await db.getMeta().catch(() => null)
if (
!force &&
bridge.sameSource(fingerprint, meta?.base) &&
sameSources(hashes, onlyOverlays(meta?.hashes)) &&
currentParser(meta)
) {
return {
status: 'unchanged',
source: 'bridge',
file: fingerprint.file,
count: meta.count ?? null,
customCount: overlays.files.length,
hashing: fingerprint.hashing,
}
}
// An overlay that was loaded last time and is not there now is refused rather
// than applied — an unmounted volume and a deliberate deletion look identical
// from here, and the wrong guess silently drops every name that file gave.
// The BASE is deliberately not part of this question: an install upgraded from
// the file pipeline is *supposed* to stop having one.
const gone = missingOverlays(hashes, meta?.hashes)
if (gone.length > 0 && !approve) {
return {
status: 'needsReview',
source: 'bridge',
reason: `${gone.length} previously-loaded cliloc overlay(s) are missing; the existing table is unchanged`,
missingSources: gone,
file: fingerprint.file,
}
}
let base
try {
base = await bridge.readCliloc({ lang: bridge.DEFAULT_LANGUAGE })
} catch (err) {
if (err instanceof bridge.ClilocBridgeError) {
return { status: 'unavailable', source: 'bridge', reason: err.message, code: err.code }
}
return { status: 'failed', source: 'bridge', reason: err.message }
}
const groups = [{ label: base.source.file, kind: 'shard', entries: base.entries }]
for (const file of overlays.files) {
try {
groups.push({ label: file.label, kind: 'custom', entries: parseCliloc(file.buffer) })
} catch (err) {
if (err instanceof ClilocFormatError) {
// Named, because "which of my six overlay files is malformed" is
// otherwise a guessing game.
return {
status: 'unavailable',
source: 'bridge',
reason: `${file.label}: ${err.message}`,
code: err.code,
}
}
return { status: 'failed', source: 'bridge', reason: err.message }
}
}
const merged = mergeSources(groups)
try {
const applied = await db.replaceAll(merged.entries, {
source: 'bridge',
base: fingerprint,
hashes,
parserVersion: PARSER_VERSION,
sources: merged.sources,
file: base.source.file,
bytes: fingerprint.size,
})
invalidate()
return {
status: 'imported',
source: 'bridge',
file: base.source.file,
count: applied.count,
parsed: merged.entries.length,
blank: applied.blank,
pages: base.source.pages,
sources: merged.sources,
// The shard says how many rows it holds; this is how many arrived. They
// agree, or the walk is wrong in a way no count on its own would show.
reported: base.source.reported,
received: base.source.received,
overlayProblem: overlays.problem ?? undefined,
acceptedMissing: gone.length > 0 ? gone : undefined,
}
} catch (err) {
return { status: 'failed', source: 'bridge', reason: err.message }
}
}
/**
* Import from a converted file on disk — the pre-protocol-8 pipeline, unchanged.
*
* Deprecated but supported: an install with no shard link has no other way to get
* a table, and development without a running ServUO is the same case.
*/
async function refreshFromFile({ force = false, approve = false, path: pathOverride = '' } = {}) {
// An explicit override wins outright — a one-off "use this file", which must
// not be silently overruled by the configured path the way an env default is.
const configured = pathOverride.trim() !== '' ? pathOverride.trim() : await getClientPath()
@@ -153,7 +397,10 @@ async function refresh({ force = false, approve = false, path: pathOverride = ''
}
try {
const applied = await db.replaceAll(parsed.entries, parsed.source)
// `source: 'file'` is what lets the NEXT refresh — and `status()` — tell a
// table built from a converted file from one built over the bridge. Without
// it an install that gains a shard link looks like it already imported.
const applied = await db.replaceAll(parsed.entries, { ...parsed.source, source: 'file' })
invalidate()
return {
status: 'imported',
@@ -177,9 +424,22 @@ async function refresh({ force = false, approve = false, path: pathOverride = ''
/**
* Boot hook. Best-effort by contract: it logs and returns, never throws, so a
* missing or malformed cliloc file can never stop the site coming up.
*
* **On the bridge it imports nothing**, deliberately. The file path can hash a
* local 5 MB file on every restart and skip in 14 ms; asking the shard would put
* a sidecar round trip in the boot sequence to answer a question whose answer is
* "no" every time except after a client patch — which is an operator action, and
* therefore something an operator can press a button for. Whatever table is
* loaded keeps serving, which is exactly what happens today when a restart finds
* nothing changed.
*/
async function refreshOnBoot() {
try {
if (await shardLinked()) {
log.info('cliloc table comes from the shard; import is admin-triggered (Admin → Shard)')
return { status: 'skipped', source: 'bridge', reason: 'the shard is the cliloc source' }
}
const result = await refresh()
switch (result.status) {
case 'imported':
@@ -220,8 +480,18 @@ async function refreshOnBoot() {
}
}
/** Everything the admin panel needs to describe cliloc state. */
/**
* Everything the admin panel needs to describe cliloc state.
*
* Two shapes, one per source, sharing every field a panel actually renders
* (`count`, `drift`, `problem`, `sources`, `missingSources`, `importedAt`). What
* differs is what `file` means and what a problem with it looks like: on the
* bridge it is the shard's own client file and the problems are transport ones,
* on disk it is a path an operator typed.
*/
async function status({ path: pathOverride = '' } = {}) {
if (pathOverride.trim() === '' && (await shardLinked())) return shardStatus()
const configured = pathOverride.trim() !== '' ? pathOverride.trim() : await getClientPath()
const meta = await db.getMeta().catch(() => null)
const loaded = await db.count().catch(() => 0)
@@ -280,6 +550,72 @@ async function status({ path: pathOverride = '' } = {}) {
}
}
/**
* Status when the shard is the source.
*
* The one thing worth knowing here that the file path has no equivalent of:
* `hashing`. The shard reports a null `sha256` for a client file it has not
* hashed yet — hashing the 343 MB of art and animation it also serves cannot fit
* in a 10 s reply, so it happens on its own thread — and a null hash means "ask
* again", never "changed". Drift falls back to (size, mtime) meanwhile, which is
* the same gate the shard itself applies, so an operator is never blocked from
* importing by a hash that has not landed.
*/
async function shardStatus() {
const configured = await getClientPath()
const meta = await db.getMeta().catch(() => null)
const loaded = await db.count().catch(() => 0)
const overlays = readOverlays(configured)
const hashes = overlayHashes(overlays.files)
let fingerprint = null
let problem = overlays.problem ?? null
let code = null
try {
fingerprint = await bridge.fingerprint()
} catch (err) {
problem = err.message
code = err.code ?? null
}
const drift = fingerprint
? !bridge.sameSource(fingerprint, meta?.base) ||
!sameSources(hashes, onlyOverlays(meta?.hashes)) ||
!currentParser(meta)
: null
return {
source: 'bridge',
configured: true,
// The overlay directory, which is all the path setting still selects on this
// source. Reported so a panel can say where `custom/` is being read from.
path: configured,
file: fingerprint?.file ?? bridge.SOURCE_FILE,
fileReadable: Boolean(fingerprint),
problem,
code,
drift,
count: loaded,
shard: fingerprint
? {
size: fingerprint.size,
mtime: fingerprint.mtime,
sha256: fingerprint.sha256,
extractorVersion: fingerprint.extractorVersion,
hashing: fingerprint.hashing,
complete: fingerprint.complete,
}
: null,
sources: Object.keys(hashes),
loadedSources: meta?.sources ?? null,
missingSources: missingOverlays(hashes, meta?.hashes),
importedAt: meta?.importedAt ?? null,
sourceBytes: meta?.base?.size ?? meta?.bytes ?? null,
}
}
// ── Lookup ─────────────────────────────────────────────────────────────────
//
// Resolution happens SERVER-SIDE, not in the browser. Two reasons: the table is
@@ -359,6 +695,7 @@ module.exports = {
SETTING_KEY,
getClientPath,
setClientPath,
shardLinked,
refresh,
refreshOnBoot,
status,

View File

@@ -14,6 +14,7 @@
const db = require('./shardMarket.db')
const clilocs = require('../shardClilocs/shardClilocs.model')
const itemArt = require('../shardAssets/shardItemArt.model')
const log = require('../../core').logger('shard-market')
// Defense in depth on top of the shard's own MarketMaxListings cap. The shard is
@@ -173,6 +174,13 @@ async function upsertVendor(ev) {
const vendor = flattenFrame(ev)
const items = await shapeItems(ev)
await db.replaceVendor(vendor, items)
// The listings name (itemId, hue) pairs, which are §5 asset keys (phase 5).
// Noticing them here is what makes the warm pass find a newly listed item's
// picture before anyone looks at the shop, rather than one page view later.
// A hint, never a queue — the pass derives its real set from this table, so a
// hint lost to a restart costs nothing.
itemArt.notice(items)
}
/** Ingest one `vendor.listing.remove` frame. */
@@ -273,8 +281,14 @@ async function search({
const info = await db.meta()
// Each listing gets `art`: the filename of the item's picture under
// uploads/items/, or null where this site does not hold one (phase 5). One
// query for the page, off the listing shape rather than the SQL, so the search
// itself stays the search and a picture lookup that fails costs a picture.
const listings = await itemArt.decorate(rows.map(shapeListing))
return {
listings: rows.map(shapeListing),
listings,
total,
limit,
offset,
@@ -292,7 +306,7 @@ async function getVendor(serial, { limit = 250, offset = 0 } = {}) {
const row = await db.getVendor(serial)
if (!row) return null
const items = await db.listVendorItems(serial, { limit, offset })
return { ...shapeVendor(row), items: items.map(shapeItem) }
return { ...shapeVendor(row), items: await itemArt.decorate(items.map(shapeItem)) }
}
/** Index size, staleness, and the facet/region filter options. */

View File

@@ -11,16 +11,31 @@ const { secretBox } = require('../../core')
// Only used before an admin has saved anything — the stored row wins once it exists,
// and UOLINK_PROTOCOL still overrides for an operator running an older sidecar.
//
// This says 5 because this build handles protocol 5's frames: house.decay's `schedule`,
// vendor.listing's `ownerAcct` + `fees`, and the new `account.login.result` kind.
// This says 8 because this build speaks protocol 8: the idempotency key and the
// participation ledger (6), the world verbs plus the targeted lease planes (7), and
// the Asset Bridge (8) -- of which this module is the first consumer, importing the
// cliloc table over `GET /cliloc` instead of reading a file an operator converted by
// hand (docs/link/v8.md §9).
//
// It said 4 before that, and 3 for a while after protocol 4 shipped — which is the bug
// this constant is now the fix for. A FRESH install pinned 3, the sidecar answered
// It said 4 before 5, and 3 for a while after protocol 4 shipped — which is the bug this
// constant was introduced to fix. A FRESH install pinned 3, the sidecar answered
// `409 protocol version mismatch` to every REST call, and a new deployment read nothing
// from its shard until an admin edited the number by hand in Admin → Shard. Bumping it
// in the SAME change as the emitters is the discipline that prevents a repeat; see the
// matching cutover in db/schema.sql.
const DEFAULT_PROTOCOL = Number(process.env.UOLINK_PROTOCOL) || 5
// from its shard until an admin edited the number by hand in Admin → Shard.
//
// **And it happened again, twice, in Phases 11a and 12a** — this constant and the two in
// `db/schema.sql` all sat at 5 while the wire went to 6 and then 7, so every sidecar call
// on a real deployment would have been refused. Both live walks set the column by hand
// while standing the rig up, which is exactly what makes a migration nobody runs
// invisible. Phase 12b carries all three to 7.
//
// **Nothing in this repo can check this against the wire**, and that is worth knowing
// before trusting the test that guards it: `schemaFragment.test.js` asserts the three
// declarations agree WITH EACH OTHER, which is a real check — they drifted apart once —
// but all three being equally stale passes it. The wire's version lives in `link`
// (`PROTOCOL_VERSION`) and the overlay's in `servuo-plugins/overlay.toml`; the thing that
// actually pairs them is the installer's bundle check, at deploy time. So bumping this in
// the same change as the emitters is still the discipline, and no test here replaces it.
const DEFAULT_PROTOCOL = Number(process.env.UOLINK_PROTOCOL) || 8
function toSafe(row) {
if (!row) {

View File

@@ -28,6 +28,7 @@ const shardOps = require('./shardOps.controller')
const shardVisibility = require('./shardVisibility.controller')
const shardAtlas = require('./shardAtlas.controller')
const shardClilocs = require('./shardClilocs.controller')
const shardAssets = require('./shardAssets.controller')
const selfShard = require('../player/shard.controller')
const { requireRole, validate } = core.middleware
@@ -250,8 +251,8 @@ shardRouter.get(
shardRouter.get(
'/atlas',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Spawn atlas status: path, drift, counts, pending review (admin only)'
// #swagger.description = 'Where the ServUO tree is, whether it can be read, whether its source files have drifted from the loaded atlas, and any refresh staged for approval. The public /atlas/meta route reports the game world only; the filesystem detail is here.'
// #swagger.summary = 'Spawn atlas status: source, drift, counts, pending review (admin only)'
// #swagger.description = 'Which source the atlas is built from — the linked shard over uo-link, or a local ServUO tree whether it can be read, whether its source files have drifted from the loaded atlas, and any refresh staged for approval. On the bridge, reading drift costs one shard round trip for the file manifest (hashes, no bytes). The public /atlas/meta route reports the game world only; this detail is here.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Atlas status', content: { "application/json": { schema: { $ref: "#/components/schemas/UoAtlasStatus" } } } } */
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
@@ -261,8 +262,8 @@ shardRouter.get(
shardRouter.post(
'/atlas/import',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Re-import the spawn atlas from the ServUO tree (admin only)'
// #swagger.description = 'Applies a map change without a restart. `force` reimports even when the source hashes match what is loaded. A refresh that would REMOVE a facet is still staged for approval rather than applied — that decision is never taken implicitly. An unreadable tree answers 200 with status "unavailable" rather than 500: the refresh contract reports outcomes instead of throwing, and the admin needs to be told what is wrong with the path.'
// #swagger.summary = 'Re-import the spawn atlas from its source (admin only)'
// #swagger.description = 'Applies a map change without a restart — and on a linked shard it is the only thing that does, because boot never calls the shard for this. `force` reimports even when the source hashes match what is loaded. A refresh that would REMOVE a facet is still staged for approval rather than applied — that decision is never taken implicitly. An unreadable source answers 200 with status "unavailable" rather than 500: the refresh contract reports outcomes instead of throwing, and the admin needs to be told what is wrong.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: false, content: { "application/json": { schema: { type: "object", properties: { force: { type: "boolean", description: "Reimport even if the tree is unchanged." } } } } } } */
/* #swagger.responses[200] = { description: 'What happened', content: { "application/json": { schema: { $ref: "#/components/schemas/UoAtlasRefreshResult" } } } } */
@@ -307,10 +308,17 @@ shardRouter.put(
)
// ── Cliloc table (admin only) ─────────────────────────────────────────────
// UO's id → display-string map, converted once by the operator from their own
// client (docs/website/CLILOCS.md). Sits beside the atlas for the same reason:
// it is static content derived from operator-supplied files rather than anything
// the sidecar sends, and operating it is shard administration.
// UO's id → display-string map, read from the shard's own UO client over the
// bridge (docs/link/v8.md §9, docs/website/CLILOCS.md). Sits beside the atlas for
// the same reason: it is static content derived from the operator's own files
// rather than anything the sidecar streams, and operating it is shard
// administration.
//
// Protocol 8 changed where the base table comes from, not what these routes are:
// the shard decompresses `Cliloc.enu` and serves it paged, so an operator no
// longer converts anything by hand. Import stays an explicit admin action,
// because the only thing that changes a client's table is an operator patching
// their client.
//
// There is deliberately NO public counterpart. The table is never served as a
// table — 123k rows would dwarf any page that used it, and the Android client
@@ -320,7 +328,7 @@ shardRouter.get(
'/clilocs',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Cliloc table status: sources, drift, entry count (admin only)'
// #swagger.description = 'Where the cliloc sources are, whether they can be read, how many entries are loaded, and whether the files on disk have drifted from them. The table is built from a SET of sources — the converted client table plus every operator-maintained overlay under `custom/`, which is how shard-added and shard-edited items get names. `missingSources` lists any source that was loaded before and is now gone; an import refuses that without `approve`. A shard with nothing configured is a supported state — item names simply render as ids.'
// #swagger.description = 'Where the cliloc sources are, whether they can be read, how many entries are loaded, and whether they have drifted from what is loaded. `source` says which pipeline is in use: `bridge` (the shard reads its own client — the normal case once uo-link is configured) or `file` (a converted file on disk, deprecated, kept for installs with no shard link). On the bridge, `shard` carries the client files size, mtime, hash and the shards extractor version, and `shard.hashing: true` means a null hash is “not computed yet”, not “changed”. The table is always a SET: the base plus every operator-maintained overlay under `custom/`, which is how shard-added and shard-edited items get names. `missingSources` lists any overlay that was loaded before and is now gone; an import refuses that without `approve`. A shard with no source at all is a supported state — item names simply render as ids.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Cliloc status', content: { "application/json": { schema: { $ref: "#/components/schemas/UoClilocStatus" } } } } */
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
@@ -330,8 +338,8 @@ shardRouter.get(
shardRouter.post(
'/clilocs/import',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Re-import the cliloc table from its source files (admin only)'
// #swagger.description = 'Applies a client patch, or a change to the shards own overlay files, without a restart. `force` reimports even when the source hashes match what is loaded. `approve` accepts a refresh in which a previously-loaded source has VANISHED — refused by default, because an unmounted volume and a deliberate deletion are indistinguishable from the server, and the wrong guess silently drops every name that file contributed. A missing path — or the common mistake of pointing at the clients own COMPRESSED Cliloc.enu — answers 200 with status "unavailable" and the reason, rather than 500: the refresh contract reports outcomes instead of throwing, and the admin needs to be told which file to convert.'
// #swagger.summary = 'Re-import the cliloc table from its source (admin only)'
// #swagger.description = 'Applies a client patch, or a change to the shards own overlay files, without a restart. On the bridge this is the ONLY thing that imports — boot deliberately does not call the shard — so it is what an operator presses after patching their client. `force` reimports even when the sources are unchanged. `approve` accepts a refresh in which a previously-loaded overlay has VANISHED — refused by default, because an unmounted volume and a deliberate deletion are indistinguishable from the server, and the wrong guess silently drops every name that file contributed. Nothing here throws for an operator-visible problem: a shard that is down, an asset plane the operator has switched off, a client with no cliloc file, or a malformed overlay all answer 200 with status "unavailable" and a reason naming what to fix.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: false, content: { "application/json": { schema: { type: "object", properties: { force: { type: "boolean", description: "Reimport even if the sources are unchanged." }, approve: { type: "boolean", description: "Accept a refresh in which a previously-loaded source has vanished." } } } } } } */
/* #swagger.responses[200] = { description: 'What happened', content: { "application/json": { schema: { $ref: "#/components/schemas/UoClilocRefreshResult" } } } } */
@@ -344,10 +352,10 @@ shardRouter.post(
shardRouter.put(
'/clilocs/path',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Set the cliloc source the site reads from (admin only)'
// #swagger.description = 'Accepts either the converted base file itself or a directory to search. Overlays are read from a `custom/` directory beside it either way pointing at a file does not forfeit them. Persisted as a setting, which wins over the UO_CLIENT_PATH deploy default so the mount can move without a redeploy. Blank clears it and resolution is skipped on the next boot. Deliberately does not import as a side effect — the response carries the refreshed status so the panel can offer that as the next step.'
// #swagger.summary = 'Set the cliloc path the site reads overlays (and any file base) from (admin only)'
// #swagger.description = 'On an install with uo-link configured this selects only where `custom/` overlays are read from — the base table comes from the shard. Without a shard link it is also where the converted base file is looked for, which is the deprecated pre-protocol-8 pipeline. Accepts either a file or a directory to search; overlays are read from a `custom/` directory beside it either way, so pointing at a file does not forfeit them. Persisted as a setting, which wins over the UO_CLIENT_PATH deploy default so the mount can move without a redeploy. Blank clears it. Deliberately does not import as a side effect — the response carries the refreshed status so the panel can offer that as the next step.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["path"], properties: { path: { type: "string", description: "Path to the converted cliloc file, or a directory containing one. Blank disables resolution." } } } } } } */
/* #swagger.requestBody = { required: true, content: { "application/json": { schema: { type: "object", required: ["path"], properties: { path: { type: "string", description: "Directory holding the custom/ overlays (and, with no shard link, a converted base file). Blank clears it." } } } } } } */
/* #swagger.responses[200] = { description: 'Cliloc status after the change', content: { "application/json": { schema: { $ref: "#/components/schemas/UoClilocStatus" } } } } */
adminOnly,
body('path').isString().isLength({ max: 512 }),
@@ -355,6 +363,58 @@ shardRouter.put(
shardClilocs.setPath,
)
// ── Client assets (admin only) ────────────────────────────────────────────
// Creature artwork, read from the shard's own UO client over the bridge
// (docs/link/v8.md §6, §8). Sits beside the cliloc routes for the same reason
// they sit beside the atlas: static content derived from the operator's own
// files, and operating it is shard administration.
//
// There is deliberately NO public counterpart. The pictures are served as
// ordinary files under `/uploads`, and `shard_spawn_creatures.art` names them on
// the atlas responses the site already returns — so nothing public needs to know
// this pipeline exists.
shardRouter.get(
'/assets',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Client asset import status: what is loaded, what the shard has, whether they differ (admin only)'
// #swagger.description = 'What the site currently holds (the imported body catalogue, how many sprites are stored, how many atlas creatures resolved to a body id) beside what the shard reports for the client files those pictures come from. `drift: true` means the client files have changed since the last import — press Import. `shard.hashing: true` means a null hash is “not computed yet”, not “changed”: the shard hashes 195 MB anim files off the request path. `shard.imaging.ok: false` is the named NO_IMAGING state — a Linux shard host without libgdiplus cannot render a sprite at all, and the reason names the package to install. A shard with no link configured, or one that is down, is a reported state with a reason rather than an error.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.responses[200] = { description: 'Asset import status', content: { "application/json": { schema: { $ref: "#/components/schemas/UoAssetStatus" } } } } */
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
shardAssets.getStatus,
)
shardRouter.post(
'/assets/import',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Import creature artwork from the shards UO client (admin only)'
// #swagger.description = 'Walks the shards asset manifest, fetches only the sprites whose hash changed, stores them under uploads/atlas/, re-resolves every atlas creature to a body id and points each creature at its picture. This is the ONLY thing that imports — boot deliberately never calls the shard — so it is what an operator presses after patching their client. `force` re-imports even when the client files are unchanged. `approve` accepts a catalogue that no longer offers assets this site holds; refused by default, because an unmounted client volume and a deliberate downgrade are indistinguishable from the server and the wrong guess deletes artwork. An operator-supplied `spawnAtlas.art.json` always wins over an imported sprite. Nothing here throws for an operator-visible problem: a shard that is down, an asset plane switched off, or a host that cannot render images all answer 200 with status "unavailable" and a reason naming what to fix. Assets a client simply does not have are NOT failures — two thirds of the playable ghost and gargoyle bodies have no art on a stock client.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: false, content: { "application/json": { schema: { type: "object", properties: { force: { type: "boolean", description: "Import even if the shards client files are unchanged." }, approve: { type: "boolean", description: "Accept a catalogue that no longer offers assets this site holds." } } } } } } */
/* #swagger.responses[200] = { description: 'What happened', content: { "application/json": { schema: { $ref: "#/components/schemas/UoAssetImportResult" } } } } */
adminOnly,
body('force').optional().isBoolean(),
body('approve').optional().isBoolean(),
validate,
shardAssets.importAssets,
)
shardRouter.post(
'/assets/warm',
// #swagger.tags = ['Admin · Shard']
// #swagger.summary = 'Fetch item and land artwork the site is missing, now (admin only)'
// #swagger.description = 'Runs one pass of the item-art warm loop instead of waiting for its timer. The pass works out which item pictures this site's own rows name — every distinct (ItemID, hue) on a player vendor, plus anything a character sheet has shown since the last pass — and fetches the ones it does not already hold from the shard, hued and stored under uploads/items/. There is deliberately NO manifest and no bulk import here: the client addresses 49,152 item graphics times three thousand hues, so the working set is defined by what the site actually displays. `force` re-fetches pictures the site already holds, which is how an operator recovers a wiped uploads volume. `limit` bounds one pass; the default is 400, because the shard serves one asset request at a time and a pass must not hold that slot against an import. Nothing throws for an operator-visible problem: no shard configured, a shard that is down, an asset plane switched off, a host with no libgdiplus, or a plugin overlay too old to serve item art all answer 200 with status "unavailable"/"skipped" and a reason naming what to fix.'
// #swagger.security = [{ "cookieAuth": [] }, { "bearerAuth": [] }]
/* #swagger.requestBody = { required: false, content: { "application/json": { schema: { type: "object", properties: { force: { type: "boolean", description: "Re-fetch pictures this site already holds." }, limit: { type: "integer", description: "How many keys this pass may fetch (1-2000)." } } } } } } */
/* #swagger.responses[200] = { description: 'What the pass did', content: { "application/json": { schema: { $ref: "#/components/schemas/UoItemArtWarmResult" } } } } */
/* #swagger.responses[403] = { description: 'Admin role required', content: { "application/json": { schema: { $ref: "#/components/schemas/Error" } } } } */
adminOnly,
body('force').optional().isBoolean(),
body('limit').optional().isInt({ min: 1, max: 2000 }),
validate,
shardAssets.warmItemArt,
)
// ── Feature visibility (admin only) ───────────────────────────────────
// Who can see which shard surface, and which sensitive fields within it. This
// decides what ANONYMOUS visitors get, so it sits above the moderator tier.

View File

@@ -0,0 +1,143 @@
// ── Admin · Client assets ──────────────────────────────────────────────────
//
// Operating the asset import: what the site holds, what the shard's client files
// currently are, and a re-import after a client patch (docs/link/v8.md §6, §8,
// §14, protocol 8 phase 3).
//
// The policy lives in the model. This controller does three things and no more:
// it validates input, it maps an import RESULT onto an HTTP status, and it
// records the action in the admin activity log.
//
// **An import result is not an exception**, exactly as for clilocs. A shard that
// is down, an asset plane the operator has switched off, a Linux host with no
// libgdiplus, a client patched halfway through the walk — each is a 200 carrying
// `status: 'unavailable'` and a reason naming what to fix, not a 500 that says
// only "something broke". The one thing that DOES 500 is this file having a bug.
//
// **This is the only thing that imports.** Boot never calls the shard for assets,
// for the same reason it stopped calling it for clilocs: the files change when an
// operator patches their client, which is an event they know about and the site
// does not. So this endpoint is what an operator presses afterwards.
//
// Phase 8 built the panel these serve (`Admin → Client Files`) and added one
// thing to this pair: the import records a summary of what it did, and the
// vanished keys it refuses to apply come back with the pictures they currently
// have. Both exist because an operator pressing Update needs to see an answer,
// and the audit log — which still receives every action here — is one unfiltered
// list of every admin action on the site, so an import from three client patches
// ago cannot be found in it (org lead, 2026-09-14).
const assets = require('../../model/shardAssets/shardAssets.model')
const itemArt = require('../../model/shardAssets/shardItemArt.model')
const { activity } = require('../../core')
const log = require('../../core').logger('admin-shard-assets')
// GET /admin/shard/assets — what is loaded, what the shard says, whether they
// disagree. No public counterpart: the assets themselves are served as ordinary
// files under /uploads, and this is the operating view of the import.
async function getStatus(req, res) {
try {
return res.json(await assets.getStatus())
} catch (err) {
log.error('getStatus', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/assets/import — import or update the body catalogue, then
// re-resolve the atlas's creatures and re-derive their artwork.
//
// `force` re-imports even when the client files are unchanged. It is also how an
// operator recovers a wiped uploads volume: the database still holds every hash,
// so the ordinary gate would report "unchanged" while every picture is missing.
// (The import checks for the file on disk per key as well, so that case usually
// heals itself — `force` is the answer when it does not.)
//
// `approve` accepts a catalogue that no longer offers keys this site holds.
// Refused by default because an unmounted client volume and a deliberate
// downgrade look identical from the server, and the wrong guess deletes artwork.
async function importAssets(req, res) {
try {
const force = !!req.body?.force
const approve = !!req.body?.approve
// From the session, never the body — the same rule the in-game ops routes
// apply, and for the same reason: this is recorded as who did it.
const result = await assets.importAssets({ force, approve, by: req.user?.username ?? null })
await activity.log({
req,
action: 'shard.assets.import',
detail: {
force,
approve,
status: result.status,
code: result.code ?? null,
assets: result.assets ?? null,
fetched: result.fetched ?? null,
written: result.written ?? null,
removed: result.removed ?? null,
// The body pass is logged as its own tally rather than as a single
// number: `unknown` means the spawn files name a type this shard's
// scripts do not define, which is real drift an operator should see, and
// it reads identically to `failed` if both are summed into "not resolved".
bodies: result.bodies?.tally ?? null,
vanished: result.vanishedCount ?? null,
},
})
return res.json(result)
} catch (err) {
log.error('importAssets', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
// POST /admin/shard/assets/warm — run one item-art warm pass now.
//
// The pass runs on its own timer and needs no operator, so this exists for the
// two moments where waiting for the interval is the wrong answer: an operator who
// has just configured the bridge and wants to see it work, and one who has just
// patched their client and would rather not wait for pictures to refresh.
//
// `force` re-fetches keys the site already holds. The body import's `force` means
// the same thing for the same reason — a wiped uploads volume leaves every
// database row correct and every picture missing, and only an explicit re-fetch
// recovers it.
//
// It is bounded: one pass asks for at most `limit` keys, because the shard's
// asset plane serves one request at a time and a pass must not hold that slot
// against the operator's own import.
async function warmItemArt(req, res) {
try {
const force = !!req.body?.force
const limit = Number.isFinite(Number(req.body?.limit)) ? Number(req.body.limit) : undefined
const result = await itemArt.warm({ force, ...(limit ? { limit } : {}) })
await activity.log({
req,
action: 'shard.assets.warm',
detail: {
force,
limit: limit ?? null,
status: result.status,
code: result.code ?? null,
wanted: result.wanted ?? null,
asked: result.asked ?? null,
written: result.written ?? null,
remaining: result.remaining ?? null,
},
})
return res.json(result)
} catch (err) {
log.error('warmItemArt', err)
return res.status(500).json({ message: 'Internal Server Error' })
}
}
module.exports = {
getStatus,
importAssets,
warmItemArt,
}

View File

@@ -1,8 +1,8 @@
// ── Admin · Cliloc table ───────────────────────────────────────────────────
//
// Operating the cliloc import: where the converted cliloc file is, whether it
// has drifted from what is loaded, and a forced reimport after a client patch
// (docs/website/CLILOCS.md).
// Operating the cliloc import: which source the table comes from, whether it has
// drifted from what is loaded, and a reimport after a client patch
// (docs/link/v8.md §9, docs/website/CLILOCS.md).
//
// The policy lives in the model. This controller does three things and no more:
// it validates input, it maps a refresh RESULT onto an HTTP status, and it
@@ -10,11 +10,16 @@
//
// **A refresh result is not an exception.** `shardClilocs.refresh()` reports
// `unavailable` / `failed` rather than throwing, because the boot path must never
// be stopped by a bad file. That contract is preserved here: a missing file, or
// the single most likely operator mistake — pointing at the client's own
// COMPRESSED `Cliloc.enu` — is a 200 carrying `status: 'unavailable'` and the
// reason, not a 500. A 500 would say only "something broke"; the operator needs
// to be told which file to convert.
// be stopped by a bad source. That contract is preserved here, and protocol 8
// widened the set of things it covers: a shard that is down, an asset plane the
// operator has switched off, a client with no cliloc file, a client patched
// halfway through the import — plus everything the file pipeline could already
// report. Each is a 200 carrying `status: 'unavailable'` and a reason naming what
// to fix, not a 500 that says only "something broke".
//
// **Import matters more than it used to.** On the bridge, boot deliberately does
// not call the shard, so this endpoint is the only thing that refreshes the
// table — the operator presses it after patching their client.
const clilocs = require('../../model/shardClilocs/shardClilocs.model')
const market = require('../../model/shardMarket/shardMarket.model')
@@ -69,6 +74,10 @@ async function importClilocs(req, res) {
force,
approve,
status: result.status,
// Which pipeline actually ran. Worth having in the audit log for the
// same reason it is in the status: an operator debugging a stale table
// needs to know whether the site asked the shard or read a file.
source: result.source ?? null,
count: result.count ?? null,
missingSources: result.missingSources ?? result.acceptedMissing ?? null,
},
@@ -80,12 +89,16 @@ async function importClilocs(req, res) {
}
}
// PUT /admin/shard/clilocs/path — point the site at a different cliloc file.
// PUT /admin/shard/clilocs/path — point the site at a different cliloc path.
//
// On an install with uo-link configured this selects where `custom/` OVERLAYS are
// read from; the base table comes from the shard either way. Without a shard link
// it is also where the converted base file is looked for.
//
// Persisted as a setting, which wins over the UO_CLIENT_PATH env default so an
// operator can move the mount without a redeploy. Blank clears it, which turns
// resolution off (boot skips, the loaded table keeps serving) — a legitimate
// thing to want, so it is allowed rather than validated away.
// overlay resolution off (the loaded table keeps serving) — a legitimate thing to
// want, so it is allowed rather than validated away.
//
// Deliberately does NOT import as a side effect, for the same reason the atlas
// path does not: changing where the table reads from and reloading it are

View File

@@ -11,6 +11,7 @@ const uoLinkClient = require('../../utils/uoLinkClient')
const shardLinks = require('../../model/shardLinks/shardLinks.model')
const shardState = require('../../model/shardState/shardState.model')
const shardClilocs = require('../../model/shardClilocs/shardClilocs.model')
const itemArt = require('../../model/shardAssets/shardItemArt.model')
const { activity } = require('../../core')
const gameSignup = require('../../utils/gameSignup')
const { salesForAccounts } = require('../../utils/shardSales')
@@ -71,6 +72,28 @@ async function resolveProfileClilocs(profile) {
}
}
/**
* Attach a picture to each equipped item (docs/link/v8.md §5, §11 — phase 5).
*
* The equipment list is the other place on this site that carries (itemId, hue),
* and unlike the marketplace it is LIVE: the profile is fetched from the shard per
* request and stored nowhere, so there is no table a warm pass could derive these
* keys from. That is what `notice` is for, and `decorate` does both — it fills in
* every picture we already hold and remembers the ones we do not, so a character
* whose sheet renders without art once renders with it a few minutes later.
*
* It never asks the shard. §17.11: the page serves what is stored and the warm
* pass does the fetching, because a route that fetched would let any visitor drive
* the shard's single-slot asset plane from a URL.
*/
async function resolveProfileArt(profile) {
const equipment = Array.isArray(profile?.equipment) ? profile.equipment : []
if (equipment.length === 0) return
await itemArt.decorate(equipment)
}
// Decorate a char.profile with cross-links from our own board data: the guild the
// character leads and any city governorship on its account, plus resolved cliloc
// names. Best-effort — a failure here never fails the profile (it's a nicety,
@@ -85,6 +108,7 @@ async function enrichCharProfile(profile) {
if (govs.length) profile.governorOf = govs.map((g) => g.city)
}
await resolveProfileClilocs(profile)
await resolveProfileArt(profile)
} catch (err) {
log.warn('enrichCharProfile failed', { serial: profile.serial, message: err.message })
}

View File

@@ -49,11 +49,14 @@ function describe(result) {
switch (result.status) {
case 'skipped':
return (
'No ServUO path configured — nothing to import.\n' +
'Set one with SERVUO_PATH, the admin panel, or --servuo <path>.\n'
'No atlas source — nothing to import.\n' +
'Either link a shard (Admin → Shard) or set a tree path with SERVUO_PATH, ' +
'the admin panel, or --servuo <path>.\n'
)
case 'unavailable':
return `ServUO tree unavailable: ${result.reason}\n`
return result.source === 'bridge'
? `The shard could not serve its configuration tree: ${result.reason}\n`
: `ServUO tree unavailable: ${result.reason}\n`
case 'unchanged':
return `Atlas is already up to date${result.reason ? ` (${result.reason})` : ''}.\n`
case 'needsReview': {

View File

@@ -454,12 +454,18 @@ module.exports = {
},
UoAtlasStatus: {
type: 'object',
description: 'Admin view of atlas state: where the tree is, whether it is readable, whether it has drifted from what is loaded, and any refresh staged for review.',
description: 'Admin view of atlas state: which source the tree comes from, whether it is readable, whether it has drifted from what is loaded, and any refresh staged for review.',
properties: {
configured: { type: 'boolean', example: true },
path: { type: 'string', example: '/srv/servuo' },
source: {
type: 'string',
enum: ['bridge', 'fs'],
description: '`bridge`: the shard serves its own configuration files over uo-link (protocol 8 phase 7, the normal case once a shard is linked). `fs`: a ServUO tree the website can read directly — development and same-host installs, and the only source where boot re-imports by itself.',
example: 'bridge',
},
path: { type: 'string', description: 'The local tree path, or `the shard bridge` when that is the source.', example: 'the shard bridge' },
treeReadable: { type: 'boolean', example: true },
drift: { type: 'boolean', nullable: true, description: 'True when the tree\'s source hashes differ from the loaded atlas. NULL when the tree could not be read.', example: false },
drift: { type: 'boolean', nullable: true, description: 'True when the source file hashes differ from the loaded atlas. NULL when the source could not be read. On the bridge this is answered from the shard\'s file MANIFEST — hashes only, no file bytes.', example: false },
facets: { type: 'array', items: { type: 'string' } },
importedAt: { type: 'string', format: 'date-time', nullable: true },
counts: { type: 'object', nullable: true, additionalProperties: true },
@@ -481,7 +487,15 @@ module.exports = {
example: 'imported',
},
reason: { type: 'string', nullable: true },
path: { type: 'string', nullable: true },
source: {
type: 'string',
enum: ['bridge', 'fs'],
nullable: true,
description: 'Which end this attempt read from. Absent only on `skipped`, where there was no source at all.',
example: 'bridge',
},
path: { type: 'string', nullable: true, description: 'The local tree path, or `the shard bridge`.' },
code: { type: 'string', nullable: true, description: 'On `unavailable`: NO_PATH, NOT_FOUND, NO_REGIONS or NO_SPAWNS from a local tree; DISABLED, SOURCE_CHANGED, INCOMPLETE, MALFORMED, BUSY, SHARD_DOWN or TOO_LARGE from the bridge.' },
counts: { type: 'object', nullable: true, additionalProperties: true },
addedFacets: { type: 'array', items: { type: 'string' } },
removedFacets: { type: 'array', items: { type: 'string' } },
@@ -490,21 +504,40 @@ module.exports = {
UoClilocStatus: {
type: 'object',
description:
'Admin view of cliloc state: where the converted file is, whether it is readable, how many entries are loaded, and whether the file has drifted from them. `configured: false` is a supported state — item names then render as ids.',
'Admin view of cliloc state: which source the base table comes from, whether it can be read, how many entries are loaded, and whether anything has drifted from them. Nothing configured at all is a supported state — item names then render as ids.',
properties: {
source: {
type: 'string',
enum: ['bridge', 'file'],
description: '`bridge`: the shard reads its own UO client (protocol 8, the normal case). `file`: a converted file on disk — the pre-protocol-8 pipeline, deprecated, kept for installs with no shard link.',
example: 'bridge',
},
configured: { type: 'boolean', example: true },
path: { type: 'string', example: '/srv/uo-client' },
file: { type: 'string', nullable: true, description: 'The file actually resolved, when the path is a directory.', example: '/srv/uo-client/clilocs.tsv' },
path: { type: 'string', description: 'On the bridge: where `custom/` overlays are read from. On a file source: the base path too.', example: '/srv/uo-client' },
file: { type: 'string', nullable: true, description: 'The base file in use — the shards own `cliloc.enu` on the bridge, the resolved local file otherwise.', example: 'cliloc.enu' },
fileReadable: { type: 'boolean', example: true },
problem: { type: 'string', nullable: true, description: 'Why the file cannot be used, when it cannot. Set (with code COMPRESSED) for a readable-but-unconverted client file.', example: null },
code: { type: 'string', nullable: true, description: 'Machine-readable cause of `problem`.', enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED'] },
problem: { type: 'string', nullable: true, description: 'Why the base cannot be used, when it cannot: a shard that is down or has assets switched off, or (on a file source) a missing or still-compressed file.', example: null },
code: { type: 'string', nullable: true, description: 'Machine-readable cause of `problem`.', enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED', 'DISABLED', 'NO_SOURCE', 'SHARD_DOWN', 'PROTOCOL', 'BUSY', 'UNAVAILABLE'] },
shard: {
type: 'object',
nullable: true,
description: 'Present on the bridge: the shards own cliloc file as it is right now. `hashing: true` with a null `sha256` means the hash has not been computed yet — “ask again”, not “changed”.',
properties: {
size: { type: 'integer', example: 4989921 },
mtime: { type: 'integer', description: 'Unix milliseconds.', example: 1757462400000 },
sha256: { type: 'string', nullable: true },
extractorVersion: { type: 'integer', description: 'The version of the shards extraction code. A bump makes everything derived from it drift.', example: 1 },
hashing: { type: 'boolean', example: false },
complete: { type: 'boolean', description: 'Every client file has a hash.', example: true },
},
},
drift: { type: 'boolean', nullable: true, description: 'True when any source hash differs from the loaded table. NULL when the sources could not be read or are not usable.', example: false },
count: { type: 'integer', description: 'Entries currently loaded.', example: 67496 },
sources: {
type: 'array',
items: { type: 'string' },
description: 'Every source found now, root-relative, base first then overlays in merge order.',
example: ['clilocs.plain', 'custom/uomysticmoon.tsv'],
example: ['custom/uomysticmoon.tsv'],
},
loadedSources: {
type: 'array',
@@ -514,7 +547,7 @@ module.exports = {
type: 'object',
properties: {
label: { type: 'string', example: 'custom/uomysticmoon.tsv' },
kind: { type: 'string', enum: ['base', 'custom'], example: 'custom' },
kind: { type: 'string', enum: ['shard', 'base', 'custom'], description: '`shard` is the table read over the bridge; `base` a converted file on disk.', example: 'custom' },
entries: { type: 'integer', example: 37 },
added: { type: 'integer', description: 'Ids this source introduced.', example: 25 },
overrode: { type: 'integer', description: 'Ids it replaced from an earlier source.', example: 12 },
@@ -543,17 +576,22 @@ module.exports = {
example: 'imported',
},
reason: { type: 'string', nullable: true },
source: { type: 'string', nullable: true, enum: ['bridge', 'file'], description: 'Which source this refresh read.', example: 'bridge' },
code: {
type: 'string',
nullable: true,
description: 'Machine-readable cause. `COMPRESSED` means the client\'s own Cliloc.enu was supplied instead of a converted one.',
enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED', 'TRUNCATED', 'EMPTY', 'NOT_BUFFER'],
description: 'Machine-readable cause. Bridge codes describe the shard (`DISABLED`: the operator switched the asset plane off; `NO_SOURCE`: its client has no cliloc file; `SHARD_DOWN`; `SOURCE_CHANGED`: the client was patched mid-import, so nothing was applied). File codes describe the path — `COMPRESSED` means the client\'s own Cliloc.enu was supplied instead of a converted one.',
enum: ['NO_PATH', 'NOT_FOUND', 'NO_FILE', 'UNREADABLE', 'COMPRESSED', 'TRUNCATED', 'EMPTY', 'NOT_BUFFER', 'DISABLED', 'NO_SOURCE', 'SHARD_DOWN', 'PROTOCOL', 'BUSY', 'UNAVAILABLE', 'SOURCE_CHANGED', 'INCOMPLETE', 'STUCK', 'MALFORMED', 'TOO_LARGE'],
},
path: { type: 'string', nullable: true },
file: { type: 'string', nullable: true },
count: { type: 'integer', nullable: true, description: 'Entries stored (blank strings are dropped).', example: 67496 },
parsed: { type: 'integer', nullable: true, description: 'Entries read across every source before blanks were dropped.', example: 123527 },
blank: { type: 'integer', nullable: true, example: 55994 },
blank: { type: 'integer', nullable: true, example: 0 },
pages: { type: 'integer', nullable: true, description: 'Bridge only: how many pages the table arrived in (a stock English table is about eleven).', example: 11 },
reported: { type: 'integer', nullable: true, description: 'Bridge only: how many rows the shard said it holds.', example: 67496 },
received: { type: 'integer', nullable: true, description: 'Bridge only: how many arrived. Disagreeing with `reported` means the walk is wrong.', example: 67496 },
overlayProblem: { type: 'string', nullable: true, description: 'The base imported, but the overlay directory could not be read. Reported rather than fatal.' },
sources: {
type: 'array',
nullable: true,
@@ -562,7 +600,7 @@ module.exports = {
type: 'object',
properties: {
label: { type: 'string' },
kind: { type: 'string', enum: ['base', 'custom'] },
kind: { type: 'string', enum: ['shard', 'base', 'custom'] },
entries: { type: 'integer' },
added: { type: 'integer' },
overrode: { type: 'integer' },
@@ -583,6 +621,232 @@ module.exports = {
},
},
},
UoAssetStatus: {
type: 'object',
description:
'Admin view of the client-asset import (docs/link/v8.md §6, §8). What the site holds beside what the shards UO client currently is. Holding nothing at all is a supported state — creature pages simply render without pictures, which is what every install did before this pipeline existed.',
properties: {
linked: {
type: 'boolean',
description: 'Whether a shard is configured and enabled at all. Stated rather than inferred: `shard: null` is also what a linked shard that is merely DOWN looks like, and the two want opposite things from an admin surface — one disables its import buttons, the other keeps them available so the operator can retry.',
example: true,
},
loaded: {
type: 'object',
description: 'What this site currently holds.',
properties: {
assets: { type: 'integer', description: 'Rows in the imported catalogue.', example: 787 },
stored: { type: 'integer', description: 'How many of those have a picture on disk. Lower than `assets` when the shard listed a key it could not render.', example: 787 },
creatures: { type: 'integer', description: 'Atlas creatures the shard has answered a body question about, resolved or not.', example: 812 },
resolved: { type: 'integer', description: 'How many of those resolved to a body id. The rest are types this shards scripts do not define, or spawn entries naming an item rather than a creature.', example: 780 },
catalog: { type: 'string', nullable: true, description: 'The shards catalogue id at the last import — derived from its client files, so it changes exactly when they do.', example: 'a3f9c21d4b8e0771' },
extractorVersion: { type: 'integer', nullable: true, description: 'The version of the shards extraction code. A bump makes every derived byte drift even though the client files did not move.', example: 1 },
importedAt: { type: 'string', format: 'date-time', nullable: true },
items: { type: 'integer', description: 'Item pictures held. Unlike the catalogue this has no total to compare against: item art is fetched because something on the site names it, so this is the working set rather than a fraction of one.', example: 1840 },
land: { type: 'integer', description: 'Land tile pictures held. Zero on every install until something asks for one.', example: 0 },
last: {
type: 'object',
nullable: true,
description: 'What the last import actually did. NULL on an install that has never imported, and on one whose last import predates this field — both of which mean "no import recorded", which is a different answer from an import that fetched nothing. The admin activity log records the same action, but it is one unfiltered list of every admin action on the site, so an import from three client patches ago is not findable there.',
properties: {
at: { type: 'string', format: 'date-time' },
by: { type: 'string', nullable: true, description: 'The admin who pressed it, from their session.' },
force: { type: 'boolean', description: 'True when it was a full re-import rather than an update.' },
approve: { type: 'boolean', description: 'True when it accepted assets the shard had stopped offering.' },
assets: { type: 'integer', example: 1095 },
fetched: { type: 'integer', example: 12 },
written: { type: 'integer', example: 12 },
removed: { type: 'integer', example: 0 },
absent: { type: 'integer', example: 0 },
unsupported: { type: 'integer', example: 0 },
bodies: {
type: 'object',
nullable: true,
description: 'The body pass, as a tally rather than one number: `unknown` is real drift — a spawn file naming a type this shards scripts do not define — and reads identically to a failure if both are summed into "not resolved".',
properties: {
ok: { type: 'integer', example: 780 },
unknown: { type: 'integer', example: 20 },
notCreature: { type: 'integer', example: 12 },
failed: { type: 'integer', example: 0 },
},
},
art: { type: 'integer', description: 'Creatures pointing at a picture afterwards.', example: 763 },
},
},
},
},
shard: {
type: 'object',
nullable: true,
description: 'The shards own client files right now. NULL when there is no shard link or it could not be reached — see `reason`.',
properties: {
files: { type: 'integer', description: 'How many of the animation/definition files this catalogue reads the shard actually has. Few clients carry all five anim files.', example: 9 },
extractorVersion: { type: 'integer', example: 1 },
hashing: { type: 'boolean', description: 'A hash is being computed in the background. A null `sha256` while this is true means “not yet”, never “changed”.', example: false },
complete: { type: 'boolean', description: 'Every client file has a content hash.', example: true },
imaging: {
type: 'object',
nullable: true,
description: 'Whether the shard host can render an image at all. `ok: false` is the named NO_IMAGING state: ServUO under Mono needs libgdiplus, and without it a Linux shard cannot decode a sprite. Cliloc and atlas import are unaffected.',
properties: {
ok: { type: 'boolean', example: true },
code: { type: 'string', nullable: true, example: null },
reason: { type: 'string', nullable: true },
},
},
families: {
type: 'array',
items: { type: 'string' },
description: 'Which asset key families this shards plugin overlay serves. An overlay older than phase 5 answers `["body"]` only — it has the creature catalogue and no item art.',
example: ['body', 'land', 'static'],
},
},
},
drift: {
type: 'boolean',
nullable: true,
description: 'True when the shards client files no longer match what was imported — press Import. NULL when they could not be read.',
example: false,
},
reason: { type: 'string', nullable: true, description: 'Why the shard could not be asked, when it could not.' },
code: {
type: 'string',
nullable: true,
description: 'Machine-readable cause of `reason`.',
enum: ['DISABLED', 'NO_SOURCE', 'SHARD_DOWN', 'PROTOCOL', 'BUSY', 'NO_IMAGING', 'SOURCE_CHANGED', 'UNAVAILABLE'],
},
},
},
UoAssetImportResult: {
type: 'object',
description:
'Outcome of an asset import. Reported rather than thrown, so a shard that is down or a host that cannot render images is an answer and not a 500.',
properties: {
status: {
type: 'string',
enum: ['skipped', 'unavailable', 'unchanged', 'imported', 'needsReview', 'failed'],
description: '`skipped`: no shard is configured. `unchanged`: the client files match what was imported and nothing was fetched. `needsReview`: assets this site holds are no longer offered by the shard, and nothing was changed — re-run with `approve` to accept it.',
example: 'imported',
},
reason: { type: 'string', nullable: true },
code: {
type: 'string',
nullable: true,
description: 'Machine-readable cause. `NO_IMAGING` is a shard host with no libgdiplus; `SOURCE_CHANGED` is a client patched partway through the walk, in which case nothing was applied.',
enum: ['DISABLED', 'NO_SOURCE', 'SHARD_DOWN', 'PROTOCOL', 'BUSY', 'NO_IMAGING', 'SOURCE_CHANGED', 'INCOMPLETE', 'STUCK', 'MALFORMED', 'TOO_LARGE', 'UNAVAILABLE'],
},
catalog: { type: 'string', nullable: true, example: 'a3f9c21d4b8e0771' },
extractorVersion: { type: 'integer', nullable: true, example: 1 },
assets: { type: 'integer', nullable: true, description: 'Catalogue rows after the import.', example: 787 },
fetched: { type: 'integer', nullable: true, description: 'How many sprites actually crossed the wire. On an Update after a client patch this is far smaller than `assets`, which is the point of the manifest.', example: 12 },
written: { type: 'integer', nullable: true, description: 'How many were written to disk.', example: 12 },
absent: {
type: 'integer',
nullable: true,
description: 'Keys the shard listed but could not render. NOT a failure: this client has no art at that key, which is the expected answer for two thirds of the playable ghost and gargoyle bodies.',
example: 0,
},
unsupported: { type: 'integer', nullable: true, description: 'Keys the shard does not serve at all. Unlike `absent` this indicates a bug on the sites side, not a gap in the client.', example: 0 },
removed: { type: 'integer', nullable: true, description: 'Assets deleted because the shard no longer offers them (only with `approve`).', example: 0 },
scanned: { type: 'integer', nullable: true, description: 'Body ids the shard walked. Far larger than `assets` — most of the addressable range has no art.', example: 2047 },
pages: { type: 'integer', nullable: true, description: 'Manifest pages. This family pages on the shards scan budget rather than on bytes, so several is normal.', example: 4 },
playerBodies: {
type: 'array',
nullable: true,
items: { type: 'integer' },
description: 'The body ids the shard reports as player-character bodies — every registered races male, female and ghost bodies, asked of the shard rather than hardcoded. These render head-on; everything else renders three-quarter.',
example: [400, 401, 402, 403, 605, 606, 607, 608, 666, 667, 694, 695],
},
vanished: {
type: 'array',
nullable: true,
description: 'On `needsReview`: up to fifty of the keys that disappeared, each with the picture this site currently serves for it. The filename is there because the decision being asked for is "is it right that these disappear?", and an asset key names nothing a human recognises — `body/820/a23` is a horse.',
items: {
type: 'object',
properties: {
key: { type: 'string', example: 'body/820/a23' },
file: { type: 'string', nullable: true, description: 'Filename under uploads/atlas/, or null if this site never stored a picture for it.', example: 'uo-body-820-a23-9f3c1a77.png' },
},
},
},
vanishedCount: { type: 'integer', nullable: true },
bodies: {
type: 'object',
nullable: true,
description: 'The slug → body id pass (§8). The shard constructs each creature and reads its body id, which is the only thing correct for a shards own custom creatures.',
properties: {
asked: { type: 'integer', example: 812 },
answered: { type: 'integer', example: 812 },
resolved: { type: 'integer', example: 780 },
tally: {
type: 'object',
description: 'Per-outcome counts. `unknown` is real drift worth acting on — a spawn file naming a type this shards scripts do not define. `notCreature` is a spawn entry for an item or decoration and is permanent.',
properties: {
ok: { type: 'integer', example: 780 },
unknown: { type: 'integer', example: 20 },
notCreature: { type: 'integer', example: 12 },
failed: { type: 'integer', example: 0 },
},
},
reason: { type: 'string', nullable: true },
},
},
art: {
type: 'object',
nullable: true,
description: 'The derivation onto `shard_spawn_creatures.art`. An operator-supplied `spawnAtlas.art.json` always wins over an imported sprite.',
properties: {
applied: { type: 'integer', description: 'Creatures now pointing at a picture.', example: 763 },
derived: { type: 'integer', description: 'From the import.', example: 763 },
operator: { type: 'integer', description: 'From the operators own map.', example: 0 },
error: { type: 'string', nullable: true },
},
},
},
},
UoItemArtWarmResult: {
type: 'object',
description:
'Outcome of one item-art warm pass (docs/link/v8.md §11, phase 5). Unlike the body catalogue there is no manifest and no set: the client addresses 49,152 item graphics times three thousand hues, so what gets fetched is defined by what this sites own rows name — every distinct (ItemID, hue) on a player vendor, plus anything a character sheet has shown since the last pass. Reported rather than thrown, so a shard that is down is an answer and not a 500.',
properties: {
status: {
type: 'string',
enum: ['skipped', 'unavailable', 'unchanged', 'imported', 'failed'],
description:
'`skipped`: no shard is configured. `unchanged`: every wanted picture is already held and current. `unavailable`: the shard could not be asked, or its plugin overlay is too old to serve item art.',
example: 'imported',
},
reason: { type: 'string', nullable: true },
code: {
type: 'string',
nullable: true,
description:
'Machine-readable cause. `NO_IMAGING` is a shard host with no libgdiplus. `UNSUPPORTED` is a plugin overlay that serves the creature catalogue but not item art — update the overlay.',
enum: ['DISABLED', 'NO_SOURCE', 'SHARD_DOWN', 'PROTOCOL', 'BUSY', 'NO_IMAGING', 'UNSUPPORTED', 'INCOMPLETE', 'STUCK', 'MALFORMED', 'TOO_LARGE', 'UNAVAILABLE'],
},
catalog: {
type: 'string',
nullable: true,
description:
'The shards art catalogue id these pictures were fetched under — a hash of the files that decide their bytes. Stored per row, which is how staleness is answered without a manifest.',
example: '7c1e04b9aa2f3d58',
},
wanted: { type: 'integer', nullable: true, description: 'Distinct keys this sites rows name right now.', example: 1840 },
held: { type: 'integer', nullable: true, description: 'How many of those are already stored and current.', example: 1440 },
asked: { type: 'integer', nullable: true, description: 'How many this pass actually requested. Bounded by `limit`.', example: 400 },
fetched: { type: 'integer', nullable: true, description: 'How many the shard returned a picture for.', example: 396 },
written: { type: 'integer', nullable: true, description: 'How many were written to disk.', example: 396 },
absent: {
type: 'integer',
nullable: true,
description:
'Keys the shard has no art for. NOT a failure — 9,963 of this clients static ids have an empty index entry, and an item using one simply has no picture.',
example: 4,
},
unsupported: { type: 'integer', nullable: true, description: 'Keys the shard does not serve at all. A bug on the sites side rather than a gap in the client.', example: 0 },
remaining: { type: 'integer', nullable: true, description: 'Wanted keys left for the next pass. Passes repeat on a timer, so a backlog drains without an operator.', example: 0 },
},
},
UoShardLinkRequest: {
type: 'object',
required: ['code'],

View File

@@ -50,7 +50,7 @@ function fakeCtx(overrides = {}) {
// MODULE_API 1.7.0. Both are fire-and-forget and return undefined by
// contract — a module gets no delivery answer back, deliberately — so the
// spies return undefined rather than a promise, which is what core does.
events: { emit: spy(undefined) },
events: { emit: spy(undefined), reconcile: spy(undefined) },
inbox: { push: spy(undefined) },
secretBox: { encrypt: spy('enc'), decrypt: spy('dec') },
middleware: {
@@ -104,6 +104,9 @@ function fakeApi() {
slashCommands: [],
triggers: null,
audiences: null,
eventActions: null,
eventBudgets: null,
eventOptionSources: null,
hooks: {},
}
const called = new Set()
@@ -134,6 +137,16 @@ function fakeApi() {
// and merging two calls would make "which group is this rule in" — the
// question the one-shot seed guard answers — unanswerable.
registerEngagementSeeds(seeds) { once('registerEngagementSeeds'); record.engagementSeeds = seeds },
// MODULE_API 1.10.0 (EVENTS.md F, EVENTS_PLAN.md Phases 7 and 9). `once` on
// all three, matching core: it stages a registrant's whole batch and applies
// it as one, so a second call is a module changing its mind mid-register().
registerEventActions(actions) { once('registerEventActions'); record.eventActions = actions },
registerEventBudgets(budgets) { once('registerEventBudgets'); record.eventBudgets = budgets },
registerEventOptionSources(sources) { once('registerEventOptionSources'); record.eventOptionSources = sources },
// And the fourth, from Phase 11b. `once` for the same reason, and present here
// for a second one: a verb this module calls and this fake does not have is a
// TypeError in `entry.test.js` rather than a surprise at somebody's boot.
registerEventLeases(leases) { once('registerEventLeases'); record.eventLeases = leases },
onBoot(fn) { once('onBoot'); record.hooks.onBoot = fn },
onShutdown(fn) { once('onShutdown'); record.hooks.onShutdown = fn },
}

View File

@@ -0,0 +1,407 @@
const { test } = require('node:test')
const assert = require('node:assert/strict')
const uoLinkClient = require('../utils/uoLinkClient')
const bridge = require('../utils/assetBridge')
// The three walks over the asset plane, driven against a stubbed sidecar client
// (docs/link/v8.md §5, §6, §8 — protocol 8, phase 3).
//
// Two families of failure are asserted here and they are not the same shape.
//
// **The envelope failures** are ways the shard can be wrong that leave this side
// holding a catalogue it believes is complete. They are invisible downstream: a
// catalogue missing its last three hundred bodies renders as a site where some
// creatures have pictures and some do not, which is exactly what NO catalogue
// looks like. Each corresponds to a field §3.4 puts on the wire specifically so
// this side can tell the difference.
//
// **The absence failures** are the opposite mistake, and phase 3's more likely
// one: treating a body this client has no art for as an error. Two thirds of the
// playable ghost and gargoyle bodies are in that state on a stock client, and an
// import that failed — or even warned loudly — on them would teach an operator to
// ignore the panel.
const saved = {}
function stub({ sources, manifest = [], fetch = [], bodies = [] } = {}) {
saved.getAssetSources = uoLinkClient.getAssetSources
saved.getAssetManifest = uoLinkClient.getAssetManifest
saved.fetchAssets = uoLinkClient.fetchAssets
saved.resolveBodies = uoLinkClient.resolveBodies
const calls = { manifest: [], fetch: [], bodies: [] }
uoLinkClient.getAssetSources = async () => sources
uoLinkClient.getAssetManifest = async ({ family, cursor } = {}) => {
calls.manifest.push({ family: family ?? null, cursor: cursor ?? null })
const next = manifest.shift()
if (!next) throw new Error('the walk asked for more manifest pages than the test supplied')
return next
}
uoLinkClient.fetchAssets = async ({ keys, catalog, cursor } = {}) => {
calls.fetch.push({ keys, catalog: catalog ?? null, cursor: cursor ?? null })
const next = fetch.shift()
if (!next) throw new Error('the walk asked for more fetch pages than the test supplied')
return next
}
uoLinkClient.resolveBodies = async (types) => {
calls.bodies.push(types)
const next = bodies.shift()
if (!next) throw new Error('the walk asked for more body chunks than the test supplied')
return next
}
return calls
}
function restore() {
for (const [name, fn] of Object.entries(saved)) {
if (fn) uoLinkClient[name] = fn
}
}
const ok = (data) => ({ ok: true, status: 200, data })
const fail = (status, data) => ({ ok: false, status, data })
const CATALOG = 'a3f9c21d4b8e0771'
const manifestPage = (rows, extra = {}) =>
ok({
kind: 'assets.manifest.ok',
family: 'body',
catalog: CATALOG,
extractorVersion: 1,
playerBodies: [400, 401, 402, 403],
scanned: rows.length,
rows,
more: false,
cut: 'end',
...extra,
})
const fetchPage = (rows, extra = {}) =>
ok({
kind: 'assets.fetch.ok',
family: 'body',
catalog: CATALOG,
rows,
more: false,
cut: 'end',
...extra,
})
const row = (body, sha = 'aa') => ({
key: `body/${body}/a0`,
sha256: sha,
bytes: 900,
width: 24,
height: 63,
body,
direction: 1,
})
const png = Buffer.from([0x89, 0x50, 0x4e, 0x47]).toString('base64')
const sourcesReply = (extra = {}) =>
ok({
kind: 'assets.sources.ok',
extractorVersion: 1,
imaging: { ok: true },
hashing: false,
complete: true,
files: [
{ name: 'anim.idx', size: 10, mtime: 1, sha256: 'a' },
{ name: 'anim.mul', size: 20, mtime: 2, sha256: 'b' },
{ name: 'body.def', size: 30, mtime: 3, sha256: 'c' },
// Not a source this family reads: `art.mul` decides item pictures, not
// creature ones, and folding it in would make every item-art change look
// like a reason to re-import the whole body catalogue.
{ name: 'art.mul', size: 148000000, mtime: 4, sha256: 'd' },
],
...extra,
})
// ── the source gate (§6 stage 1) ──────────────────────────────────────────
test('the source fingerprint keeps only the files the body catalogue reads', async (t) => {
stub({ sources: sourcesReply() })
t.after(restore)
const fingerprint = await bridge.sourceFingerprint()
assert.deepEqual(Object.keys(fingerprint.files).sort(), ['anim.idx', 'anim.mul', 'body.def'])
assert.equal(fingerprint.extractorVersion, 1)
})
test('a bumped extractor version is drift even when every client file is identical', () => {
const files = { 'anim.mul': { size: 1, mtime: 2, sha256: 'x' } }
assert.equal(
bridge.sameSources({ files, extractorVersion: 1 }, { files, extractorVersion: 1 }),
true,
)
// §7: a corrected frame offset changes every derived byte while every source
// file stays byte-identical. If this returned true the fix would never reach
// an install whose client never moves.
assert.equal(
bridge.sameSources({ files, extractorVersion: 2 }, { files, extractorVersion: 1 }),
false,
)
})
test('a client that GAINED an anim file is drift, not a match', () => {
const before = { files: { 'anim.mul': { size: 1, mtime: 2, sha256: 'x' } }, extractorVersion: 1 }
const after = {
files: {
'anim.mul': { size: 1, mtime: 2, sha256: 'x' },
// A client that grows an anim5.mul is a client whose gargoyles suddenly
// resolve. Comparing only the files present in both would call that
// unchanged and never import them.
'anim5.mul': { size: 9, mtime: 9, sha256: 'y' },
},
extractorVersion: 1,
}
assert.equal(bridge.sameSources(before, after), false)
})
test('a null hash falls back to size and mtime rather than reading as changed', () => {
// The shard hashes 195 MB anim files off the request path, so a null sha256 is
// "not computed yet". Treating it as a difference would re-import the whole
// catalogue on every restart until the background pass finished.
const a = { files: { 'anim.mul': { size: 5, mtime: 7, sha256: null } }, extractorVersion: 1 }
const b = { files: { 'anim.mul': { size: 5, mtime: 7, sha256: 'later' } }, extractorVersion: 1 }
assert.equal(bridge.sameSources(a, b), true)
})
// ── the manifest walk (§6 stage 2) ────────────────────────────────────────
test('the manifest walks every page and stops only on cut: end', async (t) => {
const calls = stub({
manifest: [
manifestPage([row(12), row(34)], { more: true, cursor: 'b:34', cut: 'limit' }),
manifestPage([row(400)]),
],
})
t.after(restore)
const result = await bridge.readManifest()
assert.equal(result.rows.length, 3)
assert.equal(result.catalog, CATALOG)
assert.deepEqual(result.playerBodies, [400, 401, 402, 403])
assert.deepEqual(
calls.manifest.map((c) => c.cursor),
[null, 'b:34'],
)
})
test('a short page that did not end the catalogue is refused', async (t) => {
// `cut: 'limit'` with `more: false` is the shard saying it stopped for its own
// reason. Importing what arrived would silently drop every body after it, and
// the result is indistinguishable from a client with fewer creatures.
stub({ manifest: [manifestPage([row(12)], { more: false, cut: 'limit' })] })
t.after(restore)
await assert.rejects(() => bridge.readManifest(), /stopped sending asset rows/)
})
test('a cursor that does not advance is refused rather than looped on', async (t) => {
stub({
manifest: [
manifestPage([row(12)], { more: true, cursor: 'b:12', cut: 'budget' }),
manifestPage([row(13)], { more: true, cursor: 'b:12', cut: 'budget' }),
],
})
t.after(restore)
await assert.rejects(() => bridge.readManifest(), /without advancing its cursor/)
})
test('the client files changing mid-walk aborts the whole import', async (t) => {
// The catalogue id is derived from the client files themselves, so a change
// between two pages means half of what we hold describes files that no longer
// exist — and nothing later can tell which half.
stub({
manifest: [
manifestPage([row(12)], { more: true, cursor: 'b:12', cut: 'limit' }),
manifestPage([row(34)], { catalog: 'something-else' }),
],
})
t.after(restore)
await assert.rejects(() => bridge.readManifest(), /changed while the manifest was being read/)
})
// ── the fetch (§5) ────────────────────────────────────────────────────────
test('a fetch passes the catalogue id and decodes the PNG', async (t) => {
const calls = stub({
fetch: [fetchPage([{ key: 'body/12/a0', status: 'ok', sha256: 'aa', bytes: 4, width: 24, height: 63, body: 12, direction: 1, png }])],
})
t.after(restore)
const { assets } = await bridge.fetchAssets({ keys: ['body/12/a0'], catalog: CATALOG })
assert.equal(calls.fetch[0].catalog, CATALOG)
assert.equal(assets.get('body/12/a0').png.length, 4)
assert.equal(assets.get('body/12/a0').width, 24)
})
test('a body catalogued at a later action keeps that action in its row', async (t) => {
// §11.2, phase 6. 73 of a stock client's bodies have no art at action 0 and are
// catalogued at the first action that does — body 820's is 23, and it is a
// horse. The action travels with the row because the atlas join needs it in
// SQL; re-deriving it from the key would put a second parser of §5's scheme in
// the schema.
stub({
manifest: [
manifestPage([
{ ...row(12), action: 0 },
{ key: 'body/820/a23', sha256: 'bb', bytes: 900, width: 68, height: 69, body: 820, action: 23, direction: 1 },
]),
],
})
t.after(restore)
const { rows } = await bridge.readManifest({})
assert.deepEqual(
rows.map((r) => [r.key, r.action]),
[
['body/12/a0', 0],
['body/820/a23', 23],
],
)
})
test('an overlay older than phase 6 reads as action 0 rather than as unknown', async (t) => {
// A phase-3 through phase-5 overlay omits `action` entirely, and every key it
// ever produced ended in `a0`. Reading that as null would make the atlas join
// COALESCE it back to 0 anyway; reading it as 0 here says so once.
stub({ manifest: [manifestPage([row(12)])] })
t.after(restore)
const { rows } = await bridge.readManifest({})
assert.equal(rows[0].action, 0)
})
test('an absent asset is a counted row, not a failed fetch', async (t) => {
// The whole reason this is not an error: two thirds of the playable ghost and
// gargoyle bodies have no art on a stock client (§5.2), and an import that
// failed on them could never succeed.
stub({
fetch: [
fetchPage([
{ key: 'body/12/a0', status: 'ok', sha256: 'aa', bytes: 4, png },
{ key: 'body/666/a0', status: 'absent' },
{ key: 'body/400/a2/f3', status: 'unsupported' },
]),
],
})
t.after(restore)
const { assets, missing } = await bridge.fetchAssets({
keys: ['body/12/a0', 'body/666/a0', 'body/400/a2/f3'],
catalog: CATALOG,
})
assert.equal(assets.size, 1)
// Counted apart, because they mean different things: `absent` is a gap in the
// operator's client and `unsupported` is a bug on this side.
assert.equal(missing.absent, 1)
assert.equal(missing.unsupported, 1)
})
test('a busy shard is retried rather than failing the walk', async (t) => {
saved.fetchAssets = uoLinkClient.fetchAssets
t.after(restore)
let attempts = 0
uoLinkClient.fetchAssets = async () => {
attempts++
if (attempts < 3) return fail(425, { reason: 'busy' })
return fetchPage([{ key: 'body/12/a0', status: 'ok', sha256: 'aa', bytes: 4, png }])
}
const { assets } = await bridge.fetchAssets({ keys: ['body/12/a0'], catalog: CATALOG })
assert.equal(attempts, 3)
assert.equal(assets.size, 1)
})
test('a shard host with no libgdiplus is named, not reported as a dead shard', async (t) => {
saved.getAssetManifest = uoLinkClient.getAssetManifest
t.after(restore)
uoLinkClient.getAssetManifest = async () =>
fail(503, { reason: "this shard host cannot render images - Mono's System.Drawing needs libgdiplus" })
await assert.rejects(
() => bridge.readManifest(),
(err) => err.code === 'NO_IMAGING',
)
})
// ── the body pass (§8) ────────────────────────────────────────────────────
test('body resolution chunks to the shard cap and records every outcome', async (t) => {
const creatures = []
for (let i = 0; i < bridge.BODY_CHUNK + 5; i++) {
creatures.push({ slug: `c-${i}`, name: `Creature${i}` })
}
const reply = (types) =>
ok({
kind: 'assets.bodies.ok',
rows: types.map((type, i) => (i === 0 ? { type, status: 'unknown' } : { type, status: 'ok', body: 100 + i })),
more: false,
cut: 'end',
})
const calls = stub({ bodies: [] })
t.after(restore)
uoLinkClient.resolveBodies = async (types) => {
calls.bodies.push(types)
return reply(types)
}
const rows = await bridge.resolveBodies({ creatures })
// Two chunks, and neither over the cap: the shard REFUSES an over-long list
// rather than truncating it, so a chunk size above its cap does not degrade —
// every request fails.
assert.equal(calls.bodies.length, 2)
assert.ok(calls.bodies.every((chunk) => chunk.length <= bridge.BODY_CHUNK))
assert.equal(rows.length, creatures.length)
// The negative answers are kept. Without them the next pass asks again, and
// the pass costs a real constructor per name on the shard's Core thread.
assert.equal(rows.filter((r) => r.status === 'unknown').length, 2)
})
test('two slugs sharing a class name are asked once and both get the answer', async (t) => {
const calls = stub({
bodies: [
ok({ kind: 'assets.bodies.ok', rows: [{ type: 'GiantSpider', status: 'ok', body: 28 }], more: false, cut: 'end' }),
],
})
t.after(restore)
const rows = await bridge.resolveBodies({
creatures: [
{ slug: 'giant-spider', name: 'GiantSpider' },
{ slug: 'giantspider', name: 'GiantSpider' },
],
})
assert.deepEqual(calls.bodies[0], ['GiantSpider'])
assert.equal(rows.length, 2)
assert.ok(rows.every((r) => r.body === 28))
})

View File

@@ -0,0 +1,156 @@
// Which atlas source runs, and what boot does with the answer
// (docs/link/v8.md §10, §17.7; docs/website/SPAWN_ATLAS.md).
//
// The model is the only place that decides between a local ServUO tree and the
// shard bridge, so these drive it with the shard, the database and the
// filesystem all stubbed. Nothing here reaches a real sidecar or a real tree.
//
// The rule under test is the one the cliloc pipeline settled first and this
// inherits: the bridge wins whenever uo-link is configured and enabled, a local
// path is what a site with no shard link uses, and an explicit path is an
// instruction that overrules both.
const { test } = require('node:test')
const assert = require('node:assert/strict')
const atlas = require('../model/shardAtlas/shardAtlas.model')
const db = require('../model/shardAtlas/shardAtlas.db')
const source = require('../utils/spawnAtlasSource')
const uoLinkConfig = require('../model/uoLinkConfig/uoLinkConfig.model')
const { ctx } = require('./_setup')
const saved = {
getMeta: db.getMeta,
getPending: db.getPending,
getFacets: db.getFacets,
hashFrom: source.hashFrom,
buildFrom: source.buildFrom,
getSafe: uoLinkConfig.getSafe,
settingsGet: ctx.settings.get,
}
function restore() {
db.getMeta = saved.getMeta
db.getPending = saved.getPending
db.getFacets = saved.getFacets
source.hashFrom = saved.hashFrom
source.buildFrom = saved.buildFrom
uoLinkConfig.getSafe = saved.getSafe
ctx.settings.get = saved.settingsGet
}
/** Whatever source the model chose, captured rather than read. */
function rig({ linked = true, treePath = '', meta = null } = {}) {
const asked = { hash: [], build: [] }
uoLinkConfig.getSafe = async () => ({
enabled: linked,
baseUrl: linked ? 'http://127.0.0.1:8099' : null,
})
ctx.settings.get = async () => treePath
db.getMeta = async () => meta
db.getPending = async () => null
db.getFacets = async () => []
source.hashFrom = async (descriptor) => {
asked.hash.push(descriptor)
return { 'Data/Regions.xml': 'aa' }
}
source.buildFrom = async (descriptor) => {
asked.build.push(descriptor)
throw new Error('the test stops before a build')
}
return asked
}
test('a linked shard is the atlas source, and the configured path is not consulted', async () => {
const asked = rig({ linked: true, treePath: '/srv/servuo' })
const status = await atlas.status()
assert.equal(status.source, 'bridge')
assert.equal(status.path, 'the shard bridge')
assert.equal(status.configured, true)
assert.deepEqual(asked.hash[0], { kind: 'bridge', root: '' })
restore()
})
test('with no shard linked the configured tree is the source', async () => {
const asked = rig({ linked: false, treePath: '/srv/servuo' })
const status = await atlas.status()
assert.equal(status.source, 'fs')
assert.equal(status.path, '/srv/servuo')
assert.deepEqual(asked.hash[0], { kind: 'fs', root: '/srv/servuo' })
restore()
})
test('an explicit path overrules the bridge — it is an instruction, not a default', async () => {
const asked = rig({ linked: true, treePath: '/srv/servuo' })
await atlas.status({ path: '/tmp/other-tree' })
assert.deepEqual(asked.hash[0], { kind: 'fs', root: '/tmp/other-tree' })
restore()
})
test('no shard and no path is "nothing configured", not an error', async () => {
rig({ linked: false, treePath: '' })
const status = await atlas.status()
assert.equal(status.configured, false)
const result = await atlas.refresh()
assert.equal(result.status, 'skipped')
restore()
})
test('boot does not call the shard; it says where the import lives instead', async () => {
// §17.7's rule, and the reason it is not free: an install whose atlas comes
// over the bridge has NO automatic refresh at all, so the skip has to be
// deliberate and visible rather than a path that quietly does nothing.
const asked = rig({ linked: true, treePath: '/srv/servuo' })
const result = await atlas.refreshOnBoot()
assert.equal(result.status, 'skipped')
assert.equal(result.source, 'bridge')
assert.equal(asked.hash.length, 0, 'boot made no shard call at all')
assert.equal(asked.build.length, 0)
restore()
})
test('boot still refreshes by itself from a local tree', async () => {
const asked = rig({ linked: false, treePath: '/srv/servuo' })
await atlas.refreshOnBoot()
assert.deepEqual(asked.hash[0], { kind: 'fs', root: '/srv/servuo' })
restore()
})
test('a source that cannot be read is reported, with which end could not read it', async () => {
rig({ linked: true, treePath: '' })
source.hashFrom = async () => {
const { TreeBridgeError } = require('../utils/treeBridge')
throw new TreeBridgeError('the shard is not serving its tree', 'DISABLED')
}
const result = await atlas.refresh()
assert.equal(result.status, 'unavailable')
assert.equal(result.source, 'bridge')
assert.equal(result.code, 'DISABLED')
restore()
})

View File

@@ -0,0 +1,303 @@
const { test } = require('node:test')
const assert = require('node:assert/strict')
const uoLinkClient = require('../utils/uoLinkClient')
const bridge = require('../utils/clilocBridge')
// The walk over `GET /cliloc`, driven against a stubbed sidecar client.
//
// Everything asserted here is a way the shard can be wrong that leaves the
// website holding a table it believes is complete. That is the failure worth
// testing, because it is invisible downstream: a truncated cliloc table renders
// some items with names and some with ids, which is exactly what NO table looks
// like. None of these are hypothetical shapes — each corresponds to a field the
// paging envelope carries specifically so this side can tell the difference
// (docs/link/v8.md §3.4).
const saved = {}
function stub({ sources, pages }) {
saved.getAssetSources = uoLinkClient.getAssetSources
saved.getClilocTable = uoLinkClient.getClilocTable
const calls = []
uoLinkClient.getAssetSources = async () => sources
uoLinkClient.getClilocTable = async ({ lang, cursor } = {}) => {
calls.push({ lang, cursor: cursor ?? null })
const next = pages.shift()
if (!next) throw new Error('the walk asked for more pages than the test supplied')
return next
}
return calls
}
function restore() {
if (saved.getAssetSources) uoLinkClient.getAssetSources = saved.getAssetSources
if (saved.getClilocTable) uoLinkClient.getClilocTable = saved.getClilocTable
}
const ok = (data) => ({ ok: true, status: 200, data })
/** One page of rows, with the source fingerprint every page echoes. */
const page = (rows, extra = {}) =>
ok({
kind: 'cliloc.table.ok',
lang: 'enu',
file: 'cliloc.enu',
size: 4989921,
mtime: 1757462400000,
total: 3,
rows,
more: false,
cut: 'end',
...extra,
})
const sourcesReply = (file = {}) =>
ok({
kind: 'assets.sources.ok',
extractorVersion: 1,
imaging: { ok: true },
hashing: false,
complete: true,
files: [
{ name: 'cliloc.enu', path: '/uo/cliloc.enu', size: 4989921, mtime: 1757462400000, sha256: 'abc', ...file },
{ name: 'art.mul', path: '/uo/art.mul', size: 148000000, mtime: 1, sha256: null },
],
})
// ── Stage 1: the fingerprint ───────────────────────────────────────────────
test('fingerprint picks the cliloc file out of the client manifest', async (t) => {
stub({ sources: sourcesReply(), pages: [] })
t.after(restore)
const fp = await bridge.fingerprint()
assert.equal(fp.file, 'cliloc.enu')
assert.equal(fp.size, 4989921)
assert.equal(fp.sha256, 'abc')
assert.equal(fp.extractorVersion, 1)
})
test('a client with no cliloc file is NO_SOURCE, not a crash', async (t) => {
stub({
sources: ok({ extractorVersion: 1, files: [{ name: 'art.mul', size: 1, mtime: 1 }] }),
pages: [],
})
t.after(restore)
await assert.rejects(bridge.fingerprint(), (err) => {
assert.equal(err.code, 'NO_SOURCE')
return true
})
})
test('the asset plane being switched off reads as a refusal, not a bug', async (t) => {
stub({
sources: { ok: false, status: 403, data: { reason: 'asset extraction is disabled on this shard' } },
pages: [],
})
t.after(restore)
await assert.rejects(bridge.fingerprint(), (err) => {
assert.equal(err.code, 'DISABLED')
return true
})
})
// A hash that has not been computed yet is the shard's ordinary first answer:
// hashing the 343 MB of art and animation it also serves cannot fit in a 10 s
// reply, so it happens off the request path. Treating a null hash as a CHANGE
// would make the panel show drift forever on a shard nobody has imported from.
test('a missing hash falls back to (size, mtime) rather than reading as drift', () => {
const before = { size: 10, mtime: 20, sha256: null, extractorVersion: 1 }
const after = { size: 10, mtime: 20, sha256: null, extractorVersion: 1 }
assert.equal(bridge.sameSource(before, after), true)
assert.equal(bridge.sameSource(before, { ...after, mtime: 21 }), false)
})
test('a hash on both sides beats size and mtime, which a patched-in-place file can preserve', () => {
const a = { size: 10, mtime: 20, sha256: 'aaa', extractorVersion: 1 }
assert.equal(bridge.sameSource(a, { ...a, sha256: 'bbb' }), false)
assert.equal(bridge.sameSource(a, { ...a, size: 11, mtime: 99 }), true)
})
test('the extractor version is part of the fingerprint, so a corrected reader drifts', () => {
const a = { size: 10, mtime: 20, sha256: 'aaa', extractorVersion: 1 }
assert.equal(bridge.sameSource(a, { ...a, extractorVersion: 2 }), false)
})
// ── Stage 2: the walk ──────────────────────────────────────────────────────
test('a one-page table comes back whole', async (t) => {
const calls = stub({
sources: sourcesReply(),
pages: [page([{ n: 3, f: 0, t: 'c' }, { n: 1, f: 2, t: 'a' }])],
})
t.after(restore)
const { entries, source } = await bridge.readCliloc()
assert.deepEqual(entries, [
{ number: 3, flag: 0, text: 'c' },
{ number: 1, flag: 2, text: 'a' },
])
assert.equal(source.pages, 1)
assert.equal(source.received, 2)
assert.equal(source.reported, 3)
assert.deepEqual(calls, [{ lang: 'enu', cursor: null }])
})
test('pages are walked by echoing the cursor back until more is false', async (t) => {
const calls = stub({
sources: sourcesReply(),
pages: [
page([{ n: 1, f: 0, t: 'a' }], { more: true, cursor: 'n:1', cut: 'budget' }),
page([{ n: 2, f: 0, t: 'b' }], { more: true, cursor: 'n:2', cut: 'budget' }),
page([{ n: 3, f: 0, t: 'c' }]),
],
})
t.after(restore)
const { entries, source } = await bridge.readCliloc()
assert.equal(entries.length, 3)
assert.equal(source.pages, 3)
assert.deepEqual(
calls.map((c) => c.cursor),
[null, 'n:1', 'n:2'],
)
})
// `cut` is the field that is easy to omit and expensive not to have. A short
// page means the source ended, the byte budget was spent, or the family hit its
// own limit — and only the first means finished.
test('a last page that did not end the table is refused, not imported', async (t) => {
stub({
sources: sourcesReply(),
pages: [page([{ n: 1, f: 0, t: 'a' }], { more: false, cut: 'limit' })],
})
t.after(restore)
await assert.rejects(bridge.readCliloc(), (err) => {
assert.equal(err.code, 'INCOMPLETE')
return true
})
})
test('a shard that does not advance its cursor is stopped rather than spun on', async (t) => {
stub({
sources: sourcesReply(),
pages: [
page([{ n: 1, f: 0, t: 'a' }], { more: true, cursor: 'n:1', cut: 'budget' }),
page([{ n: 2, f: 0, t: 'b' }], { more: true, cursor: 'n:1', cut: 'budget' }),
],
})
t.after(restore)
await assert.rejects(bridge.readCliloc(), (err) => {
assert.equal(err.code, 'STUCK')
return true
})
})
test('more:true with no cursor at all is the same refusal', async (t) => {
stub({
sources: sourcesReply(),
pages: [page([{ n: 1, f: 0, t: 'a' }], { more: true, cut: 'budget' })],
})
t.after(restore)
await assert.rejects(bridge.readCliloc(), (err) => {
assert.equal(err.code, 'STUCK')
return true
})
})
// The one failure a count cannot catch: an operator patches their client while
// the import is walking it. Half of what arrived is from a file that no longer
// exists, and nothing later can tell which half.
test('a client patched mid-walk aborts the whole import', async (t) => {
stub({
sources: sourcesReply(),
pages: [
page([{ n: 1, f: 0, t: 'a' }], { more: true, cursor: 'n:1', cut: 'budget' }),
page([{ n: 2, f: 0, t: 'b' }], { size: 5000000, mtime: 1757470000000 }),
],
})
t.after(restore)
await assert.rejects(bridge.readCliloc(), (err) => {
assert.equal(err.code, 'SOURCE_CHANGED')
return true
})
})
// 425 is flow control and the ORDINARY answer during an import — the shard's
// asset plane serves one request at a time on purpose — so it is retried rather
// than failed. (The backoff is real time, so this exercises one retry only.)
test('a busy shard is retried, because the work is happening', async (t) => {
saved.getAssetSources = uoLinkClient.getAssetSources
saved.getClilocTable = uoLinkClient.getClilocTable
t.after(restore)
let attempts = 0
uoLinkClient.getAssetSources = async () => sourcesReply()
uoLinkClient.getClilocTable = async () => {
attempts++
if (attempts === 1) return { ok: false, status: 425, data: { kind: 'bridge.busy' } }
return page([{ n: 1, f: 0, t: 'a' }])
}
const { entries } = await bridge.readCliloc()
assert.equal(attempts, 2)
assert.equal(entries.length, 1)
})
test('a page with no rows array is malformed, not an empty table', async (t) => {
stub({ sources: sourcesReply(), pages: [ok({ kind: 'cliloc.table.ok', more: false, cut: 'end' })] })
t.after(restore)
await assert.rejects(bridge.readCliloc(), (err) => {
assert.equal(err.code, 'MALFORMED')
return true
})
})
test('a shard that never ends the table is bounded by the page cap', async (t) => {
saved.getAssetSources = uoLinkClient.getAssetSources
saved.getClilocTable = uoLinkClient.getClilocTable
t.after(restore)
let n = 0
uoLinkClient.getAssetSources = async () => sourcesReply()
uoLinkClient.getClilocTable = async () => {
n++
return page([{ n, f: 0, t: 'x' }], { more: true, cursor: `n:${n}`, cut: 'budget' })
}
await assert.rejects(bridge.readCliloc(), (err) => {
assert.equal(err.code, 'TOO_LARGE')
return true
})
assert.equal(n, bridge.MAX_PAGES)
})
test('rows with an unusable id are dropped rather than stored as NaN', async (t) => {
stub({
sources: sourcesReply(),
pages: [page([{ n: 'nonsense', f: 0, t: 'a' }, { n: 7, f: 0, t: 'b' }])],
})
t.after(restore)
const { entries } = await bridge.readCliloc()
assert.deepEqual(entries, [{ number: 7, flag: 0, text: 'b' }])
})

View File

@@ -0,0 +1,330 @@
// Which cliloc source runs, and what the shard path does with the answer
// (docs/link/v8.md §9, docs/website/CLILOCS.md).
//
// The model is the only place that decides between the two pipelines, so these
// drive it with the shard, the database and the filesystem all stubbed. Nothing
// here reaches the real sidecar or a real table.
const { test } = require('node:test')
const assert = require('node:assert/strict')
const fs = require('node:fs')
const os = require('node:os')
const path = require('node:path')
const clilocs = require('../model/shardClilocs/shardClilocs.model')
const db = require('../model/shardClilocs/shardClilocs.db')
const bridge = require('../utils/clilocBridge')
const uoLinkConfig = require('../model/uoLinkConfig/uoLinkConfig.model')
const { ctx } = require('./_setup')
const saved = {
getMeta: db.getMeta,
replaceAll: db.replaceAll,
count: db.count,
fingerprint: bridge.fingerprint,
readCliloc: bridge.readCliloc,
getSafe: uoLinkConfig.getSafe,
settingsGet: ctx.settings.get,
}
function restore() {
db.getMeta = saved.getMeta
db.replaceAll = saved.replaceAll
db.count = saved.count
bridge.fingerprint = saved.fingerprint
bridge.readCliloc = saved.readCliloc
uoLinkConfig.getSafe = saved.getSafe
ctx.settings.get = saved.settingsGet
}
const FINGERPRINT = {
kind: 'bridge',
file: 'cliloc.enu',
size: 4989921,
mtime: 1757462400000,
sha256: 'abc',
extractorVersion: 1,
hashing: false,
complete: true,
}
/**
* A rig with the shard reachable (or not), the configured overlay path pointed
* at a temp directory, and every write captured rather than made.
*/
function rig({ linked = true, meta = null, clientPath = '', rows = [] } = {}) {
const applied = []
uoLinkConfig.getSafe = async () => ({ enabled: linked, baseUrl: linked ? 'http://127.0.0.1:8099' : null })
ctx.settings.get = async (key) => (key === clilocs.SETTING_KEY ? clientPath : null)
db.getMeta = async () => meta
db.count = async () => meta?.count ?? 0
db.replaceAll = async (entries, writtenMeta) => {
applied.push({ entries, meta: writtenMeta })
return { count: entries.length, blank: 0, duplicates: 0 }
}
bridge.fingerprint = async () => FINGERPRINT
bridge.readCliloc = async () => ({
entries: rows,
source: {
kind: 'bridge',
lang: 'enu',
file: 'cliloc.enu',
size: FINGERPRINT.size,
mtime: FINGERPRINT.mtime,
pages: 1,
reported: rows.length,
received: rows.length,
},
})
return applied
}
function tmpWithOverlay(contents) {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cliloc-sel-'))
fs.mkdirSync(path.join(dir, 'custom'), { recursive: true })
if (contents !== undefined) fs.writeFileSync(path.join(dir, 'custom', 'shard.tsv'), contents)
return dir
}
// ── Which source runs ──────────────────────────────────────────────────────
test('a configured shard is the base source, and the file path is not consulted', async (t) => {
const applied = rig({ rows: [{ number: 1, flag: 0, text: 'a' }] })
t.after(restore)
const result = await clilocs.refresh()
assert.equal(result.status, 'imported')
assert.equal(result.source, 'bridge')
assert.equal(applied.length, 1)
assert.equal(applied[0].meta.source, 'bridge')
})
test('no shard link falls back to the file pipeline, unchanged', async (t) => {
rig({ linked: false })
t.after(restore)
// No path configured either, so the file path reports exactly what it always
// did — which is the assertion: the fallback is the OLD code, not a new one.
const result = await clilocs.refresh()
assert.equal(result.status, 'skipped')
assert.equal(result.reason, 'no cliloc path configured')
})
test('an explicit path is still an escape hatch, even with a shard linked', async (t) => {
let asked = false
rig({ linked: true })
bridge.fingerprint = async () => {
asked = true
return FINGERPRINT
}
t.after(restore)
const result = await clilocs.refresh({ path: path.join(os.tmpdir(), 'nope-does-not-exist') })
assert.equal(asked, false, 'the shard must not be asked when a file was named')
assert.equal(result.status, 'unavailable')
})
// Boot deliberately does not call the shard: it would put a sidecar round trip
// in the startup sequence to answer a question whose answer is "no" except after
// a client patch, which is an operator action.
test('boot imports nothing over the bridge and leaves the loaded table serving', async (t) => {
let asked = false
rig({ linked: true })
bridge.fingerprint = async () => {
asked = true
return FINGERPRINT
}
t.after(restore)
const result = await clilocs.refreshOnBoot()
assert.equal(result.status, 'skipped')
assert.equal(result.source, 'bridge')
assert.equal(asked, false)
})
// ── The gate ───────────────────────────────────────────────────────────────
test('an unchanged client file and no overlays is a no-op', async (t) => {
const applied = rig({
meta: { source: 'bridge', base: FINGERPRINT, hashes: {}, parserVersion: 1, count: 67496 },
})
t.after(restore)
const result = await clilocs.refresh()
assert.equal(result.status, 'unchanged')
assert.equal(result.count, 67496)
assert.equal(applied.length, 0)
})
test('a patched client re-imports', async (t) => {
const applied = rig({
meta: {
source: 'bridge',
base: { ...FINGERPRINT, sha256: 'older' },
hashes: {},
parserVersion: 1,
count: 10,
},
rows: [{ number: 1, flag: 0, text: 'a' }],
})
t.after(restore)
assert.equal((await clilocs.refresh()).status, 'imported')
assert.equal(applied.length, 1)
})
// The upgrade path. An install that used the converted-file pipeline carries its
// base label in the stored fingerprint; on the bridge that label is SUPPOSED to
// disappear. Counting it as a vanished source would make the first import after
// the upgrade demand an approval for a change the upgrade itself made.
test('the retired file base is not reported as a vanished source', async (t) => {
const applied = rig({
meta: {
source: 'file',
hashes: { 'clilocs.plain': 'aaa' },
parserVersion: 1,
count: 67496,
},
rows: [{ number: 1, flag: 0, text: 'a' }],
})
t.after(restore)
const result = await clilocs.refresh()
assert.equal(result.status, 'imported', result.reason)
assert.equal(applied.length, 1)
})
// An overlay is a different matter: it vanished, and an unmounted volume looks
// exactly like a deliberate deletion from here.
test('a vanished OVERLAY still stages for review', async (t) => {
const applied = rig({
meta: {
source: 'bridge',
base: FINGERPRINT,
hashes: { 'custom/shard.tsv': 'aaa' },
parserVersion: 1,
count: 5,
},
})
t.after(restore)
const result = await clilocs.refresh()
assert.equal(result.status, 'needsReview')
assert.deepEqual(result.missingSources, ['custom/shard.tsv'])
assert.equal(applied.length, 0)
const accepted = await clilocs.refresh({ approve: true })
assert.equal(accepted.status, 'imported')
assert.deepEqual(accepted.acceptedMissing, ['custom/shard.tsv'])
})
// ── The merge ──────────────────────────────────────────────────────────────
test('an overlay overrides the shard table, and says so', async (t) => {
const dir = tmpWithOverlay('1023721\ta better staff\n900001\ta shard-only item\n')
const applied = rig({
clientPath: dir,
rows: [
{ number: 1023721, flag: 0, text: 'quarter staff' },
{ number: 3000001, flag: 0, text: 'Entering Britannia...' },
],
})
t.after(() => {
restore()
fs.rmSync(dir, { recursive: true, force: true })
})
const result = await clilocs.refresh()
assert.equal(result.status, 'imported', result.reason)
const stored = new Map(applied[0].entries.map((e) => [e.number, e.text]))
assert.equal(stored.get(1023721), 'a better staff', 'the overlay must win')
assert.equal(stored.get(3000001), 'Entering Britannia...')
assert.equal(stored.get(900001), 'a shard-only item')
const overlay = result.sources.find((s) => s.kind === 'custom')
assert.equal(overlay.label, 'custom/shard.tsv')
assert.equal(overlay.added, 1)
assert.equal(overlay.overrode, 1)
// Only overlay hashes are stored now — the base is fingerprinted separately,
// and mixing them is what made the upgrade case above ambiguous.
assert.deepEqual(Object.keys(applied[0].meta.hashes), ['custom/shard.tsv'])
assert.equal(applied[0].meta.base.sha256, 'abc')
})
test('a malformed overlay names the file rather than failing the import namelessly', async (t) => {
const dir = tmpWithOverlay('not a cliloc file at all\n')
rig({ clientPath: dir, rows: [{ number: 1, flag: 0, text: 'a' }] })
t.after(() => {
restore()
fs.rmSync(dir, { recursive: true, force: true })
})
const result = await clilocs.refresh()
assert.equal(result.status, 'unavailable')
assert.match(result.reason, /custom\/shard\.tsv/)
})
// An overlay path an operator has mistyped must not stop a base table that
// arrived perfectly well — but it must be visible, or the site silently serves a
// table missing every shard-added name.
test('an unreadable overlay path is reported beside a successful import', async (t) => {
const applied = rig({
clientPath: path.join(os.tmpdir(), 'cliloc-does-not-exist-at-all'),
rows: [{ number: 1, flag: 0, text: 'a' }],
})
t.after(restore)
const result = await clilocs.refresh()
assert.equal(result.status, 'imported')
assert.match(result.overlayProblem, /does not exist/)
assert.equal(applied.length, 1)
})
// ── Status ─────────────────────────────────────────────────────────────────
test('status describes the shard source, hash state and drift', async (t) => {
rig({ meta: { source: 'bridge', base: FINGERPRINT, hashes: {}, parserVersion: 1, count: 67496 } })
t.after(restore)
const status = await clilocs.status()
assert.equal(status.source, 'bridge')
assert.equal(status.file, 'cliloc.enu')
assert.equal(status.fileReadable, true)
assert.equal(status.drift, false)
assert.equal(status.shard.extractorVersion, 1)
assert.equal(status.shard.hashing, false)
})
test('a shard that cannot be reached is a problem on the status, not a throw', async (t) => {
rig({})
bridge.fingerprint = async () => {
throw new bridge.ClilocBridgeError('The shard did not answer: timeout', 'SHARD_DOWN')
}
t.after(restore)
const status = await clilocs.status()
assert.equal(status.source, 'bridge')
assert.equal(status.fileReadable, false)
assert.equal(status.code, 'SHARD_DOWN')
// Null, not false: with no fingerprint there is nothing to compare, and
// reporting "no drift" would read as "up to date".
assert.equal(status.drift, null)
})

View File

@@ -96,8 +96,10 @@ test('the seventeen in-universe families have both channels; the nine plain ones
// letter from anybody.
assert.equal(r.template_keys.digest, 'notify.digest', `${r.trigger_id} digests generically`)
}
assert.equal(bespoke, 17)
assert.equal(seeds.TEMPLATES.length, 34)
// Eighteen since protocol 6: the champion FALLS, in the same crier's voice as
// the champion walking, because they are one story told in two mails.
assert.equal(bespoke, 18)
assert.equal(seeds.TEMPLATES.length, 36)
})
test('a template key is core\'s grammar — dots and hyphens, never an underscore', () => {
@@ -222,7 +224,7 @@ test('every declared fragment carries an example that shows its own shape', () =
// The `example` is what the template editor previews and test-sends with, so a
// trailing fragment whose example omits the leading space teaches an author the
// wrong thing about where to put one.
const TRAILING = ['slainBy', 'atPlace', 'inSuccessionTo', 'candidateNote']
const TRAILING = ['slainBy', 'atPlace', 'inSuccessionTo', 'candidateNote', 'damagerNote']
for (const t of TRIGGERS) {
for (const v of t.variables.filter((x) => TRAILING.includes(x.name))) {
assert.ok(v.example.startsWith(' '), `${t.id}.${v.name} example leads with its space`)
@@ -236,7 +238,17 @@ test('one rule group, and appending to it later would reach fresh installs only'
// A group is seeded ONCE under its own settings guard, which is 11a's seed-key
// finding as a mechanism. This assertion exists so that adding a twenty-sixth
// rule has to edit a test whose name says what appending costs.
assert.equal(seeds.RULE_GROUPS.length, 1)
// TWO groups since protocol 6, and the second one is this test's whole point
// made concrete: `uo.champ.boss_killed` could not be appended to `triggers-v1`,
// because a deployment that has already stamped that key would never have
// received it. A new rule gets a new key.
assert.equal(seeds.RULE_GROUPS.length, 2)
assert.equal(seeds.RULE_GROUPS[0].key, 'triggers-v1')
assert.equal(seeds.RULE_GROUPS[0].rules.length, 26)
assert.equal(seeds.RULE_GROUPS[1].key, 'champ-boss-killed-v1')
assert.deepEqual(seeds.RULE_GROUPS[1].rules.map((r) => r.trigger_id), ['uo.champ.boss_killed'])
// No rule belongs to two groups, and between them they are the whole set.
const grouped = seeds.RULE_GROUPS.flatMap((g) => g.rules.map((r) => r.trigger_id))
assert.equal(new Set(grouped).size, grouped.length)
assert.deepEqual([...grouped].sort(), seeds.RULES.map((r) => r.trigger_id).sort())
})

View File

@@ -53,6 +53,81 @@ test('registers exactly what module.json declares', () => {
assert.deepStrictEqual(api.record.extensions.map((e) => e.slot), manifest.extensions)
assert.deepStrictEqual(api.record.legs.map((l) => l.leg), ['towncrier'])
// The event contract (MODULE_API 1.10.0, EVENTS_PLAN.md Phase 9). Asserted
// here rather than only in the actions' own suite because registration is the
// half that can silently not happen: a declaration file nothing calls is a
// deployment whose event authors simply never see the verbs, with no error
// anywhere.
assert.deepStrictEqual(
api.record.eventActions.map((a) => a.id).sort(),
[
'uo.boss.spawn',
'uo.broadcast',
'uo.creature.spawn',
'uo.decor.place',
'uo.gate.open',
'uo.item.grant',
'uo.news.post',
'uo.npc.place',
'uo.participation.collect',
'uo.participation.open',
'uo.towncrier.post',
'uo.world.save',
],
)
// Phase 12a's five are all the MODULE's dimensions, never core's (org lead,
// 2026-09-07): core meters whatever a module declares and knows nothing about
// Ultima Online. Asserted as an ordered list because the order is the order
// an author meets them in a cap meter.
assert.deepStrictEqual(api.record.eventBudgets.map((b) => b.id), [
'uo.broadcasts',
'uo.creatures',
'uo.bosses',
'uo.npcs',
'uo.decor',
'uo.gate.minutes',
'uo.rewards',
])
// Phase 11b. One key, because ServUO has almost no others: of the 158 non-Bridge
// `Config.Get` call sites in `Scripts/`, roughly eight are read live, and a lease
// on any of the rest applies cleanly and does nothing.
// Phase 12b adds five TARGETED leases beside it -- a key that names a capability
// over many things, with the target supplied per step. Four spawner properties
// (`MaxCount`, not the `Amount` EVENTS_PLAN.md named: there is no such property
// on ServUO 57.4) and the seasonal status, which is a three-value enum over eight
// events rather than the nine-value one section G described.
assert.deepStrictEqual(api.record.eventLeases.map((l) => l.id), [
'uo.playercaps.skillcap',
'uo.spawner.maxcount',
'uo.spawner.mindelay',
'uo.spawner.maxdelay',
'uo.spawner.running',
'uo.seasonal.status',
])
// Only the targeted ones declare a target, and every one of them names a source:
// a target field with no list behind it is the free-text box the option-source
// contract exists to replace.
for (const lease of api.record.eventLeases) {
if (lease.id === 'uo.playercaps.skillcap') {
assert.strictEqual(lease.target, undefined, 'a config lease has no target')
continue
}
assert.ok(lease.target && lease.target.label, `${lease.id} has no target label`)
assert.ok(lease.target.source, `${lease.id} has no target source`)
}
assert.deepStrictEqual(
api.record.eventOptionSources.map((s) => s.id).sort(),
[
'uo.options.creatures',
'uo.options.decor',
'uo.options.items',
'uo.options.landmarks',
'uo.options.regions',
'uo.options.seasonal',
'uo.options.spawners',
],
)
assert.ok(api.record.streams.length > 0)
assert.strictEqual(typeof api.record.hooks.onBoot, 'function')
assert.strictEqual(typeof api.record.hooks.onShutdown, 'function')

View File

@@ -0,0 +1,559 @@
const { test } = require('node:test')
const assert = require('node:assert/strict')
const fs = require('node:fs')
const os = require('node:os')
const path = require('node:path')
const core = require('../core')
const model = require('../model/shardAssets/shardAssets.model')
const db = require('../model/shardAssets/shardAssets.db')
const atlasDb = require('../model/shardAtlas/shardAtlas.db')
const atlasModel = require('../model/shardAtlas/shardAtlas.model')
const bridge = require('../utils/assetBridge')
const uoLinkConfig = require('../model/uoLinkConfig/uoLinkConfig.model')
// The import as a decision, with the shard and the database both stubbed
// (docs/link/v8.md §6, §12 — protocol 8, phase 3).
//
// Each of these is a way the import can be wrong that an operator would either
// never notice or notice only weeks later, on a page:
//
// - Re-fetching every sprite on every Update. Correct output, and it makes the
// manifest — the entire reason stage 2 carries hashes instead of pixels —
// dead weight.
// - Silently dropping an asset the shard stopped offering. An unmounted client
// volume and a deliberate downgrade are the same thing from here, and the
// wrong guess deletes artwork nobody asked to delete.
// - Overwriting artwork the operator drew themselves. §12 states outright that
// theirs wins, and a sprite rip replacing hand-drawn portraits is not
// recoverable by pressing anything.
// - Treating a body with no art as a failure. Two thirds of the playable ghost
// and gargoyle bodies are in that state on a stock client.
const saved = {}
let uploadDir
function stubEverything({ manifest, fetched, held = new Map(), meta = null, sources } = {}) {
saved.sourceFingerprint = bridge.sourceFingerprint
saved.readManifest = bridge.readManifest
saved.fetchAssets = bridge.fetchAssets
saved.resolveBodies = bridge.resolveBodies
saved.allAssets = db.allAssets
saved.saveAssets = db.saveAssets
saved.recordLastImport = db.recordLastImport
saved.getMeta = db.getMeta
saved.countAssets = db.countAssets
saved.countBodies = db.countBodies
saved.replaceBodies = db.replaceBodies
saved.artBySlug = db.artBySlug
saved.allCreatureTypes = atlasDb.allCreatureTypes
saved.setCreatureArt = atlasDb.setCreatureArt
saved.loadArtMap = atlasModel.loadArtMap
saved.getSafe = uoLinkConfig.getSafe
const seen = { saved: null, fetchedKeys: null, art: null, last: null }
uoLinkConfig.getSafe = async () => ({ enabled: true, baseUrl: 'http://127.0.0.1:8080' })
bridge.sourceFingerprint = async () =>
sources ?? {
files: { 'anim.mul': { size: 1, mtime: 2, sha256: 'x' } },
extractorVersion: 1,
hashing: false,
complete: true,
imaging: { ok: true },
}
bridge.readManifest = async () => manifest
bridge.fetchAssets = async ({ keys }) => {
seen.fetchedKeys = keys
return fetched ?? { assets: new Map(), missing: { absent: 0, unsupported: 0 } }
}
bridge.resolveBodies = async () => []
db.allAssets = async () => held
db.getMeta = async () => meta
db.countAssets = async () => ({ total: held.size, stored: held.size })
db.countBodies = async () => ({ total: 0, resolved: 0 })
db.saveAssets = async (rows) => {
seen.saved = rows
return rows.length
}
db.recordLastImport = async (last) => {
seen.last = last
}
db.replaceBodies = async () => 0
db.artBySlug = async () => ({})
atlasDb.allCreatureTypes = async () => []
atlasDb.setCreatureArt = async (map) => {
seen.art = map
return Object.keys(map).length
}
atlasModel.loadArtMap = () => ({})
return seen
}
function restore() {
for (const [name, fn] of Object.entries(saved)) {
if (!fn) continue
if (name in db) db[name] = fn
if (name in bridge) bridge[name] = fn
if (name in atlasDb) atlasDb[name] = fn
if (name === 'loadArtMap') atlasModel.loadArtMap = fn
if (name === 'getSafe') uoLinkConfig.getSafe = fn
}
}
/** A real uploads directory, because the import checks the disk as well as the row. */
function useTempUploads(t) {
uploadDir = fs.mkdtempSync(path.join(os.tmpdir(), 'uo-assets-'))
const previous = core.uploads
Object.defineProperty(core, 'uploads', {
configurable: true,
get: () => ({ ...previous, UPLOAD_DIR: uploadDir }),
})
t.after(() => {
Object.defineProperty(core, 'uploads', { configurable: true, get: () => previous })
fs.rmSync(uploadDir, { recursive: true, force: true })
})
return uploadDir
}
const row = (body, sha) => ({
key: `body/${body}/a0`,
family: 'body',
sha256: sha,
bytes: 900,
width: 24,
height: 63,
body,
direction: 1,
})
const manifestOf = (rows) => ({
rows,
catalog: 'cat1',
extractorVersion: 1,
playerBodies: [400],
pages: 1,
scanned: 2047,
})
const sprite = (sha) => ({
sha256: sha,
bytes: 4,
width: 24,
height: 63,
body: 12,
direction: 1,
png: Buffer.from([0x89, 0x50, 0x4e, 0x47]),
})
// ── the gate ──────────────────────────────────────────────────────────────
test('unchanged client files import nothing at all', async (t) => {
const sources = {
files: { 'anim.mul': { size: 1, mtime: 2, sha256: 'x' } },
extractorVersion: 1,
hashing: false,
complete: true,
imaging: { ok: true },
}
stubEverything({ manifest: manifestOf([]), meta: { sources }, sources })
t.after(restore)
bridge.readManifest = async () => {
throw new Error('the gate should have stopped before reading a manifest')
}
const result = await model.importAssets()
assert.equal(result.status, 'unchanged')
})
test('a host that cannot render images is named rather than walked', async (t) => {
// §4.4: reported on the SOURCE gate, so an operator meets it while setting the
// shard up rather than from an empty bestiary weeks later.
stubEverything({
manifest: manifestOf([]),
sources: {
files: {},
extractorVersion: 1,
hashing: false,
complete: true,
imaging: { ok: false, code: 'NO_IMAGING', reason: 'needs libgdiplus' },
},
})
t.after(restore)
const result = await model.importAssets()
assert.equal(result.status, 'unavailable')
assert.equal(result.code, 'NO_IMAGING')
})
// ── the diff (§6) ─────────────────────────────────────────────────────────
test('only the keys whose hash moved are fetched', async (t) => {
const dir = useTempUploads(t)
fs.mkdirSync(path.join(dir, model.ART_SUBDIR), { recursive: true })
fs.writeFileSync(path.join(dir, model.ART_SUBDIR, 'kept.png'), 'x')
const held = new Map([
['body/12/a0', { key: 'body/12/a0', sha256: 'same', file: 'kept.png' }],
['body/34/a0', { key: 'body/34/a0', sha256: 'old', file: 'kept.png' }],
])
const seen = stubEverything({
held,
manifest: manifestOf([row(12, 'same'), row(34, 'new')]),
fetched: { assets: new Map([['body/34/a0', sprite('new')]]), missing: { absent: 0, unsupported: 0 } },
})
t.after(restore)
const result = await model.importAssets({ force: true })
assert.equal(result.status, 'imported')
// The whole point of a manifest that carries hashes and not pixels.
assert.deepEqual(seen.fetchedKeys, ['body/34/a0'])
assert.equal(result.written, 1)
})
test('a body catalogued at a later action is imported under that key', async (t) => {
// §11.2, phase 6. Body 820 has no art at action 0 and a horse at action 23, so
// its key is `body/820/a23` — and the filename, the stored row and the atlas
// join all have to agree on that. A name built as `uo-body-820-a0-…` would be
// a file nothing ever asks for, with the creature page still showing text.
const dir = useTempUploads(t)
const seen = stubEverything({
manifest: manifestOf([
{ ...row(820, 'new'), key: 'body/820/a23', action: 23 },
]),
fetched: {
assets: new Map([['body/820/a23', { ...sprite('new'), action: 23 }]]),
missing: { absent: 0, unsupported: 0 },
},
})
t.after(restore)
const result = await model.importAssets({ force: true })
assert.equal(result.status, 'imported')
assert.deepEqual(seen.fetchedKeys, ['body/820/a23'])
const saved = seen.saved[0]
assert.equal(saved.key, 'body/820/a23')
assert.equal(saved.action, 23)
// Content-addressed, and the stem is the key: the action is IN the filename.
assert.equal(saved.file, 'uo-body-820-a23-new.png')
assert.ok(fs.existsSync(path.join(dir, model.ART_SUBDIR, saved.file)))
})
test('an unchanged key whose file is missing from disk is fetched again', async (t) => {
// The row and the file can disagree — a wiped uploads volume, a restore from a
// database dump. Trusting the row alone leaves a broken image on a creature
// page with nothing anywhere reporting a problem, and re-fetching a sprite is
// far cheaper than that.
useTempUploads(t)
const held = new Map([['body/12/a0', { key: 'body/12/a0', sha256: 'same', file: 'gone.png' }]])
const seen = stubEverything({
held,
manifest: manifestOf([row(12, 'same')]),
fetched: { assets: new Map([['body/12/a0', sprite('same')]]), missing: { absent: 0, unsupported: 0 } },
})
t.after(restore)
await model.importAssets({ force: true })
assert.deepEqual(seen.fetchedKeys, ['body/12/a0'])
})
test('a key that vanished from the manifest needs review before anything changes', async (t) => {
useTempUploads(t)
const held = new Map([['body/99/a0', { key: 'body/99/a0', sha256: 'a', file: 'x.png' }]])
const seen = stubEverything({ held, manifest: manifestOf([row(12, 'a')]) })
t.after(restore)
const result = await model.importAssets({ force: true })
assert.equal(result.status, 'needsReview')
assert.equal(result.vanishedCount, 1)
// Nothing was applied. An unmounted client volume and a deliberate downgrade
// look identical from here.
assert.equal(seen.saved, null)
})
test('approve accepts the vanished key and removes its file', async (t) => {
const dir = useTempUploads(t)
fs.mkdirSync(path.join(dir, model.ART_SUBDIR), { recursive: true })
fs.writeFileSync(path.join(dir, model.ART_SUBDIR, 'gone.png'), 'x')
const held = new Map([['body/99/a0', { key: 'body/99/a0', sha256: 'a', file: 'gone.png' }]])
stubEverything({
held,
manifest: manifestOf([row(12, 'a')]),
fetched: { assets: new Map([['body/12/a0', sprite('a')]]), missing: { absent: 0, unsupported: 0 } },
})
t.after(restore)
const result = await model.importAssets({ force: true, approve: true })
assert.equal(result.status, 'imported')
assert.equal(result.removed, 1)
assert.equal(fs.existsSync(path.join(dir, model.ART_SUBDIR, 'gone.png')), false)
})
// ── absence is not failure (§5.2) ─────────────────────────────────────────
test('a key the shard could not render keeps the picture already held', async (t) => {
useTempUploads(t)
const held = new Map([['body/12/a0', { key: 'body/12/a0', sha256: 'old', file: 'existing.png' }]])
const seen = stubEverything({
held,
manifest: manifestOf([row(12, 'new')]),
// Listed, asked for, and not served. A shard that suddenly cannot render one
// sprite must not cost the picture we already have.
fetched: { assets: new Map(), missing: { absent: 1, unsupported: 0 } },
})
t.after(restore)
const result = await model.importAssets({ force: true })
assert.equal(result.status, 'imported')
assert.equal(result.absent, 1)
assert.equal(seen.saved[0].file, 'existing.png')
})
// ── the derivation (§12) ──────────────────────────────────────────────────
test("the operator's own artwork wins over an imported sprite", async (t) => {
useTempUploads(t)
const seen = stubEverything({ manifest: manifestOf([]) })
t.after(restore)
db.artBySlug = async () => ({ 'giant-spider': 'uo-body-28-aaaabbbb.png', wolf: 'uo-body-34-ccccdddd.png' })
// Someone who drew their own giant spider must not have it replaced by a
// sprite rip on the next Update. §12 states this outright.
atlasModel.loadArtMap = () => ({ 'giant-spider': 'my-own-spider.png' })
await model.importAssets({ force: true })
assert.equal(seen.art['giant-spider'], 'my-own-spider.png')
assert.equal(seen.art.wolf, 'uo-body-34-ccccdddd.png')
})
test('a sprite filename carries its hash so a changed picture is a changed URL', () => {
const before = model.fileNameFor('body/34/a0', 'aaaaaaaabbbb')
const after = model.fileNameFor('body/34/a0', 'ccccccccdddd')
// A stable name would be overwritten in place, and every browser and CDN that
// had cached it would keep serving last month's client's sprite — with the
// database row correct and nothing to notice.
assert.notEqual(before, after)
assert.match(before, /^uo-body-34-a0-[0-9a-f]{8}\.png$/)
})
// ── what the panel reads (phase 8) ────────────────────────────────────────
//
// The admin surface is the only thing that imports — boot never calls the shard
// — so everything an operator can learn about an import, they learn from what
// these two return. Each of these is a way the panel would render a confident
// sentence that is not true.
test('the vanished keys come back with the pictures they currently have', async (t) => {
useTempUploads(t)
const held = new Map([
['body/820/a23', { key: 'body/820/a23', sha256: 'a', file: 'uo-body-820-a23-aabbccdd.png' }],
])
stubEverything({ held, manifest: manifestOf([row(12, 'a')]) })
t.after(restore)
const result = await model.importAssets({ force: true })
// The decision being asked for is "is it right that these disappear?", and a
// key names nothing a human recognises. Without the filename the panel has
// nothing to show but `body/820/a23`, which is a horse.
assert.equal(result.status, 'needsReview')
assert.deepEqual(result.vanished, [
{ key: 'body/820/a23', file: 'uo-body-820-a23-aabbccdd.png' },
])
})
test('an import records what it did, including the body tally and who ran it', async (t) => {
useTempUploads(t)
const seen = stubEverything({
manifest: manifestOf([row(12, 'new')]),
fetched: {
assets: new Map([['body/12/a0', sprite('new')]]),
missing: { absent: 3, unsupported: 0 },
},
})
t.after(restore)
atlasDb.allCreatureTypes = async () => [{ slug: 'wolf', name: 'Wolf' }]
bridge.resolveBodies = async () => [
{ slug: 'wolf', typeName: 'Wolf', body: 34, status: 'ok' },
{ slug: 'ghost-of-something', typeName: 'GhostOfSomething', body: null, status: 'unknown' },
]
await model.importAssets({ force: true, by: 'colby' })
assert.equal(seen.last.by, 'colby')
assert.equal(seen.last.force, true)
assert.equal(seen.last.written, 1)
assert.equal(seen.last.absent, 3)
// The body pass is kept as a TALLY rather than a single "resolved" number:
// `unknown` means the spawn files name a type this shard's scripts do not
// define, which is real drift, and it reads identically to a failure if both
// are summed into "not resolved".
assert.deepEqual(seen.last.bodies, { ok: 1, unknown: 1, notCreature: 0, failed: 0 })
})
test('a summary that cannot be written does not fail an import that applied', async (t) => {
useTempUploads(t)
stubEverything({
manifest: manifestOf([row(12, 'new')]),
fetched: {
assets: new Map([['body/12/a0', sprite('new')]]),
missing: { absent: 0, unsupported: 0 },
},
})
t.after(restore)
db.recordLastImport = async () => {
throw new Error('the meta row is locked')
}
// The pictures are already on disk and the rows are already committed. Failing
// here would report a failure for an import that succeeded, and the operator's
// next move — press it again — would re-fetch the whole catalogue for nothing.
const result = await model.importAssets({ force: true })
assert.equal(result.status, 'imported')
assert.equal(result.written, 1)
})
test('status says whether a shard is linked rather than leaving it to be inferred', async (t) => {
stubEverything({ manifest: manifestOf([]) })
t.after(restore)
db.getMeta = async () => ({ catalog: 'cat1', last: { by: 'colby', written: 4 } })
const linked = await model.getStatus()
assert.equal(linked.linked, true)
assert.deepEqual(linked.loaded.last, { by: 'colby', written: 4 })
// A shard that is linked but DOWN also reports `shard: null`, which is why the
// panel cannot read this off that: one wants its buttons disabled and the
// other wants them available so the operator can retry.
uoLinkConfig.getSafe = async () => ({ enabled: false, baseUrl: '' })
const unlinked = await model.getStatus()
assert.equal(unlinked.linked, false)
assert.equal(unlinked.reason, 'uo-link is not configured')
})
test('the catalogue count is the body family, not every asset in the table', async (t) => {
stubEverything({ manifest: manifestOf([]) })
t.after(restore)
let askedFor = 'never called'
// Item and land art live in the same table as the body catalogue (phase 5) and
// are counted separately on purpose: one is a set with a size, the other is
// however much of an unbounded space the site has happened to ask for. A
// whole-table count reported 1,095 portraits plus 313 item pictures as a
// "1,408-row catalogue" on the one screen that answers "did the import work".
db.countAssets = async (family) => {
askedFor = family
return { total: 1095, stored: 1095 }
}
const status = await model.getStatus()
assert.equal(askedFor, 'body')
assert.equal(status.loaded.assets, 1095)
})
test('item pictures are not "vanished" just because the body manifest never listed them', async (t) => {
useTempUploads(t)
// The state every install reaches within a day of its first import: a body
// catalogue, plus whatever item art the warm pass has fetched because a
// marketplace page asked for it. Both live in `shard_assets`.
const held = new Map([
['body/12/a0', { key: 'body/12/a0', family: 'body', sha256: 'a', file: 'wolf.png' }],
['static/3934/h1801', { key: 'static/3934/h1801', family: 'static', sha256: 'b', file: 'robe.png' }],
])
const seen = stubEverything({ held, manifest: manifestOf([row(12, 'a')]) })
t.after(restore)
// The family filter is the fix, so the stub has to honour it or the test
// passes against a whole-table read.
db.allAssets = async (family) =>
new Map([...held].filter(([, r]) => !family || r.family === family))
const result = await model.importAssets({ force: true })
// Before the filter this was `needsReview` naming the item picture, and
// approving it would have deleted every picture the warm pass had fetched —
// with a sentence saying the shard had stopped offering them, which it had
// not: a body manifest never mentions item art at all.
assert.equal(result.status, 'imported')
assert.equal(result.removed, 0)
assert.ok(seen.saved)
})
test('an approved vanish deletes the row, not just the picture', async (t) => {
const dir = useTempUploads(t)
fs.mkdirSync(path.join(dir, model.ART_SUBDIR), { recursive: true })
fs.writeFileSync(path.join(dir, model.ART_SUBDIR, 'gone.png'), 'x')
const held = new Map([
['body/99/a0', { key: 'body/99/a0', family: 'body', sha256: 'a', file: 'gone.png' }],
])
let removedKeys = null
const seen = stubEverything({ held, manifest: manifestOf([row(12, 'a')]) })
t.after(restore)
db.saveAssets = async (rows, meta, remove) => {
seen.saved = rows
removedKeys = remove
return rows.length
}
const result = await model.importAssets({ force: true, approve: true })
assert.equal(result.removed, 1)
// The file was already unlinked before this fix; the ROW was not. A row whose
// picture is gone keeps being counted, keeps being offered for review on every
// forced import, and can still point a creature page at a file that is not
// there — with the import reporting "nothing was changed" about a deletion it
// had already performed.
assert.deepEqual(removedKeys, ['body/99/a0'])
assert.equal(fs.existsSync(path.join(dir, model.ART_SUBDIR, 'gone.png')), false)
})

View File

@@ -30,7 +30,11 @@ const one = (event) => {
// ── The catalogue itself ───────────────────────────────────────────────────
test('the declared set is the one ENGAGEMENT.md §8.6 commits to, carve-outs included', () => {
assert.equal(TRIGGERS.length, 26)
// 27 since protocol 6: `uo.champ.boss_killed` joins the twenty-six §8.6 named.
// It is not one of the four carve-outs below being reinstated — it is a row the
// catalogue could not have, because until protocol 6 the wire had no kind for a
// boss defeat and the inference from `champ.update` was not good enough to mail.
assert.equal(TRIGGERS.length, 27)
// The four rows that do NOT ship, each with its reason recorded in §8.6. This
// assertion is the guard on the carve-outs: adding one back is a decision, and
// a decision should have to edit a test that says so.
@@ -113,6 +117,10 @@ test('every url variable a body can interpolate is actually SUPPLIED', () => {
'uo.election.opened': [city(), city({ electionPhase: 'nominate', autoPickAt: inHours(48), candidates: 2 })],
'uo.champ.started': [champ({ active: false }), champ({ active: true })],
'uo.champ.boss_up': [champ({ bossUp: false }), champ({ bossUp: true })],
// Protocol 6. A single frame, unlike its two neighbours: a defeat is an
// EVENT on the wire rather than a change spotted between two snapshots, which
// is the whole reason the kind was worth a protocol bump.
'uo.champ.boss_killed': bossKilled(),
'uo.server.up': { kind: 'server.hello', shard: 'Rig' },
'uo.server.down': { kind: 'server.shutdown' },
'uo.page.new': { kind: 'page.new', type: 'Bug', sender: { name: 'Darrow' }, message: 'stuck' },
@@ -320,6 +328,21 @@ test('the pre-decision attempt kind is not mapped at all', () => {
const champ = (over) => ({ kind: 'champ.update', serial: '0x40012345', name: 'Abyss', category: 'champion', map: 'Felucca', x: 5187, y: 570, ...over })
// Protocol 6. The spawn serial matches `champ`'s, so the pair can be walked as
// one altar's story: the boss goes up, then it comes down.
const bossKilled = (over) => ({
kind: 'champ.boss.killed',
serial: '0x40012345',
bossSerial: '0x901', category: 'champion', boss: 'Semidar', bossType: 'Semidar',
map: 'Felucca', x: 5187, y: 570, region: 'Destard',
killer: { serial: '0x55', name: 'Aldric', acct: 'seed_002', player: true },
damagers: [
{ serial: '0x55', name: 'Aldric', acct: 'seed_002', player: true, damage: 900 },
{ serial: '0x56', name: 'Bran', acct: 'seed_003', player: true, damage: 120 },
],
...over,
})
test('a first sighting is never a transition — a reconnect is not twenty spawns starting', () => {
assert.deepEqual(ids(champ({ active: true })), [])
assert.deepEqual(ids(champ({ active: true })), []) // still no change
@@ -339,6 +362,64 @@ test('champ.remove forgets the spawn, so its next appearance is a first sighting
assert.deepEqual(ids(champ({ active: true })), [])
})
// ── champ.boss.killed (Protocol 6) ─────────────────────────────────────────
test('a defeat fires on the frame itself, with no baseline to compare against', () => {
// Unlike its two neighbours above. `champ.update` is a SNAPSHOT, so a first
// sighting can never be a transition; a defeat is an event, so a first sighting
// is exactly the thing being reported.
const hit = one(bossKilled())
assert.equal(hit.triggerId, 'uo.champ.boss_killed')
assert.equal(hit.data.bossName, 'Semidar')
assert.equal(hit.data.killerName, 'Aldric')
assert.equal(hit.data.damagerCount, 2)
assert.equal(hit.data.damagerNote, ' 2 players fought it.')
assert.equal(hit.data.location, 'Felucca 5187, 570 (Destard)')
})
test('the subject is the SPAWN, so boss_up and boss_killed share one cooldown subject', () => {
map(champ({ active: true, bossUp: false }))
const up = one(champ({ active: true, bossUp: true }))
const down = one(bossKilled())
assert.equal(up.triggerId, 'uo.champ.boss_up')
assert.equal(down.data.spawnSerial, up.data.spawnSerial)
})
test('a defeat the shard could not attribute to an altar stands on the boss itself', () => {
// The sweep learns which altar a champion belongs to; a boss that popped and
// died between two sweeps arrives with no `serial`. A subject that exists once
// is all a cooldown needs, so the boss's own serial stands in rather than the
// firing being dropped.
const hit = one(bossKilled({ serial: undefined }))
assert.equal(hit.data.spawnSerial, '0x901')
})
test('a defeat clears the tracker, so the next boss on that altar is a transition again', () => {
map(champ({ active: true, bossUp: false }))
map(champ({ active: true, bossUp: true })) // fires boss_up
map(bossKilled())
// Without the tracker reset this would emit nothing: the tracker would still
// believe a boss is up, so the next one would not look like a change.
assert.deepEqual(ids(champ({ active: true, bossUp: true })), ['uo.champ.boss_up'])
})
test('the damage TABLE never becomes trigger data, only its size', () => {
// `damagers` is `staff` in the visibility config. A trigger variable is
// interpolated into mail an operator may address to every subscriber, so a
// damager name reaching `data` would undo that field rule one layer up.
const hit = one(bossKilled())
const rendered = JSON.stringify(hit.data)
assert.equal(rendered.includes('Bran'), false, 'no damager name reaches the data')
assert.equal(rendered.includes('seed_003'), false, 'no damager account reaches the data')
assert.equal(hit.data.damagers, undefined)
})
test('an unattributed kill renders no damager sentence rather than an empty one', () => {
const hit = one(bossKilled({ damagers: [] }))
assert.equal(hit.data.damagerCount, undefined)
assert.equal(hit.data.damagerNote, undefined)
})
const city = (over) => ({ kind: 'city.update', city: 'Britain', electionPhase: 'none', ...over })
test('a governor change is a transition, and never on first sight', () => {

View File

@@ -0,0 +1,126 @@
// A shard restart makes the event resource ledger a claim about a world that no
// longer exists (EVENTS.md §F, EVENTS_PLAN.md Phases 8 and 9).
//
// Core cannot notice that on its own — it has no concept of the game being up —
// so the module says when, and `server.hello` carrying a *changed* `bootId` is
// the only signal that distinguishes a shard restart from a sidecar reconnect.
// Getting that wrong in either direction is a real failure: never asking leaves
// core believing a ledger of things that are gone, and asking on every reconnect
// makes core orphan rows that are perfectly alive.
const { test, beforeEach } = require('node:test')
const assert = require('node:assert/strict')
const shardIngest = require('../utils/shardIngest')
function makeDeps() {
const order = []
const noop = async () => {}
return {
order,
shardEvents: { append: noop },
shardState: { clearOnline: async () => { order.push('clearOnline') }, upsertOnline: noop, setOffline: noop },
shardLinks: {},
shardMarket: {},
uoLinkConfig: { recordStatus: async (row) => { order.push(`recordStatus:${row.bootId}`) } },
settings: { getInstanceName: async () => 'Rig' },
broadcast: () => {},
pushDispatch: () => {},
engagement: () => {},
eventsReconcile: () => { order.push('reconcile') },
log: { info: () => {}, warn: () => {}, error: () => {}, debug: () => {} },
}
}
const hello = (bootId) => ({ kind: 'server.hello', t: '2026-09-04T10:00:00Z', shard: 'Rig', bootId })
beforeEach(() => shardIngest.reset())
test('the first hello of a process is not a restart', async () => {
// The website has just come up and the shard has not moved. Everything in the
// ledger is still in force, and asking would be core spending a round trip per
// module to be told so.
const deps = makeDeps()
await shardIngest.ingest(hello('boot-1'), deps)
assert.ok(!deps.order.includes('reconcile'))
})
test('a sidecar reconnect is not a restart either', async () => {
// `server.hello` is sent on EVERY reconnect, and the sidecar dropping its
// socket changes nothing in the game. Reconciling here would orphan every live
// row — the ledger would still be right and core would stop believing it.
const deps = makeDeps()
await shardIngest.ingest(hello('boot-1'), deps)
await shardIngest.ingest(hello('boot-1'), deps)
assert.ok(!deps.order.includes('reconcile'))
})
test('a changed bootId asks every module to reconcile its ledger', async () => {
const deps = makeDeps()
await shardIngest.ingest(hello('boot-1'), deps)
await shardIngest.ingest(hello('boot-2'), deps)
assert.equal(deps.order.filter((s) => s === 'reconcile').length, 1)
})
test('the reconcile happens AFTER the new bootId is recorded', async () => {
// The ordering is load-bearing rather than tidy. Every action decides what is
// still in force by comparing its stamp against the CURRENT boot id, which it
// reads back out of the row `recordStatus` writes. Asking first would compare
// every resource against the boot that has just ended — and every one of them
// would look live, which is the exact opposite of what a restart means.
const deps = makeDeps()
await shardIngest.ingest(hello('boot-1'), deps)
await shardIngest.ingest(hello('boot-2'), deps)
const recordedAt = deps.order.lastIndexOf('recordStatus:boot-2')
const askedAt = deps.order.indexOf('reconcile')
assert.ok(recordedAt >= 0 && askedAt >= 0)
assert.ok(askedAt > recordedAt, 'reconcile must not run before the new boot id is stored')
})
test('a hello with no bootId at all changes nothing', async () => {
// An older plugin, or a frame that lost the field. Not knowing which boot this
// is cannot be allowed to read as "a new one".
const deps = makeDeps()
await shardIngest.ingest(hello('boot-1'), deps)
await shardIngest.ingest({ kind: 'server.hello', t: '2026-09-04T10:00:00Z', shard: 'Rig' }, deps)
assert.ok(!deps.order.includes('reconcile'))
})
test('a backfill replay never reconciles, however many boots it walks through', async () => {
// **The defect the live rig found, and nothing else could.** A WS reconnect
// replays the last several `server.hello` frames in order — this rig saw three,
// each with a different `bootId` — so every replayed frame looks like a
// restart. Acting on the intermediate ones would compare a resource stamped
// with the CURRENT boot against a boot that ended hours ago and mark it
// `orphaned`: a live crier line core will never take down again, lost to
// nothing worse than the website reconnecting.
const deps = makeDeps()
await shardIngest.ingest(hello('boot-1'), deps)
for (const boot of ['boot-2', 'boot-3', 'boot-4']) {
await shardIngest.ingest(hello(boot), { ...deps, fromBackfill: true })
}
assert.ok(!deps.order.includes('reconcile'))
// The replay still moves the tracked boot on, so the NEXT live hello is
// measured against where the replay left off rather than against boot-1.
assert.ok(deps.order.includes('recordStatus:boot-4'))
})
test('a live hello after a replay is still a restart', async () => {
// The gate is about the frame, not about the module going quiet: skipping the
// replay must not make the next genuine restart invisible.
const deps = makeDeps()
await shardIngest.ingest(hello('boot-1'), deps)
await shardIngest.ingest(hello('boot-2'), { ...deps, fromBackfill: true })
await shardIngest.ingest(hello('boot-3'), deps)
assert.equal(deps.order.filter((s) => s === 'reconcile').length, 1)
})
test('a reconcile that throws does not take the ingest down with it', async () => {
// Fire-and-forget by the contract, and the feed must survive one bad module:
// `ingest()` never throws, because a single event may not kill the socket.
const deps = makeDeps()
deps.eventsReconcile = () => { throw new Error('registry exploded') }
await shardIngest.ingest(hello('boot-1'), deps)
await assert.doesNotReject(() => shardIngest.ingest(hello('boot-2'), deps))
})

View File

@@ -0,0 +1,414 @@
const { test } = require('node:test')
const assert = require('node:assert/strict')
const fs = require('node:fs')
const os = require('node:os')
const path = require('node:path')
const core = require('../core')
const model = require('../model/shardAssets/shardItemArt.model')
const db = require('../model/shardAssets/shardAssets.db')
const bridge = require('../utils/assetBridge')
const uoLinkConfig = require('../model/uoLinkConfig/uoLinkConfig.model')
// The warm pass as a decision, with the shard and the database stubbed
// (docs/link/v8.md §5, §11 — protocol 8, phase 5).
//
// Item art has no manifest, so almost everything the body import gets from a
// hash diff this side has to get right by construction instead. Each test below
// is a way that goes wrong quietly:
//
// - Asking an overlay that cannot answer. A phase-4 plugin serves the creature
// catalogue and nothing else, and every static key it is sent is refused —
// once per pass, forever, in the log, with no picture ever appearing.
// - Re-fetching pictures the site already holds. There is no manifest to make
// that obvious, so the only thing standing between a working install and a
// pass that re-downloads its whole working set every five minutes is the
// per-row catalogue id.
// - NOT re-fetching after a client patch. The same field, read the other way.
// - Writing a row for a key the shard has no art for. It would make the key
// "held", and it would never be asked again — including after the operator
// patches in the graphic that was missing.
// - Spelling `static/3922/h0`. The shard refuses it outright (hue 0 means "not
// hued"), so a disagreement here is a picture that never arrives.
const saved = {}
function stub({
families = ['body', 'land', 'static'],
wanted = [],
fresh = new Set(),
files = new Map(),
fetched,
catalog = 'cat-current',
linked = true,
} = {}) {
saved.sourceFingerprint = bridge.sourceFingerprint
saved.fetchAssets = bridge.fetchAssets
saved.freshKeys = db.freshKeys
saved.filesForKeys = db.filesForKeys
saved.saveAssets = db.saveAssets
saved.getSafe = uoLinkConfig.getSafe
saved.query = core.query
const seen = { asked: [], saved: null, freshAsked: null, calls: 0 }
uoLinkConfig.getSafe = async () =>
linked ? { enabled: true, baseUrl: 'http://127.0.0.1:8080' } : { enabled: false }
bridge.sourceFingerprint = async () => ({
files: { 'art.mul': { size: 1, mtime: 2, sha256: 'x' } },
extractorVersion: 2,
hashing: false,
complete: true,
imaging: { ok: true },
families,
})
bridge.fetchAssets = async ({ keys }) => {
seen.calls++
seen.asked.push(keys)
// The catalogue probe asks for exactly one key and throws the answer away.
if (keys.length === 1 && keys[0] === 'static/0' && !fetched?.assets?.has('static/0')) {
return { assets: new Map(), missing: { absent: 1, unsupported: 0 }, pages: 1, catalog }
}
return (
fetched ?? { assets: new Map(), missing: { absent: 0, unsupported: 0 }, pages: 1, catalog }
)
}
// The derived set: what `SELECT DISTINCT item_id, hue FROM shard_vendor_items`
// would return.
core.query = async () => wanted
db.freshKeys = async (keys, askedCatalog) => {
seen.freshAsked = { keys, catalog: askedCatalog }
return fresh
}
db.filesForKeys = async () => files
db.saveAssets = async (rows, meta) => {
seen.saved = { rows, meta }
return rows.length
}
return seen
}
function restore() {
if (saved.sourceFingerprint) bridge.sourceFingerprint = saved.sourceFingerprint
if (saved.fetchAssets) bridge.fetchAssets = saved.fetchAssets
if (saved.freshKeys) db.freshKeys = saved.freshKeys
if (saved.filesForKeys) db.filesForKeys = saved.filesForKeys
if (saved.saveAssets) db.saveAssets = saved.saveAssets
if (saved.getSafe) uoLinkConfig.getSafe = saved.getSafe
if (saved.query) core.query = saved.query
}
function useTempUploads(t) {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'uo-items-'))
const previous = core.uploads
Object.defineProperty(core, 'uploads', {
configurable: true,
get: () => ({ ...previous, UPLOAD_DIR: dir }),
})
t.after(() => {
Object.defineProperty(core, 'uploads', { configurable: true, get: () => previous })
fs.rmSync(dir, { recursive: true, force: true })
})
return dir
}
const picture = (sha) => ({
sha256: sha,
bytes: 294,
width: 22,
height: 26,
hue: null,
partialHue: null,
source: 'uop',
png: Buffer.from('not really a png'),
})
// ── keys ───────────────────────────────────────────────────────────────────
test('hue 0 is the plain key, because the shard refuses /h0 for the same reason', () => {
// The wire's hue 0 means "this item is not hued". If this spelled `/h0` the
// shard would answer `unsupported` and the picture would never arrive; if the
// shard accepted it, the identical PNG would be stored twice under two names
// and diffed separately forever. The two sides agreeing is the whole point.
assert.equal(model.staticKey(3922, 0), 'static/3922')
assert.equal(model.staticKey(3922), 'static/3922')
assert.equal(model.staticKey(3922, null), 'static/3922')
assert.equal(model.staticKey(3922, 33), 'static/3922/h33')
})
test('a key is refused rather than fabricated for input that is not an item id', () => {
assert.equal(model.staticKey(-5), null)
assert.equal(model.staticKey('frog'), null)
assert.equal(model.staticKey(undefined), null)
assert.equal(model.landKey(0x4000), null)
assert.equal(model.landKey(3), 'land/3')
})
// ── the overlay gate ───────────────────────────────────────────────────────
test('an overlay that serves only the creature catalogue is reported, not asked', async (t) => {
// A phase-3 or phase-4 plugin. Every static key sent to it comes back refused,
// so discovering this per request would mean a warn per pass forever and no
// picture ever. It is one check, once, with a sentence naming the fix.
const seen = stub({ families: ['body'], wanted: [{ item_id: 3922, hue: 0 }] })
t.after(restore)
const result = await model.warm()
assert.equal(result.status, 'unavailable')
assert.equal(result.code, 'UNSUPPORTED')
assert.match(result.reason, /does not serve item art/)
assert.equal(seen.calls, 0, 'nothing should have been asked of the shard')
})
test('no shard link is skipped, not failed', async (t) => {
stub({ linked: false })
t.after(restore)
assert.equal((await model.warm()).status, 'skipped')
})
test('a host that cannot render images is the named NO_IMAGING state', async (t) => {
stub()
t.after(restore)
bridge.sourceFingerprint = async () => ({
files: {},
extractorVersion: 2,
hashing: false,
complete: true,
imaging: { ok: false, reason: 'libgdiplus is not installed' },
families: ['body', 'static'],
})
const result = await model.warm()
assert.equal(result.status, 'unavailable')
assert.equal(result.code, 'NO_IMAGING')
})
// ── what gets asked for ────────────────────────────────────────────────────
test('only the keys we do not already hold under the shards current catalogue are fetched', async (t) => {
useTempUploads(t)
const seen = stub({
wanted: [
{ item_id: 3922, hue: 0 },
{ item_id: 597, hue: 33 },
{ item_id: 1, hue: 0 },
],
// 3922 is held and current; the other two are not.
fresh: new Set(['static/3922']),
fetched: {
assets: new Map([
['static/597/h33', picture('aaa')],
['static/1', picture('bbb')],
]),
missing: { absent: 0, unsupported: 0 },
pages: 1,
catalog: 'cat-current',
},
})
t.after(restore)
const result = await model.warm()
assert.equal(result.status, 'imported')
// The first call is the catalogue probe; the second is the real fetch.
const asked = seen.asked[seen.asked.length - 1]
assert.deepEqual(asked.sort(), ['static/1', 'static/597/h33'])
assert.equal(
seen.freshAsked.catalog,
'cat-current',
'staleness must be asked against the catalogue the shard answers under right now, ' +
'or a client patch never invalidates anything',
)
})
test('every stored row records the catalogue it was fetched under', async (t) => {
useTempUploads(t)
const seen = stub({
wanted: [{ item_id: 1, hue: 0 }],
fetched: {
assets: new Map([['static/1', picture('bbb')]]),
missing: { absent: 0, unsupported: 0 },
pages: 1,
catalog: 'cat-after-patch',
},
})
t.after(restore)
await model.warm()
// Without this field there is no way to answer "is this picture out of date?"
// for a family that has no manifest — which is the entire §7 story on this side.
assert.equal(seen.saved.rows.length, 1)
assert.equal(seen.saved.rows[0].catalog, 'cat-after-patch')
assert.equal(seen.saved.rows[0].family, 'static')
})
test('the body catalogues meta singleton is never written by a warm pass', async (t) => {
useTempUploads(t)
const seen = stub({
wanted: [{ item_id: 1, hue: 0 }],
fetched: {
assets: new Map([['static/1', picture('bbb')]]),
missing: { absent: 0, unsupported: 0 },
pages: 1,
catalog: 'cat-current',
},
})
t.after(restore)
await model.warm()
// `shard_asset_meta` is what an Update compares a BODY manifest against. A
// warm pass writing there would tell the body import that a client it never
// looked at is unchanged, and the creature catalogue would stop updating.
assert.equal(seen.saved.meta, null)
})
test('a key the shard has no art for produces no row, so it can be asked again', async (t) => {
useTempUploads(t)
const seen = stub({
wanted: [
{ item_id: 1, hue: 0 },
{ item_id: 60000, hue: 0 },
],
fetched: {
assets: new Map([['static/1', picture('bbb')]]),
missing: { absent: 1, unsupported: 0 },
pages: 1,
catalog: 'cat-current',
},
})
t.after(restore)
const result = await model.warm()
assert.equal(result.absent, 1)
assert.deepEqual(
seen.saved.rows.map((r) => r.key),
['static/1'],
'an empty row would make the key held, and it would never be asked again — ' +
'including after the operator patches in the graphic that was missing',
)
})
test('a pass is bounded, and says how much it left behind', async (t) => {
useTempUploads(t)
const wanted = []
for (let i = 1; i <= 10; i++) wanted.push({ item_id: i, hue: 0 })
const seen = stub({
wanted,
fetched: {
assets: new Map([['static/1', picture('bbb')]]),
missing: { absent: 0, unsupported: 0 },
pages: 1,
catalog: 'cat-current',
},
})
t.after(restore)
const result = await model.warm({ limit: 4 })
assert.equal(seen.asked[seen.asked.length - 1].length, 4)
assert.equal(result.asked, 4)
assert.equal(result.remaining, 6)
})
test('a picture whose bytes changed replaces its file instead of shadowing it', async (t) => {
const dir = useTempUploads(t)
const old = model.fileNameFor('static/1', 'old00000')
fs.mkdirSync(model.artDir(), { recursive: true })
fs.writeFileSync(path.join(model.artDir(), old), 'stale')
stub({
wanted: [{ item_id: 1, hue: 0 }],
files: new Map([['static/1', old]]),
fetched: {
assets: new Map([['static/1', picture('new00000')]]),
missing: { absent: 0, unsupported: 0 },
pages: 1,
catalog: 'cat-after-patch',
},
})
t.after(restore)
await model.warm()
const names = fs.readdirSync(path.join(dir, model.ART_SUBDIR))
// Content-addressed names mean a changed picture is a changed URL, so nothing
// keeps serving last client's sprite from a cache — and the superseded file is
// removed rather than left to accumulate one per client patch forever.
assert.deepEqual(names, [model.fileNameFor('static/1', 'new00000')])
})
// ── serving ────────────────────────────────────────────────────────────────
test('decorate attaches a filename, never a URL, and null where there is none', async (t) => {
stub({ files: new Map([['static/3922', 'uo-static-3922-abcd1234.png']]) })
t.after(restore)
const rows = [
{ itemId: 3922, hue: 0 },
{ itemId: 597, hue: 33 },
]
await model.decorate(rows)
// A filename, because the client is what knows where uploads are mounted —
// the same contract `shard_spawn_creatures.art` already uses.
assert.equal(rows[0].art, 'uo-static-3922-abcd1234.png')
assert.equal(rows[1].art, null)
})
test('decorate never throws a page away over a picture', async (t) => {
stub()
t.after(restore)
db.filesForKeys = async () => {
throw new Error('the database is on fire')
}
const rows = [{ itemId: 3922, hue: 0 }]
await model.decorate(rows)
assert.deepEqual(rows, [{ itemId: 3922, hue: 0 }], 'the row is returned unchanged, not lost')
})
test('what a page asked for is remembered, including what it could not show', async (t) => {
stub({ files: new Map() })
t.after(restore)
const before = model.noticedCount()
await model.decorate([{ itemId: 12345, hue: 7 }])
// The character sheet is fetched live from the shard and stored nowhere, so
// nothing on disk would ever name this key. Noticing it here is the only reason
// a warm pass can find it.
assert.ok(model.noticedCount() > before)
assert.ok((await model.wantedKeys()).includes('static/12345/h7'))
})

View File

@@ -95,6 +95,58 @@ test('an unknown viewer level cannot see a gated kind or a locked field', async
assert.equal('webId' in out.leader, false)
})
// ── Protocol 6: the champion defeat ──────────────────────────────────
const KILL = {
kind: 'champ.boss.killed',
serial: '0x40012345',
boss: 'Semidar',
killer: { serial: '0x55', name: 'Aldric', acct: 'seed_002', player: true },
damagers: [
{ serial: '0x55', name: 'Aldric', acct: 'seed_002', webId: '7', player: true, damage: 900 },
{ serial: '0x56', name: 'Bran', acct: 'seed_003', player: true, damage: 120 },
],
}
test('the kill is public and its damage table is not', () => {
const config = visibility.compileDefaults()
// The whole shape of this addition in one assertion: a champion falling is
// content the public board is FOR, and a ranked roll of who was strong enough
// to fell it is a performance record nobody published on purpose.
assert.equal(visibility.kindVisibleTo('champ.boss.killed', 'anonymous', config), true)
for (const level of ['anonymous', 'logged_in', 'player']) {
const out = visibility.projectFeature('champs', KILL, level, config)
assert.equal(out.boss, 'Semidar', `${level} sees which boss fell`)
assert.equal('damagers' in out, false, `${level} must not see the damage table`)
}
assert.equal(visibility.projectFeature('champs', KILL, 'staff', config).damagers.length, 2)
})
test('the killer rides the frame the way mob.killed already publishes one', () => {
// Deliberately NOT a configurable field. It is one actor, announced in-game to
// everyone present, and the same disclosure the public activity feed has made
// through `mob.killed` since before this framework existed.
const config = visibility.compileDefaults()
const out = visibility.projectFeature('champs', KILL, 'anonymous', config)
assert.equal(out.killer.name, 'Aldric')
assert.equal('acct' in out.killer, false, 'rule 1 still applies inside it')
})
test('an admin who lowers the damager rule still cannot see an account inside it', () => {
// Rule 1 beats a field rule wherever the two meet, and a damager entry is an
// actor object like any other. An admin who opens the table to everyone has
// published character names, which is what they chose; they have not published
// account names, which is not theirs to choose.
const config = visibility.compileDefaults()
config.champs.fields = { ...config.champs.fields, damagers: 'anonymous' }
const out = visibility.projectFeature('champs', KILL, 'anonymous', config)
assert.equal(out.damagers.length, 2)
assert.equal(out.damagers[0].name, 'Aldric')
assert.equal(out.damagers[0].damage, 900)
assert.equal('acct' in out.damagers[0], false)
assert.equal('webId' in out.damagers[0], false)
})
// ── Rule 1: locked fields ──────────────────────────────────────────────────
test('acct and webId are stripped below admin regardless of feature config', () => {
@@ -360,10 +412,21 @@ const V3_ADDED_PUBLIC_KINDS = ['world.ruleset', 'points.board']
// inside the roster's member array (see the roster test above).
const V4_ADDED_PUBLIC_KINDS = ['guild.roster', 'guild.leave']
test('derived PUBLIC_KINDS is exactly the pre-v3 allowlist plus the v3 and v4 additions', () => {
// v6 adds the champion defeat. It rides the existing `champs` feature, which is
// already anonymous, so the KIND is public — while the `damagers` table on it is
// `staff` by field rule. That split is the point: a shard announces that its
// champion fell without publishing a roll of who was strong enough to fell it.
const V6_ADDED_PUBLIC_KINDS = ['champ.boss.killed']
test('derived PUBLIC_KINDS is exactly the pre-v3 allowlist plus the v3, v4 and v6 additions', () => {
assert.deepEqual(
[...visibility.PUBLIC_KINDS].sort(),
[...PRE_V3_PUBLIC_KINDS, ...V3_ADDED_PUBLIC_KINDS, ...V4_ADDED_PUBLIC_KINDS].sort(),
[
...PRE_V3_PUBLIC_KINDS,
...V3_ADDED_PUBLIC_KINDS,
...V4_ADDED_PUBLIC_KINDS,
...V6_ADDED_PUBLIC_KINDS,
].sort(),
)
})

View File

@@ -14,6 +14,7 @@ const {
buildFacetIndex,
resolveFacetName,
slugify,
parseDecoration,
decodeEntities,
} = require('../utils/spawnAtlasParse')
@@ -144,9 +145,14 @@ test('parsePoints: reads the kept fields and drops the rest', () => {
assert.equal(covetous.minDelay, 300)
assert.equal(covetous.maxDelay, 600)
assert.deepEqual(covetous.types, [{ type: 'Lizardman', max: 3 }])
// Dropped fields must not survive into the artifact — this is what keeps it
// under 1 MB.
assert.equal(covetous.uniqueId, undefined)
// **The UniqueId is KEPT from Phase 12b**, having been dropped since the atlas
// shipped. It is `XmlSpawner.UniqueId` — carried in the spawn files and on the
// live spawner — so it is the only name for one particular spawner that exists
// off the shard, and a property lease targets by it. A serial cannot do that
// job: serials are assigned when the world is built and nothing here knows one.
assert.equal(covetous.uniqueId, '001a34e5-0efa-46de-9c93-b6a163d96370')
// The rest of the dropped fields still are. Triggering, refractory windows,
// proximity and sounds are what the site has no use for.
assert.equal(covetous.proximityTriggerSound, undefined)
})
@@ -599,3 +605,49 @@ test('parsePoints: DelayInSec decides the unit, and both come out in seconds', (
assert.equal(seconds.minDelay, 5)
assert.equal(seconds.maxDelay, 10)
})
// ── parseDecoration (Phase 12a) ───────────────────────────────
test('parseDecoration: reads the type off each header and ignores the placements', () => {
const rows = parseDecoration(`# switch
Static 0x108F
5552 1864 11
5399 1875 17
# crate
LargeCrate 0x0E3C
5408 607 45
`)
assert.deepEqual(rows, [
{ type: 'Static', itemId: 0x108f },
{ type: 'LargeCrate', itemId: 0x0e3c },
])
})
test('parseDecoration: a parenthesised property list is not part of the type', () => {
// These are the shard's own decoration details — which way a door faces, what
// hue a banner is — and an event author is choosing neither. Only the class
// name is, because that is what the plugin constructs from.
assert.deepEqual(parseDecoration('AnkhNorth 0x0004 (Hue=0x47E)'), [
{ type: 'AnkhNorth', itemId: 4 },
])
assert.deepEqual(parseDecoration('ArmsAndWeaponsPrimer 0x0FEF (Name=a life of travel)'), [
{ type: 'ArmsAndWeaponsPrimer', itemId: 0x0fef },
])
})
test('parseDecoration: a negative z on a placement line is not mistaken for a type', () => {
// The real trap in this format: a coordinate line starts with a digit OR a
// minus, so "not a comment" is not the test. A z of -12 is ordinary in every
// dungeon file in the tree.
assert.deepEqual(parseDecoration(`Static 0x07A4
5558 1826 -12
-5 -5 -5
`), [{ type: 'Static', itemId: 0x07a4 }])
})
test('parseDecoration: empty, comment-only and absent input all yield nothing', () => {
assert.deepEqual(parseDecoration(''), [])
assert.deepEqual(parseDecoration(null), [])
assert.deepEqual(parseDecoration('# nothing but a comment\n\n'), [])
})

View File

@@ -39,11 +39,27 @@ function writeTree(root, { facets = ['Sosaria'], includeChampions = true } = {})
fs.mkdirSync(path.join(root, 'Data', 'Locations'), { recursive: true })
fs.mkdirSync(path.join(root, 'Config'), { recursive: true })
// Decoration, NESTED, because the real tree nests two deep in places
// (`Magincia/Trammel`, `Stygian Abyss/Ter Mur`) and a flat read would index a
// fraction of it while looking like it worked.
fs.mkdirSync(path.join(root, 'Data', 'Decoration', 'Deep', 'Deeper'), { recursive: true })
fs.writeFileSync(
path.join(root, 'Data', 'Decoration', 'top.cfg'),
'# a brazier\nBrazier 0x0E31\n100 100 0\n200 200 -5\n\nStatic 0x108F\n300 300 0\n',
'utf8',
)
fs.writeFileSync(
path.join(root, 'Data', 'Decoration', 'Deep', 'Deeper', 'nested.cfg'),
'Brazier 0x0E31\n400 400 0\nLargeCrate 0x0E3C\n500 500 0\n',
'utf8',
)
for (const facet of facets) {
fs.writeFileSync(
path.join(root, 'Spawns', `${facet}.xml`),
`<Spawns>
<Points><Name>${facet}A</Name><Map>${facet}</Map><X>1100</X><Y>1100</Y>
<Points><Name>${facet}A</Name><UniqueId>uid-${facet}-A</UniqueId>
<Map>${facet}</Map><X>1100</X><Y>1100</Y>
<MaxCount>3</MaxCount><IsRunning>True</IsRunning>
<Objects2>Lizardman:MX=3:SB=0:OBJ=Orc:MX=1:SB=0</Objects2></Points>
<Points><Name>${facet}B</Name><Map>${facet}</Map><X>9000</X><Y>9000</Y>
@@ -97,6 +113,29 @@ function tempTree(options) {
// ── buildAtlas against a custom-facet tree ─────────────────────────────────
test('buildAtlas: a point keeps the UniqueId a property lease targets', () => {
// The field is asserted on the AGGREGATOR's output, not the parser's, which is
// the whole point of this test. `parsePoints` produced it from Phase 12b
// onwards and `PARSER_VERSION`'s own note said a point kept it, while the
// mapping in `buildAtlas` rebuilt each point from an explicit field list that
// omitted it — so `shard_spawn_points.unique_id` was NULL on every row, and
// `listSpawners`, whose WHERE is `unique_id IS NOT NULL`, answered empty. That
// left `uo.options.spawners` an empty dropdown and every Phase 12b
// object-property lease unauthorable. Found by the Phase 16b released-artefact
// walk, against a real tree whose files carry ~6,400 of these.
//
// The fixture above had no <UniqueId> at all until this test, which is exactly
// why a green suite said nothing about it.
const root = tempTree({ facets: ['Sosaria'] })
const atlas = buildAtlas(root)
const named = atlas.points.find((p) => p.name === 'SosariaA')
assert.equal(named.uniqueId, 'uid-Sosaria-A')
// And a point whose file names none is absent rather than empty-string, so the
// DB layer's `unique_id IS NOT NULL AND <> ''` reads it the same way either way.
const unnamed = atlas.points.find((p) => p.name === 'SosariaB')
assert.ok(!unnamed.uniqueId)
})
test('buildAtlas: works entirely on facets that do not exist in stock UO', () => {
const root = tempTree({ facets: ['Sosaria', 'Underdark'] })
const atlas = buildAtlas(root)
@@ -397,3 +436,81 @@ test('refresh: an explicit path overrides the configured one', async () => {
assert.equal(result.status, 'imported')
assert.deepEqual(result.addedFacets, ['Override'])
})
// ── The decoration index (Phase 12a) ────────────────────────
test('decoration is read recursively and rolled up per type', () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'atlas-decor-'))
try {
writeTree(root)
const atlas = buildAtlas(root)
// Sorted by type, and `uses` counts every header line across the whole tree
// — the nested file's Brazier is the second use of the same type, not a
// second type.
assert.deepEqual(atlas.decor, [
{ type: 'Brazier', itemId: 0x0e31, uses: 2 },
{ type: 'LargeCrate', itemId: 0x0e3c, uses: 1 },
{ type: 'Static', itemId: 0x108f, uses: 1 },
])
assert.equal(atlas.meta.counts.decor, 3)
// Every decoration file is fingerprinted like every other source, so an
// operator editing one is a tree change the boot path notices.
const labels = Object.keys(atlas.meta.source).filter((l) => l.startsWith('Data/Decoration/'))
assert.deepEqual(labels.sort(), ['Data/Decoration/Deep/Deeper/nested.cfg', 'Data/Decoration/top.cfg'])
} finally {
fs.rmSync(root, { recursive: true, force: true })
}
})
test('two spellings of one decoration type fold into one row', () => {
// The Phase 16 acceptance walk's blocking finding. Stock ServUO 57.4's own
// `Data/Decoration/` names four types under two casings each —
// CheckerBoard/Checkerboard, ChessBoard/Chessboard, MetalChest/Metalchest,
// SpinningWheelEastAddon/SpinningwheelEastAddon — and in every pair exactly one
// is a real class; the other is a mis-cased line the shard's own loader resolves
// anyway.
//
// A case-SENSITIVE Map keeps both. `shard_decor_types.type` is a PRIMARY KEY
// under MariaDB's default `..._ai_ci` collation, which folds case, so the second
// row raised `1062 Duplicate entry` and took the WHOLE atlas import transaction
// down with it. The blast radius is not decoration: with no atlas, EVERY option
// source answers empty and no world verb can be authored at all.
//
// Asserted on the count as well as the row, because the failure mode was two
// rows that a database — not this function — would later refuse.
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'atlas-decorcase-'))
try {
writeTree(root)
fs.writeFileSync(
path.join(root, 'Data', 'Decoration', 'miscased.cfg'),
'checkerboard 0x0FA6\n600 600 0\nCheckerBoard 0x0FA6\n700 700 0\n',
)
const atlas = buildAtlas(root)
const boards = atlas.decor.filter((d) => d.type.toLowerCase() === 'checkerboard')
assert.equal(boards.length, 1, 'two casings of one type must not be two rows')
// First spelling seen wins, exactly as the first item id does. Which one
// survives is cosmetic — the shard resolves either.
assert.equal(boards[0].type, 'checkerboard')
assert.equal(boards[0].uses, 2, 'both lines still count as uses of the one type')
} finally {
fs.rmSync(root, { recursive: true, force: true })
}
})
test('a tree with no decoration at all still builds', () => {
// Optional, like the champion file. A shard that has stripped its decoration
// has a perfectly good atlas; the decoration verb simply has nothing to offer.
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'atlas-nodecor-'))
try {
writeTree(root)
fs.rmSync(path.join(root, 'Data', 'Decoration'), { recursive: true, force: true })
const atlas = buildAtlas(root)
assert.deepEqual(atlas.decor, [])
assert.equal(atlas.meta.counts.decor, 0)
} finally {
fs.rmSync(root, { recursive: true, force: true })
}
})

View File

@@ -0,0 +1,413 @@
const fs = require('fs')
const os = require('os')
const path = require('path')
const zlib = require('zlib')
const crypto = require('crypto')
const { test, after } = require('node:test')
const assert = require('node:assert/strict')
// Installs the `ctx` core would have handed over — treeBridge takes a logger
// from it at call time, so a test that skips this dies on the first log line.
require('./_setup')
const uoLinkClient = require('../utils/uoLinkClient')
const treeBridge = require('../utils/treeBridge')
const { buildFrom, readFrom } = require('../utils/spawnAtlasSource')
// The atlas source walk over the bridge (docs/link/v8.md §10 — protocol 8,
// phase 7), driven against a stub that behaves the way `BridgeTree.cs` does.
//
// The test that matters most is the LAST one: the same synthetic tree, read off
// a disk and read over the bridge, must produce a byte-identical atlas. Every
// other test here is one specific way a walk can end in something that LOOKS
// imported — which is the failure mode this whole family is shaped around, since
// XML is forgiving enough that a tree reassembled wrong still parses and simply
// has fewer spawns in it.
const saved = {}
function restore() {
for (const [name, fn] of Object.entries(saved)) {
if (fn) uoLinkClient[name] = fn
}
}
after(restore)
const ok = (data) => ({ ok: true, status: 200, data })
const fail = (status, data) => ({ ok: false, status, data })
const sha = (buf) => crypto.createHash('sha256').update(buf).digest('hex')
// ── A stub shard ───────────────────────────────────────────────────────────
//
// Chunks and gzips exactly as the overlay does, so the reader under test is
// exercised against the wire shape rather than against a convenience.
function serveTree(files, { chunkBytes = 64, catalog = 'cafebabe12345678', tweak = {} } = {}) {
saved.getAssetManifest = saved.getAssetManifest ?? uoLinkClient.getAssetManifest
saved.fetchAssets = saved.fetchAssets ?? uoLinkClient.fetchAssets
const chunksOf = (bytes) => Math.max(1, Math.ceil(bytes.length / chunkBytes))
const rows = files.map(([label, bytes]) => ({
key: `tree/${label}`,
label,
bytes: bytes.length,
mtime: 1700000000000,
chunks: chunksOf(bytes),
sha256: sha(bytes),
}))
const byLabel = new Map(files)
const calls = { manifest: 0, fetch: 0 }
uoLinkClient.getAssetManifest = async ({ family, cursor } = {}) => {
calls.manifest++
assert.equal(family, 'tree', 'the walk must name its family')
assert.equal(cursor ?? null, null, 'this stub answers in one page')
if (tweak.manifestReply) return tweak.manifestReply(rows, catalog)
return ok({
kind: 'assets.manifest.ok',
family: 'tree',
catalog,
chunkBytes,
total: rows.length,
rows,
more: false,
cut: 'end',
})
}
uoLinkClient.fetchAssets = async ({ keys, catalog: asked } = {}) => {
calls.fetch++
assert.equal(asked, catalog, 'a fetch must assert the catalog it was listed under')
const out = []
for (const key of keys) {
const slash = key.lastIndexOf('/')
const label = key.slice('tree/'.length, slash)
const chunk = Number(key.slice(slash + 2))
const bytes = byLabel.get(label)
if (!bytes) {
out.push({ key, status: 'absent', reason: 'no such file' })
continue
}
const raw = bytes.subarray(chunk * chunkBytes, (chunk + 1) * chunkBytes)
out.push({
key,
status: 'ok',
label,
chunk,
chunks: chunksOf(bytes),
offset: chunk * chunkBytes,
bytes: raw.length,
sha256: sha(raw),
gzip: zlib.gzipSync(raw).toString('base64'),
})
}
if (tweak.fetchRows) tweak.fetchRows(out)
return ok({
kind: 'assets.fetch.ok',
family: 'tree',
catalog,
rows: out,
more: false,
cut: 'end',
...(tweak.fetchEnvelope || {}),
})
}
return calls
}
const FILES = [
['Data/Regions.xml', Buffer.from('<ServerRegions><Region /></ServerRegions>', 'utf8')],
['Spawns/Sosaria.xml', Buffer.from('<Spawns>' + 'x'.repeat(400) + '</Spawns>', 'utf8')],
]
// ── The walk ───────────────────────────────────────────────────────────────
test('a chunked, gzipped tree reassembles to the exact bytes the shard holds', async () => {
const calls = serveTree(FILES)
const { files } = await treeBridge.readSources()
assert.equal(files.length, 2)
assert.equal(calls.manifest, 1, 'one manifest call')
for (const [label, bytes] of FILES) {
const got = files.find((f) => f.label === label)
assert.ok(got, `${label} came back`)
assert.equal(got.text, bytes.toString('utf8'))
assert.equal(got.bytes, bytes.length)
assert.equal(got.sha256, sha(bytes))
}
restore()
})
test('chunks are placed by their declared index, not by the order they arrive in', async () => {
// The rows come back in the order they were asked for today. A reader that
// appended them would agree with this test until the day something reorders a
// page — and then produce a file that still parses and is quietly wrong.
serveTree(FILES, { tweak: { fetchRows: (rows) => rows.reverse() } })
const { files } = await treeBridge.readSources()
const spawns = files.find((f) => f.label === 'Spawns/Sosaria.xml')
assert.equal(spawns.text, FILES[1][1].toString('utf8'))
restore()
})
test('a chunk the shard refuses fails the import rather than shortening a file', async () => {
serveTree(FILES, {
tweak: {
fetchRows: (rows) => {
rows[rows.length - 1] = { key: rows[rows.length - 1].key, status: 'absent', reason: 'gone' }
},
},
})
await assert.rejects(() => treeBridge.readSources(), /refused .*absent: gone/)
restore()
})
test('a missing chunk is named, with which one and out of how many', async () => {
serveTree(FILES, { tweak: { fetchRows: (rows) => rows.splice(2, 1) } })
await assert.rejects(() => treeBridge.readSources(), /missing chunk 1 of/)
restore()
})
test('a chunk that does not match its own hash is refused', async () => {
serveTree(FILES, {
tweak: {
fetchRows: (rows) => {
rows[1].gzip = zlib.gzipSync(Buffer.from('not what was hashed')).toString('base64')
rows[1].bytes = 19
},
},
})
await assert.rejects(() => treeBridge.readSources(), /does not match its own hash/)
restore()
})
test('a file whose reassembly does not match its manifest hash is refused', async () => {
// Every chunk is individually honest and the whole is not — which is what a
// dropped or duplicated chunk looks like from here.
serveTree(FILES, {
tweak: {
manifestReply: (rows, catalog) =>
ok({
kind: 'assets.manifest.ok',
family: 'tree',
catalog,
chunkBytes: 64,
total: rows.length,
rows: rows.map((r) => ({ ...r, sha256: r.sha256.replace(/^./, '0') })),
more: false,
cut: 'end',
}),
},
})
await assert.rejects(() => treeBridge.readSources(), /does not match the hash its manifest row carried/)
restore()
})
test('a tree that moves mid-read is refused rather than stitched together', async () => {
serveTree(FILES, { tweak: { fetchEnvelope: { catalog: 'deadbeefdeadbeef' } } })
await assert.rejects(() => treeBridge.readSources(), {
code: 'SOURCE_CHANGED',
})
restore()
})
test('a short page that did not end the walk is refused', async () => {
serveTree(FILES, { tweak: { fetchEnvelope: { more: false, cut: 'budget' } } })
await assert.rejects(() => treeBridge.readSources(), { code: 'INCOMPLETE' })
restore()
})
test('403 names the tree switch, not the asset switch', async () => {
// The two consents are different settings with different fixes, and sending an
// operator to Bridge.AssetsEnabled when the answer is Bridge.TreeEnabled costs
// them an afternoon.
saved.getAssetManifest = saved.getAssetManifest ?? uoLinkClient.getAssetManifest
uoLinkClient.getAssetManifest = async () => fail(403, { reason: 'not served' })
await assert.rejects(() => treeBridge.readSources(), {
code: 'DISABLED',
message: /Bridge\.TreeEnabled/,
})
restore()
})
test('the manifest alone answers the drift gate, with no file bytes at all', async () => {
const calls = serveTree(FILES)
const listing = await treeBridge.manifest()
const fingerprint = treeBridge.fingerprintOf(listing.files)
assert.equal(calls.fetch, 0, 'nothing was fetched to answer "has anything changed"')
assert.deepEqual(Object.keys(fingerprint).sort(), [
'Data/Regions.xml',
'Spawns/Sosaria.xml',
])
assert.equal(fingerprint['Spawns/Sosaria.xml'], sha(FILES[1][1]))
restore()
})
test('a zero-byte file crosses as one chunk carrying a real gzip stream', async () => {
// Stock ServUO 57.4 ships TWO empty decoration files, so this is the ordinary
// case rather than an edge one — and it is the case .NET gets wrong on its own:
// `GZipStream` writes the gzip header lazily, so zero bytes in produces zero
// bytes out, which is not a gzip stream at all. The overlay answers with a
// literal empty member; a reader that accepted an empty payload instead would
// have hidden the bug rather than caught it.
const empty = Buffer.alloc(0)
serveTree([
['Data/Regions.xml', Buffer.from('<ServerRegions />', 'utf8')],
['Data/Decoration/nothing.cfg', empty],
])
const { files } = await treeBridge.readSources()
const blank = files.find((f) => f.label === 'Data/Decoration/nothing.cfg')
assert.equal(blank.bytes, 0)
assert.equal(blank.text, '')
assert.equal(blank.sha256, sha(empty))
restore()
})
test('an empty payload for a chunk is refused, whatever the row declares', async () => {
serveTree(FILES, { tweak: { fetchRows: (rows) => { rows[0].gzip = '' } } })
await assert.rejects(() => treeBridge.readSources(), /Could not decompress/)
restore()
})
test('a tree-only shard tells the client-file readers so, rather than looking empty', async () => {
// Phase 7 opened `assets.sources` to a shard that serves ONLY its configuration
// tree, so a 200 from it stopped meaning "the client files are on offer". Both
// client-file readers have to say DISABLED rather than read the empty file list
// as "your UO client has no cliloc.enu", which sends an operator to their client
// install for a setting that lives on their shard.
const clilocBridge = require('../utils/clilocBridge')
const assetBridge2 = require('../utils/assetBridge')
saved.getAssetSources = saved.getAssetSources ?? uoLinkClient.getAssetSources
uoLinkClient.getAssetSources = async () =>
ok({
kind: 'assets.sources.ok',
extractorVersion: 3,
assetsEnabled: false,
treeEnabled: true,
imaging: { ok: true },
families: ['tree'],
files: [],
more: false,
cut: 'end',
complete: true,
})
await assert.rejects(() => clilocBridge.fingerprint(), {
code: 'DISABLED',
message: /Bridge\.AssetsEnabled/,
})
await assert.rejects(() => assetBridge2.sourceFingerprint(), {
code: 'DISABLED',
message: /Bridge\.AssetsEnabled/,
})
restore()
})
// ── The parity test ────────────────────────────────────────────────────────
test('the same tree read off a disk and read over the bridge builds the same atlas', async () => {
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'atlas-parity-'))
const tree = [
[
'Data/Regions.xml',
'<?xml version="1.0"?><ServerRegions>'
+ '<region type="Region"><name>Yew</name><map>Sosaria</map>'
+ '<rect x="100" y="100" width="200" height="200" /></region>'
+ '</ServerRegions>',
],
[
'Data/Locations/Sosaria.xml',
'<?xml version="1.0"?><locations><location><name>Yew Bank</name>'
+ '<x>150</x><y>150</y><z>0</z></location></locations>',
],
[
'Spawns/Sosaria.xml',
'<?xml version="1.0"?><Spawns>'
+ Array.from({ length: 40 }, (_, i) =>
`<Spawn Name="s${i}" X="${120 + i}" Y="${130 + i}" Map="Sosaria" Count="3" `
+ 'Running="True" MinDelay="00:05:00" MaxDelay="00:10:00" SpawnRange="5" '
+ 'HomeRange="5"><Object>Lizardman</Object></Spawn>').join('')
+ '</Spawns>',
],
['Config/ChampionSpawns.xml', '<?xml version="1.0"?><champions />'],
['Data/Decoration/top.cfg', 'Brazier 0x0E31\n100 100 0\n'],
['Data/Decoration/Deep/nested.cfg', 'LargeCrate 0x0E3C\n500 500 0\n'],
]
for (const [label, text] of tree) {
const file = path.join(root, label.replace(/\//g, path.sep))
fs.mkdirSync(path.dirname(file), { recursive: true })
fs.writeFileSync(file, text, 'utf8')
}
const fromDisk = await buildFrom({ kind: 'fs', root })
// A chunk size small enough that the spawn file alone is dozens of chunks,
// because a one-chunk-per-file test proves nothing about reassembly.
serveTree(tree.map(([label, text]) => [label, Buffer.from(text, 'utf8')]), { chunkBytes: 37 })
const fromBridge = await buildFrom({ kind: 'bridge' })
// `generatedAt` is a timestamp and the only field that legitimately differs.
delete fromDisk.meta.generatedAt
delete fromBridge.meta.generatedAt
// **Serialised, not deepEqual.** `deepEqual` ignores object key order, and key
// order is precisely what differed between the two readers on a real tree —
// which a live walk caught and this test, written first, did not.
assert.equal(JSON.stringify(fromBridge), JSON.stringify(fromDisk))
assert.deepEqual(fromBridge, fromDisk)
// And the source fingerprints agree, which is what makes switching backends on
// an existing install NOT look like a change to the drift gate.
assert.deepEqual(fromBridge.meta.source, fromDisk.meta.source)
restore()
fs.rmSync(root, { recursive: true, force: true })
})
test('readFrom hands both backends back in one shape', async () => {
serveTree(FILES)
const bridged = await readFrom({ kind: 'bridge' })
assert.deepEqual(Object.keys(bridged), ['files'])
assert.deepEqual(Object.keys(bridged.files[0]).sort(), ['bytes', 'label', 'sha256', 'text'])
restore()
})

View File

@@ -0,0 +1,957 @@
// module-uo's event verbs, wave 1 (EVENTS_PLAN.md Phase 9).
//
// The declarations are data plus three `perform()`s, so most of this suite is
// about the *shapes* core will check and the failure paths a live rig cannot be
// made to produce on demand — a sidecar that answers 409, a shard that restarts
// between two steps, a crier line one character over the cap.
//
// **The first test is the one the whole phase rests on.** Every other property
// here — "a broadcast is sent once", "a failed post is retried" — is a claim
// about what the MODULE decided, and the module only gets to decide when its
// client answers before core's dispatch deadline. Assert the relationship, not
// the numbers, or the day someone tunes one of them the suite stays green while
// the behaviour inverts.
const { test, beforeEach, afterEach } = require('node:test')
const assert = require('node:assert/strict')
const uoLinkClient = require('../utils/uoLinkClient')
const uoLinkConfig = require('../model/uoLinkConfig/uoLinkConfig.model')
const shardAtlas = require('../model/shardAtlas/shardAtlas.model')
require('./_setup')
const actions = require('../config/uoEventActions')
const byId = (id) => actions.ACTIONS.find((a) => a.id === id)
let calls
const saved = {}
beforeEach(() => {
calls = {
broadcast: [], crier: [], crierDel: [], news: [], newsDel: [],
spawn: [], despawn: [], owned: [],
}
for (const name of [
'adminBroadcast', 'postTownCrier', 'deleteTownCrier', 'postNews', 'deleteNews',
'spawnWorld', 'ownedWorld', 'despawnWorld',
]) {
saved[name] = uoLinkClient[name]
}
saved.getSafe = uoLinkConfig.getSafe
saved.listRegions = shardAtlas.listRegions
saved.listLandmarks = shardAtlas.listLandmarks
saved.searchCreatures = shardAtlas.searchCreatures
saved.listDecorTypes = shardAtlas.listDecorTypes
saved.getDecorType = shardAtlas.getDecorType
uoLinkClient.adminBroadcast = async (b) => { calls.broadcast.push(b); return { ok: true, status: 200 } }
uoLinkClient.postTownCrier = async (b) => { calls.crier.push(b); return { ok: true, status: 200 } }
uoLinkClient.deleteTownCrier = async (id) => { calls.crierDel.push(id); return { ok: true, status: 200 } }
uoLinkClient.postNews = async (b) => { calls.news.push(b); return { ok: true, status: 200 } }
uoLinkClient.deleteNews = async (id) => { calls.newsDel.push(id); return { ok: true, status: 200 } }
// Phase 12a. Two serials back by default, so a spawn produces a resource list
// longer than one and the per-serial ledger shape is what the suite exercises.
uoLinkClient.spawnWorld = async (b) => {
calls.spawn.push(b)
const n = b.count || 1
return {
ok: true,
status: 200,
data: { serials: Array.from({ length: n }, (_, i) => `0x4000000${i}`) },
}
}
uoLinkClient.ownedWorld = async (b) => {
calls.owned.push(b)
return { ok: true, status: 200, data: { owned: [{ serial: '0x40000000', what: 'creature' }] } }
}
uoLinkClient.despawnWorld = async (b) => {
calls.despawn.push(b)
return { ok: true, status: 200, data: { removed: b.serials || [], gone: [], refused: [] } }
}
uoLinkConfig.getSafe = async () => ({ bootId: 'boot-1' })
// Phase 11b. `uo.participation.open` resolves its `place` param against the
// atlas, so the dry-run sweep below reaches this rather than the database.
// Two landmarks, because Phase 12a's gate verb resolves a SECOND place: its
// destination. One would make the dry-run sweep below pass for the wrong
// reason, by never exercising the leg that can name a different point.
shardAtlas.listLandmarks = async () => [
{ facet: 'Felucca', name: 'Britain', x: 1496, y: 1628, z: 10 },
{ facet: 'Felucca', name: 'Yew', x: 542, y: 982, z: 0 },
]
shardAtlas.listDecorTypes = async () => [{ type: 'Brazier', itemId: 0x0E31, uses: 42 }]
shardAtlas.getDecorType = async (type) =>
type === 'Brazier' ? { type: 'Brazier', itemId: 0x0E31, uses: 42 } : null
})
afterEach(() => {
for (const name of ['adminBroadcast', 'postTownCrier', 'deleteTownCrier', 'postNews', 'deleteNews']) {
uoLinkClient[name] = saved[name]
}
uoLinkConfig.getSafe = saved.getSafe
shardAtlas.listRegions = saved.listRegions
shardAtlas.listLandmarks = saved.listLandmarks
shardAtlas.searchCreatures = saved.searchCreatures
shardAtlas.listDecorTypes = saved.listDecorTypes
shardAtlas.getDecorType = saved.getDecorType
for (const name of ['spawnWorld', 'ownedWorld', 'despawnWorld']) {
uoLinkClient[name] = saved[name]
}
})
// ── The rule everything else depends on ────────────────────────────────────
test('every action outlives the sidecar client, so the module classifies its own failures', () => {
// `dispatch.classify()` answers `retry` for a budget timeout unconditionally
// and never asks the action. If core's deadline can fire before the client
// gives up, `retry: false` below is unreachable and a broadcast is retried.
for (const action of actions.ACTIONS) {
assert.ok(
action.budgetMs > uoLinkClient.TIMEOUT_MS,
`${action.id} budgetMs (${action.budgetMs}) must exceed uoLinkClient.TIMEOUT_MS (${uoLinkClient.TIMEOUT_MS})`,
)
}
})
// ── The declarations, against the checks core will run ─────────────────────
test('the declarations satisfy the shape core validates them with', () => {
const RISKS = ['notify', 'inspect', 'change', 'irreversible']
const REVERSIBLE = ['none', 'self', 'ledger', 'override']
const PARAM_TYPES = ['string', 'int', 'float', 'boolean', 'datetime', 'url']
for (const a of actions.ACTIONS) {
assert.ok(a.id.startsWith('uo.'), `${a.id} must be namespaced to this module`)
assert.ok(a.label && a.description, `${a.id} needs a label and a description`)
assert.ok(RISKS.includes(a.risk), `${a.id} has an unknown risk class`)
assert.ok(REVERSIBLE.includes(a.reversible), `${a.id} has an unknown reversible class`)
assert.equal(typeof a.perform, 'function')
// `revert` is required iff ledger, and forbidden otherwise — a revert on a
// non-ledgering action is an undo core will never call.
assert.equal(
typeof a.revert === 'function',
a.reversible === 'ledger',
`${a.id} revert() must be present exactly when reversible is 'ledger'`,
)
// `reconcile` is optional, but only meaningful where something is ledgered.
if (a.reconcile !== undefined) {
assert.equal(typeof a.reconcile, 'function')
assert.ok(a.reversible === 'ledger' || a.reversible === 'override', `${a.id} reconciles but ledgers nothing`)
}
if (a.cost !== undefined) assert.equal(typeof a.cost, 'function')
const names = new Set()
for (const p of a.params) {
assert.ok(!names.has(p.name), `${a.id} declares ${p.name} twice`)
names.add(p.name)
assert.ok(PARAM_TYPES.includes(p.type), `${a.id}.${p.name} has an unsupported type "${p.type}"`)
// Required on every param including the optional ones: it is the authoring
// placeholder, and an unattended world write typed into a blank box is how
// a typo gets scheduled.
assert.ok(
p.example !== undefined && p.example !== null && p.example !== '',
`${a.id}.${p.name} needs an example`,
)
assert.ok(p.description, `${a.id}.${p.name} needs a description`)
}
}
})
test('every dimension a cost names is one this module declares', () => {
const declared = new Set(actions.BUDGETS.map((b) => b.id))
// Phase 12a's six and Phase 12b's seventh are all the MODULE's (org lead,
// 2026-09-07): core meters what a module declares and holds no UO knowledge, so
// a `uo.` dimension core knew about would be a leak of this game into the engine.
//
// `uo.rewards` counts ITEMS rather than grants: a step giving 500 gold to forty
// people and one giving a candle to forty people are not the same imposition, and
// a count of grants would price them identically.
assert.deepEqual(
[...declared],
[
'uo.broadcasts',
'uo.creatures',
'uo.bosses',
'uo.npcs',
'uo.decor',
'uo.gate.minutes',
'uo.rewards',
],
)
for (const b of actions.BUDGETS) {
assert.ok(b.id.startsWith('uo.'), 'a budget dimension must be namespaced')
assert.ok(b.label && b.unit, 'a dimension is rendered as a label and a unit beside a number')
}
// Every dimension a cost names must be one the module declared, or core is
// asked to bound something nothing defines.
const cost = byId('uo.broadcast').cost({})
assert.deepEqual(cost, { 'uo.broadcasts': 1 })
for (const id of Object.keys(cost)) assert.ok(declared.has(id), `${id} is spent but never declared`)
// The keyed verbs deliberately spend nothing: a repeat REPLACES under the same
// id, so there is no runaway for a cap to bound.
assert.equal(byId('uo.towncrier.post').cost, undefined)
assert.equal(byId('uo.news.post').cost, undefined)
// Phase 12a. Asserted across EVERY action rather than one at a time, because
// the failure this catches is a typo in one dimension name out of six, which
// core answers by refusing the whole registration at load.
for (const action of actions.ACTIONS) {
if (typeof action.cost !== 'function') continue
const params = {}
for (const p of action.params) params[p.name] = p.example
for (const id of Object.keys(action.cost(params))) {
assert.ok(declared.has(id), `${action.id} spends "${id}", which nothing declares`)
}
}
// A gate is priced in MINUTES, not in gates. One standing all day and twelve
// standing five minutes each are not the same imposition on a world, and a
// count would price them identically.
assert.deepEqual(byId('uo.gate.open').cost({ durationMinutes: 120 }), { 'uo.gate.minutes': 120 })
assert.deepEqual(byId('uo.creature.spawn').cost({ count: 8 }), { 'uo.creatures': 8 })
})
// ── uo.broadcast: retried, because protocol 6 made that safe ───────────────
test('a broadcast is retried on a transient failure and never on a permanent one', async () => {
const broadcast = byId('uo.broadcast')
// Wave 1 asserted the opposite of this — every failure terminal, including the
// two that are plainly transient — because nothing on the wire could stop a
// retry announcing to everyone twice. Protocol 6 puts an idempotency key on the
// command and the shard refuses the repeat, so the trade that test recorded is
// no longer one that has to be made.
//
// 425 is the new status in this list: `bridge.busy`, the shard saying a command
// under this key is still in flight. Transient by construction.
const TRANSIENT = new Set([0, 425, 503, 504])
for (const status of [0, 400, 401, 403, 409, 425, 503, 504]) {
uoLinkClient.adminBroadcast = async () => ({ ok: false, status, error: `status ${status}` })
const result = await broadcast.perform({ runId: 7, params: { text: 'hear ye' }, verify: false })
assert.equal(result.ok, false)
assert.equal(result.retry, TRANSIENT.has(status), `a ${status} retries iff it is transient`)
}
})
test('every write carries the step idempotency key, unchanged', async () => {
// The key is what makes the retry above safe, so a verb that dropped it would
// silently restore the wave-1 hazard while every other assertion still passed.
// Asserted per verb rather than once, because each builds its own body.
const KEY = 'a'.repeat(40)
const seen = {}
uoLinkClient.adminBroadcast = async (body) => { seen.broadcast = body; return { ok: true } }
uoLinkClient.postTownCrier = async (body) => { seen.crier = body; return { ok: true } }
uoLinkClient.postNews = async (body) => { seen.news = body; return { ok: true } }
await byId('uo.broadcast').perform({
runId: 7, idempotencyKey: KEY, params: { text: 'hear ye' }, verify: false,
})
await byId('uo.towncrier.post').perform({
runId: 7, idempotencyKey: KEY, params: { lines: 'hear ye' }, verify: false,
})
await byId('uo.news.post').perform({
runId: 7, idempotencyKey: KEY, params: { title: 'A thing', body: 'happened' }, verify: false,
})
assert.equal(seen.broadcast.idempotencyKey, KEY)
assert.equal(seen.crier.idempotencyKey, KEY)
assert.equal(seen.news.idempotencyKey, KEY)
// The two keyed verbs post under an id DERIVED from the key. Both travel: the
// id is what makes a repeat replace, the key is what stops it re-announcing.
assert.equal(seen.crier.id, `evt-${KEY}`)
assert.equal(seen.news.id, `evt-${KEY}`)
})
test("the shard's own words reach the run log, not just a status code", async () => {
// **The rig found this.** The sidecar refuses a broadcast with
// `{"reason":"admin write plane disabled"}` and `legError` looks for
// `data.message`, so the run console read "sidecar responded 403" for a cause
// the shard had already explained in a sentence. A staff member clicking a
// button knows what they switched off; an event that ran at four in the morning
// leaves the run log as the only place anyone will learn why.
uoLinkClient.adminBroadcast = async () => ({
ok: false,
status: 403,
data: { kind: 'admin.error', reason: 'admin write plane disabled' },
error: 'sidecar responded 403',
})
const result = await byId('uo.broadcast').perform({ runId: 1, params: { text: 'hear ye' }, verify: false })
assert.match(result.error, /admin write plane disabled/)
// And NOT the double-announce clause: a 403 will not succeed on any attempt, so
// pointing an operator at a policy decision misdirects them away from the
// switch they actually have to flip.
assert.doesNotMatch(result.error, /announce twice/)
assert.equal(result.retry, false)
})
test('a permanent refusal of a keyed verb is not retried either', async () => {
// Same distinction on the other side: the keyed verbs DO retry a transient, and
// must not burn three attempts on a refusal that cannot change.
uoLinkClient.postTownCrier = async () => ({ ok: false, status: 403, data: { reason: 'admin write plane disabled' } })
const result = await byId('uo.towncrier.post').perform({
runId: 1, idempotencyKey: 'k'.repeat(40), params: { lines: 'hear ye' }, verify: false,
})
assert.equal(result.retry, false)
assert.match(result.error, /admin write plane disabled/)
})
test('a broadcast names its run in the shard audit, not a staff member', async () => {
await byId('uo.broadcast').perform({ runId: 42, params: { text: 'hear ye', hue: 1153 }, verify: false })
assert.equal(calls.broadcast.length, 1)
assert.equal(calls.broadcast[0].actor, 'event:42')
assert.equal(calls.broadcast[0].hue, 1153)
})
test('an over-long broadcast is refused by the DRY RUN, before anything is sent', async () => {
const broadcast = byId('uo.broadcast')
const text = 'x'.repeat(actions.MAX_BROADCAST_LEN + 1)
const dry = await broadcast.perform({ runId: 1, params: { text }, verify: true })
assert.equal(dry.ok, false)
assert.equal(dry.retry, false)
assert.match(dry.error, new RegExp(String(actions.MAX_BROADCAST_LEN)))
const live = await broadcast.perform({ runId: 1, params: { text }, verify: false })
assert.equal(live.ok, false)
assert.deepEqual(calls.broadcast, [], 'nothing may reach the shard once the cap is breached')
})
test('a dry run sends nothing at all', async () => {
for (const action of actions.ACTIONS) {
const params = {}
for (const p of action.params) if (p.required) params[p.name] = p.example
const result = await action.perform({ runId: 1, stepId: 1, idempotencyKey: 'k'.repeat(40), params, verify: true })
assert.equal(result.ok, true, `${action.id} refused its own example params`)
assert.equal(result.resources, undefined, `${action.id} reported a resource it never created`)
}
assert.deepEqual(
[calls.broadcast.length, calls.crier.length, calls.news.length],
[0, 0, 0],
'a dry run reached the shard',
)
})
// ── The keyed verbs: one id, stable across a retry ─────────────────────────
test('the crier and the news gump post under a run-stable id a retry replaces', async () => {
const key = 'a1b2c3'.padEnd(40, '0')
await byId('uo.towncrier.post').perform({ runId: 3, idempotencyKey: key, params: { lines: 'hear ye' }, verify: false })
await byId('uo.towncrier.post').perform({ runId: 3, idempotencyKey: key, params: { lines: 'hear ye' }, verify: false })
assert.equal(calls.crier.length, 2)
assert.equal(calls.crier[0].id, calls.crier[1].id, 'a retry must replace, not stack')
assert.equal(calls.crier[0].id, `evt-${key}`)
// The sidecar's own cap on the id column.
assert.ok(calls.crier[0].id.length <= 64)
})
test('an event article cannot collide with a website post in the news gump', async () => {
// `newsGump.js` posts site articles under the bare post id and re-pushes that
// whole set on every reconnect. An event article numbered into the same space
// would silently be a collision with a post, in whichever direction wrote last.
await byId('uo.news.post').perform({
runId: 9,
idempotencyKey: 'f'.repeat(40),
params: { title: 'The Fair', body: 'Merchants gather.' },
verify: false,
})
assert.equal(calls.news.length, 1)
assert.doesNotMatch(calls.news[0].id, /^\d+$/, 'an event article must not be numbered like a post')
assert.match(calls.news[0].id, /^evt-/)
assert.match(calls.news[0].body, /<CENTER>The Fair<\/CENTER>/)
assert.equal(calls.news[0].announce, true, 'announce defaults on, as the gump does')
})
test('the keyed verbs DO retry, because a repeat replaces', async () => {
for (const [id, stub] of [['uo.towncrier.post', 'postTownCrier'], ['uo.news.post', 'postNews']]) {
const params = { lines: 'hear ye', title: 'The Fair', body: 'Merchants gather.' }
// The announce leg's own classification of this transport, reused rather
// than re-decided: a config or data problem is terminal, the rest transient.
for (const [status, retry] of [[400, false], [401, false], [403, false], [409, false], [503, true], [504, true], [0, true]]) {
uoLinkClient[stub] = async () => ({ ok: false, status, error: `status ${status}` })
const result = await byId(id).perform({ runId: 1, idempotencyKey: 'k'.repeat(40), params, verify: false })
assert.equal(result.ok, false)
assert.equal(result.retry, retry, `${id} misclassified a ${status}`)
}
}
})
test('a crier post is refused before it is sent when it is not eight short lines', async () => {
const crier = byId('uo.towncrier.post')
const cases = [
['', /empty/],
[' \n ', /empty/],
[Array.from({ length: actions.MAX_CRIER_LINES + 1 }, (_, i) => `line ${i}`).join('\n'), /criers carry/],
['x'.repeat(actions.MAX_CRIER_LINE_LEN + 1), /capped at/],
]
for (const [lines, expected] of cases) {
const result = await crier.perform({ runId: 1, idempotencyKey: 'k'.repeat(40), params: { lines }, verify: false })
assert.equal(result.ok, false)
assert.equal(result.retry, false, 'a badly shaped message is just as badly shaped next minute')
assert.match(result.error, expected)
}
assert.deepEqual(calls.crier, [])
})
test('blank lines are dropped rather than counted against the cap', () => {
// A textarea an operator has pressed enter in twice still holds two lines.
const parsed = actions.crierLines('hear ye\n\n \nseek the herald\n')
assert.equal(parsed.ok, true)
assert.deepEqual(parsed.lines, ['hear ye', 'seek the herald'])
})
test('a crier duration is taken in minutes and bounded at the sidecar cap', async () => {
const crier = byId('uo.towncrier.post')
const base = { runId: 1, idempotencyKey: 'k'.repeat(40), verify: false }
await crier.perform({ ...base, params: { lines: 'hear ye', durationMinutes: 90 } })
assert.equal(calls.crier[0].durationSec, 5400)
await crier.perform({ ...base, params: { lines: 'hear ye', durationMinutes: 60 * 48 } })
assert.equal(calls.crier[1].durationSec, 86400, 'a duration past the sidecar cap is clamped, not refused')
// Left out entirely, so the sidecar applies its own default rather than the
// module inventing one.
await crier.perform({ ...base, params: { lines: 'hear ye' } })
assert.equal(calls.crier[2].durationSec, undefined)
const bad = await crier.perform({ ...base, params: { lines: 'hear ye', durationMinutes: 'soon' } })
assert.equal(bad.ok, false)
assert.equal(bad.retry, false)
})
// ── Giving it back ─────────────────────────────────────────────────────────
test('a resource that is already gone is a successful revert', async () => {
// §L: "gone, and that is fine". A crier line whose duration ran out is a 404,
// and it is the outcome teardown wanted.
uoLinkClient.deleteTownCrier = async () => ({ ok: false, status: 404 })
uoLinkClient.deleteNews = async () => ({ ok: false, status: 404 })
for (const id of ['uo.towncrier.post', 'uo.news.post']) {
const result = await byId(id).revert({ runId: 1, resources: [{ kind: 'x', ref: 'evt-1' }] })
assert.equal(result.ok, true)
assert.ok(!result.failed || !result.failed.length)
}
})
test('a revert names the resources that did not come back', async () => {
uoLinkClient.deleteTownCrier = async (id) => {
calls.crierDel.push(id)
return id === 'evt-bad' ? { ok: false, status: 503 } : { ok: true, status: 200 }
}
const result = await byId('uo.towncrier.post').revert({
runId: 1,
resources: [{ ref: 'evt-ok' }, { ref: 'evt-bad' }],
})
// `ok: true` with a `failed` list, not `ok: false`: the group was worked, and
// one member of it is outstanding. Core keeps the row and tries it again.
assert.equal(result.ok, true)
assert.deepEqual(result.failed, ['evt-bad'])
assert.deepEqual(calls.crierDel, ['evt-ok', 'evt-bad'], 'one failure must not stop the group')
})
// ── reconcile: the boot stamp ──────────────────────────────────────────────
test('a resource stamped with the current boot is still in force', async () => {
const resources = [
{ kind: 'towncrier', ref: 'evt-a', payload: { bootId: 'boot-1' } },
{ kind: 'towncrier', ref: 'evt-b', payload: { bootId: 'boot-0' } },
]
const result = await actions.reconcileByBootId({ resources })
assert.equal(result.ok, true)
// Only the row from the boot that is still running. Core orphans the other —
// which is the honest sentence: it vanished while nobody was looking, rather
// than core having put it back.
assert.deepEqual(result.inForce, ['evt-a'])
})
test('a resource with no stamp is reported in force, because "I do not know" is not "it is gone"', async () => {
const result = await actions.reconcileByBootId({
resources: [{ ref: 'evt-old', payload: null }, { ref: 'evt-older', payload: {} }],
})
assert.deepEqual(result.inForce, ['evt-old', 'evt-older'])
})
test('with no shard boot to compare against, reconcile declines rather than orphaning everything', async () => {
uoLinkConfig.getSafe = async () => ({ bootId: null })
const result = await actions.reconcileByBootId({ resources: [{ ref: 'evt-a', payload: { bootId: 'boot-1' } }] })
// Core treats anything that is not an explicit answer as unanswered and leaves
// the ledger alone. An `ok: true, inForce: []` here would abandon every live row
// on a website that came up before its sidecar did.
assert.equal(result.ok, false)
})
test('a write with an unreadable config still happens, and simply carries no stamp', async () => {
uoLinkConfig.getSafe = async () => { throw new Error('pool is down') }
const result = await byId('uo.towncrier.post').perform({
runId: 1,
idempotencyKey: 'k'.repeat(40),
params: { lines: 'hear ye' },
verify: false,
})
assert.equal(result.ok, true, 'a config read must not fail a world write')
assert.equal(result.resources[0].payload.bootId, null)
})
// ── Option sources ─────────────────────────────────────────────────────────
const source = (id) => actions.OPTION_SOURCES.find((s) => s.id === id)
test('every option source is namespaced and answers', () => {
for (const s of actions.OPTION_SOURCES) {
assert.ok(s.id.startsWith('uo.options.'), `${s.id} must be namespaced`)
assert.ok(s.label && s.description)
assert.equal(typeof s.resolve, 'function')
}
})
test('a place is named by its facet, because two facets both have a Britain', async () => {
shardAtlas.listRegions = async () => [
{ facet: 'Felucca', name: 'Britain' },
{ facet: 'Trammel', name: 'Britain' },
]
const options = await source('uo.options.regions').resolve()
assert.equal(new Set(options.map((o) => o.value)).size, 2, 'two different places must not share a value')
assert.deepEqual(options[0], { value: 'Felucca/Britain', label: 'Britain', group: 'Felucca' })
})
test('a landmark groups by the atlas grouping where it has one, the facet otherwise', async () => {
shardAtlas.listLandmarks = async () => [
{ facet: 'Felucca', name: 'Despise', group: 'Dungeons' },
{ facet: 'Felucca', name: 'Cove', group: null },
]
const options = await source('uo.options.landmarks').resolve()
assert.deepEqual(options.map((o) => o.group), ['Dungeons', 'Felucca'])
})
test('a creature option carries the type the shard can build, not the atlas slug', async () => {
// Changed in Phase 12a, and the reason is the point of the source existing.
// Wave 1 declared it before anything consumed it and used the slug — unique,
// stable, and unusable: the shard constructs from a ServUO class name, and
// `orc-brute` is not one. The atlas's `name` IS the raw type token from the
// spawn files, so the fix was to stop discarding the half that works.
shardAtlas.searchCreatures = async ({ limit }) => {
assert.equal(limit, actions.MAX_OPTIONS, 'the source must bound what it asks the atlas for')
return { creatures: [{ slug: 'orcbrute', name: 'OrcBrute' }] }
}
assert.deepEqual(await source('uo.options.creatures').resolve(), [
{ value: 'OrcBrute', label: 'OrcBrute' },
])
})
test('decoration options come from the shard\'s own decoration files', async () => {
const options = await source('uo.options.decor').resolve()
assert.deepEqual(options, [{ value: 'Brazier', label: 'Brazier' }])
})
test('an atlas larger than the dropdown bound is truncated and said so', async () => {
const { ctx } = require('./_setup')
shardAtlas.listRegions = async () =>
Array.from({ length: actions.MAX_OPTIONS + 5 }, (_, i) => ({ facet: 'Felucca', name: `Region ${i}` }))
const options = await source('uo.options.regions').resolve()
assert.equal(options.length, actions.MAX_OPTIONS)
// Silently serving 2000 of 2005 is the defect the bound would otherwise
// introduce: an author cannot find the landmark they are looking for and
// nothing anywhere says why.
const warned = ctx.logs
.filter((l) => l.namespace === 'uo-events')
.flatMap((l) => l.log.warn.calls)
.some(([message]) => /truncated/.test(message))
assert.ok(warned, 'a truncated source must leave a log line naming itself')
})
// ── A landmark option value names ONE landmark (Phase 16b) ────────────────
test('two landmarks sharing a name are two different options, and both resolve', async () => {
// A stock 57.4 tree has 558 landmarks under 320 distinct `facet/name` pairs:
// `Trammel/Entrance` is 23 different dungeons. The source emitted `facet/name`
// and `landmarkPoint` resolved with `.find()`, so 22 of the 23 were unreachable
// — an author who picked "Entrance — Destard" got Blighted Grove, with a
// successful run and no warning. The group was already the disambiguator and it
// was shown to the eye while being left out of the value.
//
// Asserted as an INEQUALITY between two resolved points rather than against a
// literal value string, so it survives someone changing the value's format
// again as long as the two options still address two places.
shardAtlas.listLandmarks = async () => [
{ facet: 'Felucca', name: 'Entrance', group: 'Blighted Grove', x: 586, y: 1643, z: 0 },
{ facet: 'Felucca', name: 'Entrance', group: 'Destard', x: 1176, y: 2637, z: 0 },
]
const source = actions.OPTION_SOURCES.find((s) => s.id === 'uo.options.landmarks')
const options = await source.resolve({})
assert.equal(options.length, 2)
assert.equal(new Set(options.map((o) => o.value)).size, 2, 'both options must be addressable')
const points = []
for (const option of options) {
const result = await byId('uo.creature.spawn').perform({
runId: 41,
idempotencyKey: `L${option.value}`.padEnd(40, 'x'),
params: { place: option.value, creature: 'Orc', count: 1 },
verify: true,
})
assert.equal(result.ok, true, `${option.value} must resolve`)
points.push(option.value)
}
assert.notEqual(points[0], points[1])
})
test('a place published before the group was carried still resolves', async () => {
// Every event published before the fix stores `facet/name`, and a published
// version is immutable — so a parse that stopped understanding the two-part
// form would break those runs rather than correct them. It keeps the old
// first-match read, which is imprecise in exactly the way it always was.
shardAtlas.listLandmarks = async () => [
{ facet: 'Felucca', name: 'Entrance', group: 'Blighted Grove', x: 586, y: 1643, z: 0 },
{ facet: 'Felucca', name: 'Entrance', group: 'Destard', x: 1176, y: 2637, z: 0 },
// A name carrying a slash reads as three parts too; the two-part read is what
// resolves it, which is why the three-part attempt must not answer for it.
{ facet: 'Felucca', name: 'Odd/Name', group: null, x: 10, y: 20, z: 0 },
]
for (const place of ['Felucca/Entrance', 'Felucca/Odd/Name']) {
const result = await byId('uo.creature.spawn').perform({
runId: 42,
idempotencyKey: `P${place}`.padEnd(40, 'x'),
params: { place, creature: 'Orc', count: 1 },
verify: true,
})
assert.equal(result.ok, true, `${place} must still resolve`)
}
// And a three-part value whose group is gone REFUSES rather than silently
// landing somewhere else. That is the honest answer: it asked for one place.
const gone = await byId('uo.creature.spawn').perform({
runId: 42,
idempotencyKey: 'G'.repeat(40),
params: { place: 'Felucca/Renamed/Entrance', creature: 'Orc', count: 1 },
verify: true,
})
assert.equal(gone.ok, false)
assert.match(gone.error, /no landmark called/)
})
// ── The world verbs (Phase 12a) ───────────────────────────────
test('a spawn files one ledger row per serial, not one per call', async () => {
// Per serial, because a group half of which a player killed has to reconcile
// per creature. One row per call would make teardown all-or-nothing over eight
// orcs of which six are gone, which is neither true nor useful.
const result = await byId('uo.creature.spawn').perform({
runId: 7,
idempotencyKey: 'c'.repeat(40),
params: { place: 'Felucca/Britain', creature: 'Orc', count: 3 },
verify: false,
})
assert.equal(result.ok, true)
assert.equal(result.resources.length, 3)
for (const resource of result.resources) {
assert.equal(resource.kind, actions.OWNED_KIND)
assert.equal(resource.payload.runId, '7')
assert.equal(resource.payload.what, 'creature')
assert.equal(resource.payload.type, 'Orc')
}
// The place is resolved to a point HERE, so the shard is never handed a
// facet/name it would have to know how to read.
assert.equal(calls.spawn.length, 1)
assert.deepEqual(
{ map: calls.spawn[0].map, x: calls.spawn[0].x, y: calls.spawn[0].y },
{ map: 'Felucca', x: 1496, y: 1628 },
)
})
test('a boss is a creature plus multipliers, and is refused above the ceiling', async () => {
const boss = byId('uo.boss.spawn')
const params = {
place: 'Felucca/Britain',
creature: 'OrcCaptain',
name: 'Gruk the Unbroken',
hitsMultiplier: 3,
damageMultiplier: 1.5,
}
assert.equal((await boss.perform({ runId: 7, idempotencyKey: 'b'.repeat(40), params, verify: false })).ok, true)
assert.equal(calls.spawn[0].what, 'boss')
assert.equal(calls.spawn[0].hitsMultiplier, 3)
assert.equal(calls.spawn[0].damageMultiplier, 1.5)
// Absent, not zero: a multiplier nobody set must not arrive as a number the
// shard would then apply.
assert.equal(calls.spawn[0].statMultiplier, undefined)
const tooMuch = await boss.perform({
runId: 7,
idempotencyKey: 'b'.repeat(40),
params: { ...params, hitsMultiplier: actions.MAX_BOSS_MULTIPLIER + 1 },
verify: false,
})
assert.equal(tooMuch.ok, false)
assert.equal(tooMuch.retry, false, 'a ceiling will not move on a retry')
assert.equal(calls.spawn.length, 1, 'nothing may reach the shard once it is refused here')
// Named, because an unnamed boss is just a hard orc — and because the name is
// what an operator reads in the ledger afterwards.
const unnamed = await boss.perform({
runId: 7,
idempotencyKey: 'b'.repeat(40),
params: { ...params, name: ' ' },
verify: false,
})
assert.equal(unnamed.ok, false)
})
test('an oracle\'s dialogue is parsed from one textarea, and a bad row is named', async () => {
const parsed = actions.oracleLines('fire, flame = It burns beneath the keep.\n gate = At dusk. ')
assert.deepEqual(parsed, {
ok: true,
rows: [
{ keywords: 'fire,flame', text: 'It burns beneath the keep.' },
{ keywords: 'gate', text: 'At dusk.' },
],
})
// Split on the FIRST `=`, so an answer may contain one.
assert.deepEqual(actions.oracleLines('sum = 2 = 2 is four').rows, [
{ keywords: 'sum', text: '2 = 2 is four' },
])
assert.equal(actions.oracleLines('just some prose').ok, false)
assert.equal(actions.oracleLines('fire =').ok, false, 'a keyword with nothing to say is a mistake')
assert.equal(actions.oracleLines('= something').ok, false, 'something to say with no keyword is too')
const tooMany = actions.oracleLines(
Array.from({ length: actions.MAX_ORACLE_LINES + 1 }, (_, i) => `w${i} = t${i}`).join('\n'),
)
assert.equal(tooMany.ok, false)
})
test('an oracle with nothing to say is refused before it is stood up', async () => {
// `required: true` on the greeting catches an ABSENT field, at the edge, and
// this catches the one holding nothing but spaces — which reaches `perform`
// looking exactly like a filled-in form.
const result = await byId('uo.npc.place').perform({
runId: 7,
idempotencyKey: 'n'.repeat(40),
params: { place: 'Felucca/Britain', name: 'Marisa', greeting: ' ' },
verify: false,
})
assert.equal(result.ok, false)
assert.equal(result.retry, false)
assert.match(result.error, /silence/)
assert.deepEqual(calls.spawn, [])
})
test('a keyword line reaches the shard as keywords and text, and nothing executable', async () => {
// The whole argument for not building this on `XmlSpawner2.XmlDialog`, which
// implements exactly this vocabulary and one field more: an `Action` string
// that runs commands. What crosses here is what an oracle SAYS.
const result = await byId('uo.npc.place').perform({
runId: 7,
idempotencyKey: 'n'.repeat(40),
params: {
place: 'Felucca/Britain',
name: 'Marisa',
greeting: 'You have questions.',
lines: 'fire, flame = It burns beneath the keep.',
sex: 'female',
},
verify: false,
})
assert.equal(result.ok, true)
assert.deepEqual(calls.spawn[0].lines, [
{ keywords: 'fire,flame', text: 'It burns beneath the keep.' },
])
assert.equal(calls.spawn[0].sex, 'female')
for (const key of Object.keys(calls.spawn[0])) {
assert.notEqual(key, 'action', 'nothing executable may cross to the shard')
}
})
test('a gate crosses as a DURATION, and names both ends as points', async () => {
const result = await byId('uo.gate.open').perform({
runId: 7,
idempotencyKey: 'g'.repeat(40),
params: { place: 'Felucca/Britain', destination: 'Felucca/Yew', durationMinutes: 120 },
verify: false,
})
assert.equal(result.ok, true)
const sent = calls.spawn[0]
// A duration, never an absolute time: an absolute deadline computed here and
// honoured there is measured against two clocks, and a shard ten minutes fast
// would collect the gate the instant it opened.
assert.equal(sent.holdMs, 120 * 60_000)
assert.equal(sent.untilMs, undefined, 'an absolute deadline must not cross')
assert.deepEqual(sent.target, { map: 'Felucca', x: 542, y: 982 })
const tooLong = await byId('uo.gate.open').perform({
runId: 7,
idempotencyKey: 'g'.repeat(40),
params: {
place: 'Felucca/Britain',
destination: 'Felucca/Yew',
durationMinutes: actions.MAX_GATE_MINUTES + 1,
},
verify: false,
})
assert.equal(tooLong.ok, false)
assert.equal(tooLong.retry, false)
})
test('teardown reports a refused serial as failed, and a killed creature as done', async () => {
const resources = [
{ kind: 'world', ref: '0x40000000', payload: {} },
{ kind: 'world', ref: '0x40000001', payload: {} },
]
// `gone` is not a failure. A creature a player killed is the point of having
// spawned it, and §L already says "gone, and that is fine" is a successful
// revert — so a run does not end `incomplete` because its event worked.
uoLinkClient.despawnWorld = async () => ({
ok: true,
status: 200,
data: { removed: ['0x40000000'], gone: ['0x40000001'], refused: [] },
})
assert.deepEqual(await actions.revertOwned({ runId: 7, resources }), { ok: true })
// `refused` IS. The shard denies this run ever owned it, so nothing will ever
// delete it through this path: the row must land unresolved with a reason
// rather than be quietly marked reverted.
uoLinkClient.despawnWorld = async () => ({
ok: true,
status: 200,
data: { removed: ['0x40000000'], gone: [], refused: ['0x40000001'] },
})
assert.deepEqual(await actions.revertOwned({ runId: 7, resources }), {
ok: true,
failed: ['0x40000001'],
})
// An unreachable shard has not said anything about anything.
uoLinkClient.despawnWorld = async () => ({ ok: false, status: 503, data: null })
assert.equal((await actions.revertOwned({ runId: 7, resources })).ok, false)
})
test('the despawn carries NO idempotency key, whatever core hands revert()', async () => {
// The Phase 16 acceptance walk's critical finding, as the test that would have
// caught it. `revertOwned` used to forward core's `idempotencyKey` onto the
// despawn — and core's key is the STEP's, the one `placeOwned` spawned under.
// The shard's at-most-once store is keyed on the key ALONE
// (`BridgeIdempotency.Intercept` does `_byKey.TryGetValue(key, …)`, with no
// reference to which command carried it), so the despawn was taken for a repeat
// and answered with the SPAWN's stored reply. `OnDespawn` never ran. Core read
// `ok` with no `refused` and marked every row `reverted` while the shard still
// held every object — teardown of all five world verbs was a no-op that
// reported success.
//
// Every other stub in this file ignores the body, which is why the suite was
// green throughout. This one asserts on the body, and it asserts ABSENCE — the
// property that matters — rather than pinning the rest of the shape.
let sent = null
uoLinkClient.despawnWorld = async (body) => {
sent = body
return { ok: true, status: 200, data: { removed: ['0x40000000'], gone: [], refused: [] } }
}
await actions.revertOwned({
runId: 7,
resources: [{ kind: 'world', ref: '0x40000000', payload: {} }],
// Core passes this on every call (MODULE_API.md), and it must not reach the wire.
idempotencyKey: 'the-step-key-the-spawn-went-out-under',
})
assert.ok(sent, 'despawnWorld was not called')
assert.equal(
Object.prototype.hasOwnProperty.call(sent, 'idempotencyKey'),
false,
'the despawn must not carry an idempotency key — the shard would replay the spawn',
)
// MODULE_API.md: revert is sometimes called with the key and an EMPTY list,
// meaning "a command went out under this key and core never learned what it
// did". No serials is the shard's own idiom for "everything this run owns",
// which is the correct sweep for exactly that case.
sent = null
await actions.revertOwned({ runId: 7, resources: [], idempotencyKey: 'lost-dispatch' })
assert.deepEqual(sent.serials, [])
assert.equal(Object.prototype.hasOwnProperty.call(sent, 'idempotencyKey'), false)
})
test('reconcile ASKS the shard, because these resources survive a restart', async () => {
// The one property that separates this from every other resource in the file.
// A crier line lives in shard memory, so a changed `bootId` IS proof it is
// gone; a spawned creature is in the world SAVE and survives the restart the
// boot stamp would report it lost by.
const resources = [
{ kind: 'world', ref: '0x40000000', payload: {} },
{ kind: 'world', ref: '0x40000001', payload: {} },
]
assert.deepEqual(await actions.reconcileOwned({ runId: 7, resources }), {
ok: true,
inForce: ['0x40000000'],
})
assert.deepEqual(calls.owned, [{ runId: '7' }])
// "I could not ask" must never be read as "it is gone": an unanswered group
// leaves every row alone rather than orphaning the lot.
uoLinkClient.ownedWorld = async () => ({ ok: false, status: 504, data: null })
assert.equal((await actions.reconcileOwned({ runId: 7, resources })).ok, false)
})
test('every world verb declares the same undo contract', async () => {
// Five declarations sharing one spread object, asserted rather than assumed:
// a verb that quietly lost its `reconcile` would leave its rows unanswered for
// the life of the run, and nothing would report it — which is exactly the hole
// Phase 11b found in `core.lease`.
for (const id of ['uo.creature.spawn', 'uo.boss.spawn', 'uo.npc.place', 'uo.gate.open', 'uo.decor.place']) {
const action = byId(id)
assert.equal(action.risk, 'change', `${id} must be a world change`)
assert.equal(action.reversible, 'ledger', `${id} owns what it made`)
assert.equal(typeof action.revert, 'function', `${id} has no undo`)
assert.equal(typeof action.reconcile, 'function', `${id} can never be asked what it still holds`)
assert.ok(action.budgetMs > 12000, `${id} must outlast the client's own timeout`)
assert.equal(typeof action.cost, 'function', `${id} is capped by nothing`)
}
})
test('decoration carries the graphic, and a type this shard never decorates with is refused', async () => {
const decor = byId('uo.decor.place')
const ok = await decor.perform({
runId: 7,
idempotencyKey: 'd'.repeat(40),
params: { place: 'Felucca/Britain', item: 'Brazier', count: 2 },
verify: false,
})
assert.equal(ok.ok, true)
assert.equal(ok.resources.length, 2)
// **The item id crosses, and it has to.** Measured on ServUO 57.4, `Static`
// accounts for 5031 decoration placements under 1992 DIFFERENT graphics,
// because for that class the graphic is the identity: a bare `new Static()`
// is never the paving stone the author picked. 131 of 313 types carry more
// than one id.
assert.equal(calls.spawn[0].type, 'Brazier')
assert.equal(calls.spawn[0].itemId, 0x0e31)
// Resolving through the atlas is also the boundary: the verb places what this
// shard's own decoration files name, which is tighter than "any item that is
// not a container" and is the rule the decision actually took.
const unknown = await decor.perform({
runId: 7,
idempotencyKey: 'd'.repeat(40),
params: { place: 'Felucca/Britain', item: 'BlackrockCrate', count: 1 },
verify: false,
})
assert.equal(unknown.ok, false)
assert.equal(unknown.retry, false)
assert.match(unknown.error, /never mention/)
assert.equal(calls.spawn.length, 1)
})

View File

@@ -0,0 +1,349 @@
// module-uo's half of protocol 7 part b (EVENTS_PLAN.md Phase 12b).
//
// What an event BORROWS — five targeted leases over two planes — and the two
// one-shots that are neither borrowed nor owned.
//
// The tests below are the places where the obvious implementation is subtly the
// wrong one and nothing would fail if it were written the other way:
//
// • every callable of a targeted lease must PASS THE TARGET ON. A read that
// dropped it would answer about the wrong spawner, and a restore that
// dropped it would write a baseline onto one
// • a target the shard can no longer read is a REFUSAL at apply time, never a
// value: taking the lease anyway records a fictional baseline and later
// writes it onto whatever next holds that id
// • a target that vanished mid-run is a SUCCESSFUL restore, not a failure —
// there is nothing to give back, and reporting it failed leaves a ledger row
// unresolved for ever over an object that is gone
// • `inForce()` reads the frame's `holds`, which is the only thing that can
// answer for a targeted key: there is no list of spawners to walk
// • a grant that reached NOBODY is a success, because an event nobody attended
// still happened — while a run the shard was never told to count is a 404
// • a non-stackable granted in quantity is refused at BOTH ends
const { test, beforeEach, afterEach } = require('node:test')
const assert = require('node:assert/strict')
const uoLinkClient = require('../utils/uoLinkClient')
const shardAtlas = require('../model/shardAtlas/shardAtlas.model')
require('./_setup')
const actions = require('../config/uoEventActions')
const byId = (id) => actions.ACTIONS.find((a) => a.id === id)
const leaseById = (id) => actions.LEASES.find((l) => l.id === id)
const STUBBED = ['getLeases', 'applyLease', 'releaseLease', 'grantItem', 'saveWorld']
let calls
let frame
const saved = {}
beforeEach(() => {
calls = { leases: [], apply: [], release: [], grant: [], save: [] }
frame = {
leases: [{ key: 'Spawner.MaxCount', kind: 'property', current: '3', held: false }],
holds: [],
}
for (const name of STUBBED) saved[name] = uoLinkClient[name]
saved.listSpawners = shardAtlas.listSpawners
uoLinkClient.getLeases = async (q) => {
calls.leases.push(q)
return { ok: true, status: 200, data: frame }
}
uoLinkClient.applyLease = async (b) => { calls.apply.push(b); return { ok: true, status: 200, data: {} } }
uoLinkClient.releaseLease = async (b) => { calls.release.push(b); return { ok: true, status: 200, data: {} } }
uoLinkClient.grantItem = async (b) => {
calls.grant.push(b)
return { ok: true, status: 200, data: { granted: 2, missed: [] } }
}
uoLinkClient.saveWorld = async (b) => { calls.save.push(b); return { ok: true, status: 200, data: {} } }
shardAtlas.listSpawners = async (opts) => {
calls.spawners = opts
return [
{ uniqueId: 'uid-1', name: 'fel orc fort', facet: 'Felucca', region: 'Britain', maxCount: 9 },
{ uniqueId: 'uid-2', name: null, facet: 'Trammel', region: null, landmark: null, maxCount: 1 },
]
}
})
afterEach(() => {
for (const name of STUBBED) uoLinkClient[name] = saved[name]
shardAtlas.listSpawners = saved.listSpawners
})
// ── The targeted leases ────────────────────────────────────────────────────
test('every callable carries the target through to the shard', async () => {
// The one thing that cannot be got wrong quietly. Core composes the ledger ref
// as `<lease id>#<target>` and hands the target back on every call; a callable
// that ignored it would read, apply to and restore whichever spawner the shard
// happened to answer about, and nothing here or there would report an error.
const lease = leaseById('uo.spawner.maxcount')
const target = '003f11b8-9bfa-4587-991e-ca263004efe6'
const read = await lease.read({ target })
assert.deepEqual(read, { ok: true, value: '3' })
assert.deepEqual(calls.leases[0], { key: 'Spawner.MaxCount', target })
await lease.apply('30', new Date(Date.now() + 600_000), { target })
assert.equal(calls.apply[0].key, 'Spawner.MaxCount')
assert.equal(calls.apply[0].target, target)
// A DURATION, not the deadline — 11b's rule, unchanged by targeting. A shard
// whose clock runs fast would restore an absolute deadline the instant it
// took it.
assert.ok(calls.apply[0].holdMs > 0 && calls.apply[0].holdMs <= 600_000)
await lease.restore('3', { expected: '30', target })
assert.deepEqual(calls.release[0], {
key: 'Spawner.MaxCount',
target,
expected: '30',
baseline: '3',
})
})
test('a target the shard cannot read refuses the lease rather than defaulting', async () => {
// The failure this guards is silent and permanent: a lease taken over a
// spawner that is not there records whatever came back as the baseline, and
// teardown then WRITES that baseline onto whatever next holds the id.
frame.leases = [{ key: 'Spawner.MaxCount', unreadable: "nothing on this shard has serial 0x99" }]
const refused = await leaseById('uo.spawner.maxcount').read({ target: '0x99' })
assert.equal(refused.ok, false)
assert.match(refused.error, /nothing on this shard has serial/)
// A row with neither a value nor a reason is refused too. The shard should
// always send one of them, and "it sent neither" must not read as zero.
frame.leases = [{ key: 'Spawner.MaxCount' }]
const empty = await leaseById('uo.spawner.maxcount').read({ target: 'uid-1' })
assert.equal(empty.ok, false)
assert.match(empty.error, /could not read/)
})
test('a target that vanished mid-run is a successful restore, not a failure', async () => {
// 12a's `gone` in the lease plane's vocabulary. Somebody deleted the spawner
// while the run held it: there is nothing to give back and nothing is owed.
// Reported as a failure it would sit in the ledger unresolved for ever, over
// an object that no longer exists — and every sweep would try again.
uoLinkClient.releaseLease = async () => ({
ok: true,
status: 200,
data: { kind: 'lease.ok', released: true, targetGone: true, reason: 'that object has been deleted' },
})
const done = await leaseById('uo.spawner.maxcount').restore('3', { expected: '30', target: 'uid-1' })
assert.deepEqual(done, { ok: true })
})
test('drift is still drift, and is still not an error', async () => {
// Unchanged from 11b and asserted again because targeting rewrote the whole
// callable: core records drift as a distinct SUCCESSFUL outcome, so an error
// here would put the row on the retry ladder and eventually report the lease
// as vanished rather than as somebody having moved it.
uoLinkClient.releaseLease = async () => ({
ok: true,
status: 200,
data: { kind: 'lease.drifted', current: '12' },
})
const drifted = await leaseById('uo.spawner.maxcount').restore('3', { expected: '30', target: 'uid-1' })
assert.deepEqual(drifted, { ok: false, drifted: true, current: '12' })
})
test('inForce reads the holds list, which is the only thing that can answer', async () => {
// A catalog walk can enumerate the KEYS but never the holds on a targeted one
// — there is no list of spawners to walk — so the frame carries every hold the
// shard has, and this is what reads it.
const lease = leaseById('uo.spawner.maxcount')
assert.deepEqual(await lease.inForce({ target: 'uid-1' }), { ok: true, held: false })
frame.holds = [{ key: 'Spawner.MaxCount', target: 'uid-1', runId: '7' }]
assert.deepEqual(await lease.inForce({ target: 'uid-1' }), { ok: true, held: true })
// ...and it is the hold on THIS target, not any hold on the key. A run holding
// one spawner must not make every other spawner look leased.
assert.deepEqual(await lease.inForce({ target: 'uid-2' }), { ok: true, held: false })
})
test('a shard that cannot answer is never read as "the lease is gone"', async () => {
// Core's posture everywhere: "I could not ask" must not be recorded as "it is
// gone", because the second orphans the row and stops teardown ever trying.
uoLinkClient.getLeases = async () => ({ ok: false, status: 503, data: null })
const answer = await leaseById('uo.spawner.maxcount').inForce({ target: 'uid-1' })
assert.equal(answer.ok, false)
})
test('the seasonal lease is a three-value enum over eight events', () => {
// §G called `SeasonalEventSystem.GetEntry(type).Status` "a nine-value enum" and
// had it backwards: `EventStatus` has three values, `EventType` has nine
// entries — and one of those nine is excluded, so it is eight.
const lease = leaseById('uo.seasonal.status')
assert.equal(lease.type, 'string')
assert.deepEqual(lease.values, ['Inactive', 'Active', 'Seasonal'])
assert.equal(actions.SEASONAL_EVENTS.length, 8)
// TreasuresOfTokuno reads its own era rather than this status, so leasing it
// would apply cleanly and change nothing — §N10's "a capability that lies",
// and the one instance no runtime probe can catch.
assert.ok(!actions.SEASONAL_EVENTS.includes('TreasuresOfTokuno'))
})
test('every targeted lease bounds what it can hold', () => {
// §F requires a range on the numeric types because, unlike a cap, a bad lease
// value is in force the moment it is applied. Restated over the five because
// they are built by a shared factory: one missing bound would be missing in a
// way no single declaration shows.
for (const lease of actions.LEASES) {
if (lease.id === 'uo.playercaps.skillcap') continue
assert.ok(lease.maxDurationMs > 0, `${lease.id} has no duration bound`)
if (lease.type === 'int' || lease.type === 'float') {
assert.ok(Number.isFinite(lease.min) && Number.isFinite(lease.max), `${lease.id} has no range`)
assert.ok(lease.min <= lease.max, `${lease.id} has min above max`)
}
if (lease.type === 'string') {
assert.ok(Array.isArray(lease.values) && lease.values.length, `${lease.id} has no value set`)
}
}
})
// ── The spawner source ─────────────────────────────────────────────────────
test('the spawner source searches, and says so', async () => {
// The first source with more entries than a dropdown holds: 6,707 spawn points
// against MAX_OPTIONS' 2,000. A flat list would drop two thirds of the world
// and say nothing about which two thirds.
const source = actions.OPTION_SOURCES.find((s) => s.id === 'uo.options.spawners')
assert.equal(source.searchable, true)
const rows = await source.resolve({ q: 'orc' })
assert.equal(calls.spawners.q, 'orc')
assert.equal(calls.spawners.limit, actions.SPAWNER_OPTIONS)
// The value is the UniqueId, because it is the only name for one particular
// spawner that exists off the shard.
assert.deepEqual(rows[0], { value: 'uid-1', label: 'fel orc fort', group: 'Britain' })
// A nameless spawner still answers, labelled by its id. It is still a spawner
// somebody may need to turn down, and dropping it would be a dropdown quietly
// missing rows again.
assert.deepEqual(rows[1], { value: 'uid-2', label: 'uid-2', group: 'Trammel' })
})
// ── The one-shots ──────────────────────────────────────────────────────────
test('a grant sends a run and never a recipient list', async () => {
// The shard has held this run's participation ledger since it opened, keyed by
// the same serials core stores as `member_key`. Sending a list would put it on
// the wire twice with a window in which the two disagree — and would have
// needed a core surface handing a module core's own participants.
const out = await byId('uo.item.grant').perform({
runId: 7,
idempotencyKey: 'k',
params: { item: 'gold', amount: 500, where: 'bank' },
})
assert.equal(out.ok, true)
assert.deepEqual(calls.grant[0], {
runId: 7,
item: 'gold',
amount: 500,
hue: undefined,
name: undefined,
where: 'bank',
idempotencyKey: 'k',
})
assert.equal(out.detail.granted, 2)
})
test('a grant that reached nobody is a success', async () => {
// An event nobody attended still happened. Reported as a failure the run would
// retry against a ledger that will be just as empty next time, and pause. The
// shard draws the distinction that matters: a run it was never told to count
// is a 404, which fails below.
uoLinkClient.grantItem = async () => ({ ok: true, status: 200, data: { granted: 0, missed: [] } })
const out = await byId('uo.item.grant').perform({
runId: 7,
idempotencyKey: 'k',
params: { item: 'gold', amount: 1 },
})
assert.equal(out.ok, true)
assert.equal(out.detail.granted, 0)
uoLinkClient.grantItem = async () => ({
ok: false,
status: 404,
data: { reason: 'run 7 has no participation ledger open on this shard' },
})
const missing = await byId('uo.item.grant').perform({
runId: 7,
idempotencyKey: 'k',
params: { item: 'gold', amount: 1 },
})
assert.equal(missing.ok, false)
// 404 is permanent: the ledger will not appear because we asked again.
assert.equal(missing.retry, false)
})
test('a non-stackable granted in quantity is refused before the wire', async () => {
// Five cloaks would be five items — five chances to overflow a backpack
// halfway through with no way to say which half landed. Refused here so the
// author sees it on the form, and refused again on the shard because this copy
// of the allowlist is the one that can be wrong.
const out = await byId('uo.item.grant').perform({
runId: 7,
idempotencyKey: 'k',
params: { item: 'cloak', amount: 3 },
})
assert.equal(out.ok, false)
assert.equal(out.retry, false)
assert.match(out.error, /does not stack/)
assert.equal(calls.grant.length, 0)
const unknown = await byId('uo.item.grant').perform({
runId: 7,
idempotencyKey: 'k',
params: { item: 'castle', amount: 1 },
})
assert.equal(unknown.ok, false)
assert.equal(unknown.retry, false)
assert.equal(calls.grant.length, 0)
})
test('a grant is retryable, and protocol 6 is the reason', async () => {
// §G called a grant un-retryable because a lost acknowledgement and a grant
// that never applied were the same event — the argument that made
// `uo.broadcast` answer `retry: false` in Phase 9. An idempotency key closes
// it: a repeat is answered by the original reply, so a retried grant cannot be
// one winner receiving two.
uoLinkClient.grantItem = async () => ({ ok: false, status: 503, data: null })
const out = await byId('uo.item.grant').perform({
runId: 7,
idempotencyKey: 'k',
params: { item: 'gold', amount: 1 },
})
assert.equal(out.ok, false)
assert.notEqual(out.retry, false)
// And the action declares itself irreversible, which is the honest class: the
// world is altered and cannot be put back.
assert.equal(byId('uo.item.grant').risk, 'irreversible')
assert.equal(byId('uo.item.grant').reversible, 'none')
})
test('a save refused for coming too soon is retried, not abandoned', async () => {
// 429 is the shard's rate limit and is the one refusal on this plane that
// waiting fixes. It is deliberately not in PERMANENT_STATUSES, so a phase
// boundary is retried rather than dropped.
assert.ok(!actions.PERMANENT_STATUSES.has(429))
uoLinkClient.saveWorld = async () => ({
ok: false,
status: 429,
data: { reason: 'this shard saves at most every 300 seconds, and the last save was 12 seconds ago' },
})
const out = await byId('uo.world.save').perform({ idempotencyKey: 'k' })
assert.equal(out.ok, false)
assert.notEqual(out.retry, false)
})
test('a save reports only that it started', async () => {
// What actually happened rides `world.save.before`/`after` on the event stream.
// Asserting anything more here would be asserting something the reply does not
// know.
const out = await byId('uo.world.save').perform({ idempotencyKey: 'k' })
assert.deepEqual(out, { ok: true, detail: { started: true } })
assert.deepEqual(calls.save[0], { idempotencyKey: 'k' })
})

View File

@@ -0,0 +1,382 @@
// module-uo's half of protocol 6 part b (EVENTS_PLAN.md Phase 11b).
//
// One lease and two participation verbs. What is worth asserting here is not that
// the calls happen — a rig proves that better — but the handful of places where
// the obvious implementation is subtly the wrong one, and where nothing would fail
// if it were written the other way:
//
// • a lease's `restore()` must turn `lease.drifted` into `{ drifted: true }`
// rather than an error, because core records drift as a distinct SUCCESSFUL
// outcome and an error would put the row on the retry ladder instead
// • `inForce()` must not be a comparison against `read()` — a changed value is
// drift, which teardown reports, and orphaning the row first destroys it
// • `apply()` must send a DURATION, not the deadline, or a shard whose clock is
// fast restores the lease the instant it takes it
// • `uo.participation.open` must NOT reconcile by boot stamp, which every other
// resource in this module does — the ledger is persisted in the world save
// precisely so that it survives the restart the stamp would report it lost by
// • a `userId` is a foreign key and a character serial is not, so an unresolved
// one is undefined rather than coerced
const { test, beforeEach, afterEach } = require('node:test')
const assert = require('node:assert/strict')
const uoLinkClient = require('../utils/uoLinkClient')
const shardAtlas = require('../model/shardAtlas/shardAtlas.model')
require('./_setup')
const actions = require('../config/uoEventActions')
const byId = (id) => actions.ACTIONS.find((a) => a.id === id)
const lease = () => actions.LEASES.find((l) => l.id === 'uo.playercaps.skillcap')
const STUBBED = [
'getLeases',
'applyLease',
'releaseLease',
'openParticipation',
'snapshotParticipation',
'closeParticipation',
]
let calls
const saved = {}
beforeEach(() => {
calls = { apply: [], release: [], open: [], snapshot: [], close: [] }
for (const name of STUBBED) saved[name] = uoLinkClient[name]
saved.listLandmarks = shardAtlas.listLandmarks
uoLinkClient.getLeases = async () => ({
ok: true,
status: 200,
data: { leases: [{ key: 'PlayerCaps.SkillCap', current: '1000', held: false }] },
})
uoLinkClient.applyLease = async (b) => { calls.apply.push(b); return { ok: true, status: 200, data: {} } }
uoLinkClient.releaseLease = async (b) => { calls.release.push(b); return { ok: true, status: 200, data: {} } }
uoLinkClient.openParticipation = async (b) => { calls.open.push(b); return { ok: true, status: 200, data: {} } }
uoLinkClient.snapshotParticipation = async (b) => {
calls.snapshot.push(b)
return { ok: true, status: 200, data: { participants: [] } }
}
uoLinkClient.closeParticipation = async (b) => { calls.close.push(b); return { ok: true, status: 200, data: {} } }
shardAtlas.listLandmarks = async () => [{ facet: 'Felucca', name: 'Britain', x: 1496, y: 1628, z: 10 }]
})
afterEach(() => {
for (const name of STUBBED) uoLinkClient[name] = saved[name]
shardAtlas.listLandmarks = saved.listLandmarks
})
// ── The lease ──────────────────────────────────────────────────────────────
test('the lease satisfies the shape core validates it with', () => {
const l = lease()
assert.ok(l.id.startsWith('uo.'), 'a lease is namespaced to its module')
assert.ok(l.label && l.description)
assert.equal(l.type, 'float')
// Required for the numeric types, and unlike a cap a bad lease value is in
// force the moment it is applied.
assert.ok(Number.isFinite(l.min) && Number.isFinite(l.max) && l.min < l.max)
assert.ok(Number.isInteger(l.maxDurationMs) && l.maxDurationMs > 0)
for (const fn of ['read', 'apply', 'restore', 'inForce']) {
assert.equal(typeof l[fn], 'function', `a lease needs ${fn}()`)
}
})
test('apply sends a DURATION, because a deadline is measured against two clocks', async () => {
const until = new Date(Date.now() + 90 * 60_000)
const answer = await lease().apply(1200, until)
assert.equal(answer.ok, true)
const sent = calls.apply[0]
// The number the shard arms its timer off. Computed here from the deadline, so
// a shard running ten minutes fast holds the lease for ninety minutes of its
// own time rather than restoring it the instant it takes it.
assert.ok(Math.abs(sent.holdMs - 90 * 60_000) < 2000, `holdMs was ${sent.holdMs}`)
// And the absolute time still rides along, for a console that wants to say when
// the hold ends in terms the operator's own clock agrees with.
assert.equal(sent.untilMs, until.getTime())
// The action hands the value on unchanged; `uoLinkClient.applyLease` is what
// renders it as TEXT, which is the wire's contract for every lease type: `1200`
// and `1200.0` are one number to a JSON parser and two different strings to a
// compare-and-set.
assert.equal(sent.value, 1200)
})
test('a deadline that has already passed is refused rather than sent as a negative hold', async () => {
const answer = await lease().apply(1200, new Date(Date.now() - 60_000))
assert.equal(answer.ok, false)
assert.match(answer.error, /already passed/)
assert.equal(calls.apply.length, 0)
})
test('drift comes back as drifted, not as an error', async () => {
// The distinction core acts on. `cleanup.js` records `drifted` as its own
// outcome — the module did exactly what it was asked and found somebody else's
// value in place — while an error would put the row on the retry ladder and
// eventually spend its attempts on a situation only a human can resolve.
uoLinkClient.releaseLease = async () => ({
ok: true,
status: 200,
data: { kind: 'lease.drifted', key: 'PlayerCaps.SkillCap', current: '1300' },
})
const answer = await lease().restore('1000', { expected: '1200' })
assert.equal(answer.ok, false)
assert.equal(answer.drifted, true)
assert.equal(answer.current, '1300')
assert.equal(answer.error, undefined)
})
test('restore sends both what it applied and what to put back', async () => {
await lease().restore('1000', { expected: '1200' })
// Core's `restore(baseline, { expected })` carries no key of its own -- teardown
// is core's own sweep rather than a step dispatch -- so neither does this.
assert.deepEqual(calls.release[0], {
key: 'PlayerCaps.SkillCap',
expected: '1200',
baseline: '1000',
})
})
test('inForce asks whether the shard still HOLDS it, not whether the value still matches', async () => {
// The reason this callable exists at all. A shard reporting a value that is not
// what the run applied is reporting DRIFT, which teardown delivers through
// `restore()` so the ledger row lands `drifted` with the current value beside
// it. Answering "not in force" here would orphan the row first and tell the
// operator the lease vanished rather than that somebody moved it.
uoLinkClient.getLeases = async () => ({
ok: true,
status: 200,
data: { leases: [{ key: 'PlayerCaps.SkillCap', current: '1300', held: true }] },
})
assert.deepEqual(await lease().inForce(), { ok: true, held: true })
// And a shard that restarted: a config lease is memory-only there by design, so
// the value is back at baseline AND the record is gone. This is the case core
// could not see before this phase.
uoLinkClient.getLeases = async () => ({
ok: true,
status: 200,
data: { leases: [{ key: 'PlayerCaps.SkillCap', current: '1000', held: false }] },
})
assert.deepEqual(await lease().inForce(), { ok: true, held: false })
})
test('a shard that cannot answer leaves the ledger alone', async () => {
uoLinkClient.getLeases = async () => ({ ok: false, status: 503, error: 'shard not connected' })
const answer = await lease().inForce()
assert.equal(answer.ok, false)
// `ok: false` is what core reads as "I could not ask", and it keeps believing
// its own ledger. Never `held: false`, which would orphan a live lease the
// first time a sidecar was slow.
assert.equal(answer.held, undefined)
assert.equal((await lease().read()).ok, false)
})
// ── Participation ──────────────────────────────────────────────────────────
test('open resolves a named place to the point the shard counts around', async () => {
const answer = await byId('uo.participation.open').perform({
runId: 42,
idempotencyKey: 'k-1',
params: { place: 'Felucca/Britain', radius: 40, durationMinutes: 120 },
})
assert.equal(answer.ok, true)
assert.deepEqual(calls.open[0], {
runId: 42,
map: 'Felucca',
x: 1496,
y: 1628,
radius: 40,
holdMs: 7_200_000,
idempotencyKey: 'k-1',
})
assert.deepEqual(answer.resources, [
{ kind: 'participation', ref: '42', payload: { runId: 42, place: 'Felucca/Britain', radius: 40 } },
])
})
test('a place the atlas does not know is a refusal an author can read, not a retry', async () => {
const answer = await byId('uo.participation.open').perform({
runId: 42,
idempotencyKey: 'k-1',
params: { place: 'Felucca/Atlantis', radius: 40 },
})
assert.equal(answer.ok, false)
assert.equal(answer.retry, false)
assert.match(answer.error, /no landmark called "Atlantis"/)
assert.equal(calls.open.length, 0)
})
test('an area outside the bound is refused before anything is sent', async () => {
for (const radius of [0, -1, actions.MAX_AREA_RADIUS + 1, 1.5]) {
const answer = await byId('uo.participation.open').perform({
runId: 42,
idempotencyKey: 'k-1',
params: { place: 'Felucca/Britain', radius },
})
assert.equal(answer.ok, false, String(radius))
assert.equal(answer.retry, false, String(radius))
}
assert.equal(calls.open.length, 0)
})
test('a dry run checks the place and the radius and opens nothing', async () => {
const good = await byId('uo.participation.open').perform({
runId: 42,
idempotencyKey: 'k-1',
params: { place: 'Felucca/Britain', radius: 40 },
verify: true,
})
assert.deepEqual(good, { ok: true })
assert.equal(calls.open.length, 0)
// And it is a real check rather than an unconditional yes: the failure an
// author most wants caught before the night of the event is a place that is not
// on this shard's map.
const bad = await byId('uo.participation.open').perform({
runId: 42,
idempotencyKey: 'k-1',
params: { place: 'Felucca/Atlantis', radius: 40 },
verify: true,
})
assert.equal(bad.ok, false)
})
test('the ledger is NOT reconciled by boot stamp, unlike everything else here', async () => {
// The phase's one genuine divergence from wave 1. `reconcileByBootId` works
// because a crier line and a news article live in shard memory, so a changed
// `bootId` IS the proof they are gone. A participation ledger is written into
// the world save specifically so that it survives a restart — reporting it lost
// on a boot change would orphan the one resource the phase persisted.
const open = byId('uo.participation.open')
assert.notEqual(open.reconcile, actions.reconcileByBootId)
// No stamp on the resource either, so nothing downstream can be tempted to
// compare one.
const answer = await open.perform({
runId: 42,
idempotencyKey: 'k-1',
params: { place: 'Felucca/Britain', radius: 40 },
})
assert.equal(answer.resources[0].payload.bootId, undefined)
// It asks instead, and only an explicit 404 takes a row out.
assert.deepEqual(await open.reconcile({ resources: [{ ref: '42' }] }), { ok: true, inForce: ['42'] })
uoLinkClient.snapshotParticipation = async () => ({ ok: false, status: 404, data: {} })
assert.deepEqual(await open.reconcile({ resources: [{ ref: '42' }] }), { ok: true, inForce: [] })
// A shard that is down has not said the ledger is gone.
uoLinkClient.snapshotParticipation = async () => ({ ok: false, status: 503, data: {} })
assert.deepEqual(await open.reconcile({ resources: [{ ref: '42' }] }), { ok: true, inForce: ['42'] })
})
test('a run the shard has already forgotten is a successful revert', async () => {
// §L: "gone, and that is fine". A shard that restarted past its grace window,
// or a second teardown attempt, must not leave a row failing forever.
uoLinkClient.closeParticipation = async () => ({ ok: false, status: 404, data: {} })
assert.deepEqual(await byId('uo.participation.open').revert({ resources: [{ ref: '42' }] }), { ok: true })
uoLinkClient.closeParticipation = async () => ({ ok: false, status: 503, data: {} })
assert.deepEqual(
await byId('uo.participation.open').revert({ resources: [{ ref: '42' }] }),
{ ok: true, failed: ['42'] },
)
})
test('collect files the tally as participants, keyed by character serial', async () => {
uoLinkClient.snapshotParticipation = async (b) => {
calls.snapshot.push(b)
return {
ok: true,
status: 200,
data: {
participants: [
{
serial: '0x400150E8',
name: 'Darrow',
acct: 'seed_001',
webId: '17',
seconds: 3600,
minutes: '60.00',
kills: 3,
score: '75.0000',
firstMs: 1788550182074,
},
// No account link: the shard reports no webId, and there is nothing to
// resolve. Most characters are this one.
{
serial: '0x1',
name: 'Nobody',
seconds: 60,
minutes: '1.00',
kills: 0,
score: '1.0000',
firstMs: 1788550182074,
},
],
},
}
}
const answer = await byId('uo.participation.collect').perform({ runId: 42, idempotencyKey: 'k-2' })
assert.equal(answer.ok, true)
assert.equal(calls.snapshot[0].idempotencyKey, 'k-2')
assert.deepEqual(answer.participants.map((p) => p.memberKey), ['0x400150E8', '0x1'])
// The one field core will not take on trust: it is a foreign key into `users`,
// so a serial passed here would either fail the insert or attribute somebody's
// attendance to a stranger.
assert.equal(answer.participants[0].userId, 17)
assert.equal(answer.participants[1].userId, undefined)
// The score is opaque to core; the components are carried so a results table
// can say why somebody scored what they did.
assert.deepEqual(answer.participants[0].meta, {
name: 'Darrow', seconds: 3600, minutes: '60.00', kills: 3,
})
})
test('a webId that is not a positive integer resolves to nothing at all', () => {
for (const bad of [null, undefined, '', 'abc', '0', '-3', '1.5', {}]) {
assert.equal(actions.webUserId(bad), undefined, JSON.stringify(bad))
}
assert.equal(actions.webUserId('17'), 17)
assert.equal(actions.webUserId(17), 17)
})
test('a busy shard is retried, because the work is happening', async () => {
// 425 is `bridge.busy`: a snapshot of this run is already walking across Core
// ticks. Transient by construction, and deliberately not in PERMANENT_STATUSES.
uoLinkClient.snapshotParticipation = async () => ({
ok: false,
status: 425,
data: { kind: 'bridge.busy', reason: 'a command under this key is in flight' },
})
const answer = await byId('uo.participation.collect').perform({ runId: 42, idempotencyKey: 'k-2' })
assert.equal(answer.ok, false)
assert.equal(answer.retry, true)
// Where the event plane simply being switched off is not: 403 is an operator's
// deliberate refusal and will still be true in sixty seconds.
uoLinkClient.snapshotParticipation = async () => ({
ok: false,
status: 403,
data: { kind: 'participation.error', reason: 'the event plane is disabled on this shard' },
})
const off = await byId('uo.participation.collect').perform({ runId: 42, idempotencyKey: 'k-2' })
assert.equal(off.retry, false)
// And the shard's own words reach the run log, because for an event that ran at
// four in the morning that log is the only place anyone will learn why.
assert.match(off.error, /event plane is disabled/)
})
test('a dry run of collect reads nothing', async () => {
assert.deepEqual(
await byId('uo.participation.collect').perform({ runId: 42, idempotencyKey: 'k-2', verify: true }),
{ ok: true },
)
assert.equal(calls.snapshot.length, 0)
})

607
server/utils/assetBridge.js Normal file
View File

@@ -0,0 +1,607 @@
// The Asset Bridge client (docs/link/v8.md §5, §6, §8 — protocol 8, phase 3).
//
// Three walks over the same request/reply path `clilocBridge.js` already uses,
// and everything that file says about the envelope holds here unchanged: only
// `cut: 'end'` means finished, the cursor must advance, and 425 is the ordinary
// answer during an import rather than an error.
//
// What is different is what each walk is FOR.
//
// ── `readManifest` — what the shard could serve, without the pixels ────────
//
// §6's stage 2. Every row is `{ key, sha256, bytes, width, height }`, so the
// site can diff against what it already holds and ask for only the keys whose
// hash moved. On the ordinary case — a shard restart that changed nothing —
// that diff is empty and no pixels cross at all.
//
// This family pages on the shard's WALL CLOCK, not on bytes. Its rows are about
// ninety bytes and the whole catalogue is one page by the byte budget, but
// producing that page means decoding hundreds of sprites and the sidecar waits
// ten seconds for a reply. So `cut: 'limit'` is the normal page ending here,
// where for clilocs it would have signalled something wrong.
//
// ── `fetchAssets` — the pixels, for keys we chose ─────────────────────────
//
// Each row carries a base64 PNG. The shard encodes it: `System.Drawing` is
// already in its decode path, so PNG costs it no new dependency, and having the
// hash cover exactly the bytes we store is what makes the next Update a diff.
//
// **`catalog` is passed on every fetch and it is not optional in practice.** It
// is an id the shard derives from the client files themselves, so handing it back
// makes the shard refuse if those files moved since the manifest was read.
// Without it an operator patching their client mid-import produces one asset set
// stitched out of two, with no error anywhere — the same failure `clilocBridge`
// guards against by comparing (size, mtime) across pages.
//
// ── `resolveBodies` — the atlas's creatures, by class name ────────────────
//
// §8. The shard constructs each type and reads `Body.BodyID`, which is the only
// thing that is correct for a shard's own custom creatures. That runs on its Core
// thread, so the batch is small and the shard REFUSES an over-long list rather
// than truncating it — hence the chunking here, and hence a chunk size that is a
// constant rather than "as many as fit".
// Required as a namespace, not destructured: a test that stubs the sidecar
// replaces these on the module object, and a destructured copy taken at load
// time would keep calling the real one.
const uoLinkClient = require('./uoLinkClient')
const log = require('../core').logger('asset-bridge')
/** The only family phase 3 serves. §5's key scheme covers statics and land later. */
const FAMILY = 'body'
// Chunk size for the body pass. The shard's own cap defaults to 100 and it
// refuses rather than truncates, so this must stay at or under it — a mismatch
// here does not degrade, it fails every chunk.
const BODY_CHUNK = 100
// Chunk size for a fetch request. The shard cuts the PAGE by byte budget within
// whatever it is handed, so this only bounds how large a single request is; a
// chunk of 400 one-kilobyte sprites is a couple of pages.
const FETCH_CHUNK = 400
// Bounds on each walk. None is expected to be reached — the catalogue is under a
// thousand rows — and each exists so that a shard answering nonsense costs a
// bounded amount of time rather than an unbounded amount of memory.
const MAX_PAGES = 200
const MAX_ROWS = 100000
// 425 is flow control, not failure: the shard's asset plane serves one request at
// a time because its outbound queue is bounded in lines rather than bytes. During
// an import a page coming back busy is expected, so it is retried with a backoff
// rather than failing the walk.
const BUSY_RETRIES = 6
const BUSY_BACKOFF_MS = [200, 400, 800, 1600, 3200, 5000]
class AssetBridgeError extends Error {
constructor(message, code) {
super(message)
this.name = 'AssetBridgeError'
this.code = code
}
}
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms))
/**
* Map a sidecar response onto one of this module's codes.
*
* Deliberately the same vocabulary `clilocBridge.describeFailure` uses, because
* the admin panel reports them side by side and an operator should not have to
* learn two names for "you have not switched this on".
*
* 422 is the one that means something different here: on the cliloc path it is a
* file the shard cannot decode, and on this one it is *also* the mid-import guard
* firing — the client files moved between the manifest and the fetch.
*/
function describeFailure(res, what) {
const reason = res?.data?.reason || res?.error || `sidecar responded ${res?.status}`
switch (res?.status) {
case 403:
return new AssetBridgeError(
`The shard is refusing to serve client assets (Bridge.AssetsEnabled is off): ${reason}`,
'DISABLED',
)
case 404:
return new AssetBridgeError(`The shard has no ${what}: ${reason}`, 'NO_SOURCE')
case 409:
return new AssetBridgeError(
`The sidecar refused the protocol version this build declares: ${reason}`,
'PROTOCOL',
)
case 422:
return new AssetBridgeError(reason, 'SOURCE_CHANGED')
case 425:
return new AssetBridgeError(
'The shard stayed busy serving another asset request',
'BUSY',
)
case 503:
// The named `NO_IMAGING` outcome arrives this way: a Linux shard host with
// no libgdiplus cannot render a sprite at all, and §4.4 requires that be an
// actionable sentence rather than a stack trace. The shard's own wording
// already names the package and the command, so it is passed through.
return new AssetBridgeError(reason, /libgdiplus/i.test(reason) ? 'NO_IMAGING' : 'SHARD_DOWN')
case 504:
return new AssetBridgeError(`The shard did not answer: ${reason}`, 'SHARD_DOWN')
default:
return new AssetBridgeError(reason, 'UNAVAILABLE')
}
}
/** One call, with the 425 backoff. `send` returns the client's `{ ok, ... }`. */
async function withBusyRetry(send, what) {
for (let attempt = 0; ; attempt++) {
const res = await send()
if (res.ok) return res.data
if (res.status === 425 && attempt < BUSY_RETRIES) {
await sleep(BUSY_BACKOFF_MS[Math.min(attempt, BUSY_BACKOFF_MS.length - 1)])
continue
}
throw describeFailure(res, what)
}
}
/**
* Shared page-envelope checks (§3.4).
*
* Every one of these is a way a walk can end in something that LOOKS like a
* complete import and is not, which is why they are assertions rather than
* warnings: a truncated catalogue is indistinguishable downstream from a client
* that simply has fewer creatures.
*/
function checkPage(page, { arrayName, cursor, pages, noun = 'asset' }) {
if (!page || !Array.isArray(page[arrayName])) {
throw new AssetBridgeError(
`The shard sent a ${noun} page with no ${arrayName} array`,
'MALFORMED',
)
}
if (!page.more) {
if (page.cut !== 'end') {
throw new AssetBridgeError(
`The shard stopped sending ${noun} rows after ${pages} page(s) (cut: ${page.cut || 'unknown'})`,
'INCOMPLETE',
)
}
return { done: true }
}
if (!page.cursor || page.cursor === cursor) {
throw new AssetBridgeError(
`The shard asked for another ${noun} page without advancing its cursor (${page.cursor || 'none'})`,
'STUCK',
)
}
return { done: false, cursor: page.cursor }
}
/**
* SHA-256 of a buffer, lowercase hex.
*
* Here rather than in each caller because the shard's `BridgeAssets.Sha256Hex`
* is one function on its side too, and a hash that has to match across a wire
* should have exactly one spelling at each end.
*/
function sha256Of(buffer) {
return require('crypto').createHash('sha256').update(buffer).digest('hex')
}
// The client files the body catalogue is derived from. `assets.sources` reports
// every file the shard can see; these are the ones that decide a sprite.
//
// `body.def` and `bodyconv.def` are in the list and it would be easy to leave
// them out — they hold no pixels. They decide WHICH record a body id resolves to,
// so an operator editing one changes what every affected creature looks like
// while every anim file stays byte-identical. That is precisely the drift a
// content hash of the art files cannot see.
const SOURCE_FILES = [
'anim.idx', 'anim.mul',
'anim2.idx', 'anim2.mul',
'anim3.idx', 'anim3.mul',
'anim4.idx', 'anim4.mul',
'anim5.idx', 'anim5.mul',
'body.def', 'bodyconv.def',
'verdata.mul',
]
/**
* Stage 1 of the import gate (§6): have the client files this family reads
* changed at all?
*
* Returns `{ files, extractorVersion, hashing, complete, imaging }` where `files`
* is a `{ name: { size, mtime, sha256 } }` map over `SOURCE_FILES` — a file the
* shard does not have is simply absent, which is normal (few clients carry all
* five anim files).
*
* **A null `sha256` means "not computed yet", never "changed".** The shard hashes
* off the request path because `anim.mul` alone is 195 MB and hashing it cannot
* fit inside a reply, and it reports `hashing: true` while that runs.
* `sameSources` below falls back to (size, mtime) in that case, which is the same
* gate the shard itself applies.
*/
async function sourceFingerprint() {
const res = await uoLinkClient.getAssetSources()
if (!res.ok) throw describeFailure(res, 'client file manifest')
// A 200 from this call stopped meaning "client files are on offer" in phase 7:
// it now answers whenever either plane is enabled, so a shard serving only its
// configuration tree reports an empty file list rather than a 403. Read as-is
// that becomes "your client has no animation files", which sends an operator to
// the wrong place entirely.
if (res.data?.assetsEnabled === false) {
throw new AssetBridgeError(
'The shard is refusing to serve client assets (Bridge.AssetsEnabled is off)',
'DISABLED',
)
}
const wanted = new Set(SOURCE_FILES)
const files = {}
for (const entry of res.data?.files ?? []) {
const name = String(entry?.name || '').toLowerCase()
if (!wanted.has(name)) continue
files[name] = {
size: Number(entry.size) || 0,
mtime: Number(entry.mtime) || 0,
sha256: entry.sha256 ?? null,
}
}
return {
files,
extractorVersion: Number(res.data?.extractorVersion) || 0,
hashing: Boolean(res.data?.hashing),
complete: Boolean(res.data?.complete),
imaging: res.data?.imaging ?? null,
// Which §5 key families this overlay can be asked for (phase 5). Absent on a
// phase-3 or phase-4 overlay, which served bodies and nothing else — so the
// fallback is `['body']` rather than `[]`: an older shard is not a shard with
// no assets, and treating it as one would turn a working bestiary off.
families: Array.isArray(res.data?.families) && res.data.families.length > 0
? res.data.families.map(String)
: [FAMILY],
}
}
/**
* True when two source fingerprints describe the same client files.
*
* The file SET has to match as well as each file's contents: a client that gained
* an `anim5.mul` it did not have before is a client whose gargoyles suddenly
* resolve, and comparing only the files present in both would call that
* unchanged.
*/
function sameSources(a, b) {
if (!a || !b) return false
if (a.extractorVersion !== b.extractorVersion) return false
const names = new Set([...Object.keys(a.files ?? {}), ...Object.keys(b.files ?? {})])
for (const name of names) {
const left = a.files?.[name]
const right = b.files?.[name]
if (!left || !right) return false
if (left.sha256 && right.sha256) {
if (left.sha256 !== right.sha256) return false
continue
}
if (left.size !== right.size || left.mtime !== right.mtime || left.size <= 0) return false
}
return names.size > 0
}
/**
* Stage 2: the whole manifest for the body family.
*
* Returns `{ rows, catalog, extractorVersion, playerBodies, pages, scanned }`.
* No pixels — `rows` is `[{ key, sha256, bytes, width, height, body, direction }]`.
*/
async function readManifest({ family = FAMILY } = {}) {
const started = Date.now()
const rows = []
let cursor = null
let pages = 0
let catalog = null
let extractorVersion = 0
let playerBodies = []
let scanned = 0
let finished = false
while (pages < MAX_PAGES) {
const page = await withBusyRetry(
() => uoLinkClient.getAssetManifest({ family, cursor }),
`${family} asset manifest`,
)
pages++
if (catalog === null) {
catalog = page.catalog ?? null
extractorVersion = Number(page.extractorVersion) || 0
playerBodies = Array.isArray(page.playerBodies) ? page.playerBodies.map(Number) : []
} else if (page.catalog !== catalog) {
// The client files moved between two pages of one walk. Refusing is the
// only honest answer: half of what we hold describes files that no longer
// exist, and nothing later can tell which half.
throw new AssetBridgeError(
"The shard's client files changed while the manifest was being read; nothing was imported",
'SOURCE_CHANGED',
)
}
scanned += Number(page.scanned) || 0
for (const row of page.rows) {
const key = String(row?.key ?? '')
if (key === '') continue
rows.push({
key,
family,
sha256: String(row?.sha256 ?? ''),
bytes: Number(row?.bytes) || 0,
width: Number(row?.width) || 0,
height: Number(row?.height) || 0,
body: Number.isFinite(Number(row?.body)) ? Number(row.body) : null,
// Which action the thumbnail came from (§11.2, phase 6). All but 73 of
// this client's bodies answer 0; the rest have no art there and are
// catalogued deeper, with the key naming the action. An overlay older
// than phase 6 omits it, and 0 is the right reading of that.
action: Number.isFinite(Number(row?.action)) ? Number(row.action) : 0,
direction: Number.isFinite(Number(row?.direction)) ? Number(row.direction) : null,
})
}
if (rows.length > MAX_ROWS) {
throw new AssetBridgeError(
`The shard listed more than ${MAX_ROWS} assets; refusing to keep reading`,
'TOO_LARGE',
)
}
const state = checkPage(page, { arrayName: 'rows', cursor, pages })
if (state.done) {
finished = true
break
}
cursor = state.cursor
}
if (!finished) {
throw new AssetBridgeError(
`The asset manifest did not end within ${MAX_PAGES} pages; nothing was imported`,
'TOO_LARGE',
)
}
log.info('asset manifest read from the shard', {
family,
rows: rows.length,
scanned,
pages,
ms: Date.now() - started,
})
return { rows, catalog, extractorVersion, playerBodies, pages, scanned }
}
/**
* The bytes for an explicit list of keys.
*
* Returns a Map of key → `{ sha256, bytes, width, height, body, action, direction, png }`
* where `png` is a Buffer. A key the shard could not serve is **absent from the
* map** rather than present with a null — the caller then decides what that means
* for its own row, and the two ways it happens (`absent`, `unsupported`) are
* counted separately in the returned tallies so an operator can tell "this client
* has no art for that body" from "the site asked for a key shape this shard does
* not serve", which is a bug rather than a gap.
*/
async function fetchAssets({ keys, catalog } = {}) {
const started = Date.now()
const out = new Map()
const missing = { absent: 0, unsupported: 0 }
const list = Array.isArray(keys) ? keys.filter((k) => typeof k === 'string' && k !== '') : []
if (list.length === 0) return { assets: out, missing, pages: 0, catalog: catalog ?? null }
let pages = 0
// The catalogue the shard actually answered under. The body import already knows
// it from the manifest, but the on-demand families have no manifest to learn it
// from (§11) — so it is read back off the reply and stored with the rows, which
// is what makes a later "is this stale?" answerable per key.
let answered = catalog ?? null
for (let i = 0; i < list.length; i += FETCH_CHUNK) {
const chunk = list.slice(i, i + FETCH_CHUNK)
let cursor = null
let finished = false
let walked = 0
while (walked < MAX_PAGES) {
const page = await withBusyRetry(
() => uoLinkClient.fetchAssets({ keys: chunk, catalog, cursor }),
'asset content',
)
pages++
walked++
if (typeof page.catalog === 'string' && page.catalog !== '') {
if (answered !== null && page.catalog !== answered) {
// Two pages of one walk describing two different clients. The shard
// refuses this when it is told what to expect; when it was not told —
// the first fetch of a warm pass — this is where it is caught.
throw new AssetBridgeError(
`The shard's client files changed mid-fetch (catalog ${answered} became ${page.catalog})`,
'UNAVAILABLE',
)
}
answered = page.catalog
}
for (const row of page.rows ?? []) {
const key = String(row?.key ?? '')
if (key === '') continue
if (row?.status !== 'ok') {
if (row?.status === 'unsupported') missing.unsupported++
else missing.absent++
continue
}
if (typeof row.png !== 'string' || row.png === '') {
missing.absent++
continue
}
out.set(key, {
sha256: String(row.sha256 ?? ''),
bytes: Number(row.bytes) || 0,
width: Number(row.width) || 0,
height: Number(row.height) || 0,
body: Number.isFinite(Number(row.body)) ? Number(row.body) : null,
action: Number.isFinite(Number(row.action)) ? Number(row.action) : null,
direction: Number.isFinite(Number(row.direction)) ? Number(row.direction) : null,
// Phase 5's art families carry these; the body catalogue does not, and a
// consumer that wants neither is unaffected by either.
hue: Number.isFinite(Number(row.hue)) ? Number(row.hue) : null,
partialHue: typeof row.partialHue === 'boolean' ? row.partialHue : null,
source: typeof row.source === 'string' ? row.source : null,
png: Buffer.from(row.png, 'base64'),
})
}
const state = checkPage(page, { arrayName: 'rows', cursor, pages: walked })
if (state.done) {
finished = true
break
}
cursor = state.cursor
}
if (!finished) {
throw new AssetBridgeError(
`An asset fetch did not end within ${MAX_PAGES} pages; nothing was imported`,
'TOO_LARGE',
)
}
}
log.info('asset content fetched from the shard', {
catalog: answered,
asked: list.length,
got: out.size,
absent: missing.absent,
unsupported: missing.unsupported,
pages,
ms: Date.now() - started,
})
return { assets: out, missing, pages, catalog: answered }
}
/**
* Slug → body id, for the atlas's own creature list (§8).
*
* `creatures` is `[{ slug, name }]` where `name` is the ServUO class name — which
* `shard_spawn_creatures.name` already holds, because the atlas build picks the
* winning spelling of the spawn TYPE token rather than inventing a display name.
* That is why this needs no new column to ask its question.
*
* Returns `[{ slug, typeName, body, status }]`, one row per creature asked, with
* every outcome recorded — including the negative ones. A creature the shard says
* it does not have is a fact worth keeping: without it, the next pass asks again,
* and the pass costs a real constructor per name on the shard's Core thread.
*/
async function resolveBodies({ creatures } = {}) {
const started = Date.now()
const list = Array.isArray(creatures) ? creatures : []
const out = []
for (let i = 0; i < list.length; i += BODY_CHUNK) {
const chunk = list.slice(i, i + BODY_CHUNK)
const bySlug = new Map()
for (const creature of chunk) {
const typeName = String(creature?.name ?? '').trim()
if (typeName === '') continue
// Several slugs can share a type name only if the atlas slugified two
// spellings to one slug, in which case they ARE one creature; asking once
// per distinct name is what keeps the batch inside the shard's cap.
if (!bySlug.has(typeName)) bySlug.set(typeName, [])
bySlug.get(typeName).push(String(creature.slug))
}
const types = [...bySlug.keys()]
if (types.length === 0) continue
const page = await withBusyRetry(() => uoLinkClient.resolveBodies(types), 'body resolution')
if (!page || !Array.isArray(page.rows)) {
throw new AssetBridgeError('The shard sent a body resolution with no rows array', 'MALFORMED')
}
for (const row of page.rows) {
const typeName = String(row?.type ?? '')
const slugs = bySlug.get(typeName)
if (!slugs) continue
const status = String(row?.status ?? 'failed')
const body = status === 'ok' && Number.isFinite(Number(row?.body)) ? Number(row.body) : null
for (const slug of slugs) out.push({ slug, typeName, body, status })
}
}
const resolved = out.filter((r) => r.status === 'ok').length
log.info('creature bodies resolved by the shard', {
asked: list.length,
answered: out.length,
resolved,
ms: Date.now() - started,
})
return out
}
module.exports = {
AssetBridgeError,
// Shared with `treeBridge.js` (phase 7): the 425 backoff, the page-envelope
// checks and the hash are properties of this PLANE, not of the body family, and
// a second copy of any of them is a second place for the envelope to drift.
withBusyRetry,
checkPage,
sha256Of,
FAMILY,
BODY_CHUNK,
FETCH_CHUNK,
MAX_PAGES,
MAX_ROWS,
SOURCE_FILES,
sourceFingerprint,
sameSources,
readManifest,
fetchAssets,
resolveBodies,
}

View File

@@ -0,0 +1,318 @@
// Cliloc table — the SHARD source (docs/link/v8.md §9, protocol 8 phase 2).
//
// `clilocSource.js` is the filesystem half of this story and predates it. This is
// the half that replaces the part of it nobody enjoyed: until protocol 8 the base
// table reached the site because an operator installed UOFiddler, built a
// converter against its `Ultima.dll`, ran it over their client's compressed
// `Cliloc.enu` and copied a five-megabyte file to the web host — every time they
// patched their client.
//
// The shard has always had those files (a ServUO server cannot boot without a UO
// client) and, as of phase 2, has the decompressor too. So the base table now
// arrives over the same request/reply path as every other shard read, and the
// operator installs nothing.
//
// **What is NOT here.** Overlays. Shard-added items carry cliloc ids no client
// table has, ServUO has no server-side notion of a custom cliloc, and there is
// therefore nothing on the shard to ask for. `custom/` stays a directory the site
// reads (`clilocSource.readOverlays`), and the model merges it OVER whatever
// arrives here. That division is the whole of CLILOCS.md §Shard-added items and
// it is unchanged by this file.
//
// ── Why this walks pages instead of asking for a table ────────────────────
//
// The sidecar's reply timeout is 10 s and its inbound line cap is 1 MiB, so a
// five-megabyte table cannot be one answer. The shard cuts pages at a 512 KiB
// byte budget and hands back a cursor; this walks them. A stock English table is
// about eleven pages.
//
// Three properties of that envelope are load-bearing and each has a check below:
//
// - **Only `cut: 'end'` means finished.** A short page can equally mean the
// budget was spent (`budget`) or the family stopped at its own limit
// (`limit`). Treating a short page as the end would import a truncated table,
// which is indistinguishable downstream from a complete one — some items
// named, some not, exactly what "no table at all" looks like.
// - **The cursor must advance.** A shard that answered the same cursor forever
// would spin this loop until the request timeout with nothing to show.
// - **The file must not change underneath the walk.** Every page echoes the
// source's size and mtime; an operator patching their client mid-import would
// otherwise produce one table stitched from two, with no error anywhere.
// Required as a namespace, not destructured: a test that stubs the sidecar
// replaces these on the module object, and a destructured copy taken at load
// time would keep calling the real one.
const uoLinkClient = require('./uoLinkClient')
const log = require('../core').logger('cliloc-bridge')
/** The client file the base table comes from, as `assets.sources` names it. */
const SOURCE_FILE = 'cliloc.enu'
const DEFAULT_LANGUAGE = 'enu'
// Bounds on the walk. Neither is expected to be reached — a stock table is ~11
// pages and ~67k rows — and both exist so that a shard answering nonsense costs a
// bounded amount of time rather than an unbounded amount of memory.
const MAX_PAGES = 200
const MAX_ROWS = 500000
// 425 is the ordinary answer during an import, not an error: the shard's asset
// plane serves one request at a time on purpose, because its outbound queue is
// bounded in lines rather than bytes. So a page that comes back busy is retried
// with a short backoff rather than failing the import.
const BUSY_RETRIES = 5
const BUSY_BACKOFF_MS = [200, 400, 800, 1600, 3200]
class ClilocBridgeError extends Error {
constructor(message, code) {
super(message)
this.name = 'ClilocBridgeError'
this.code = code
}
}
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms))
/**
* Map a sidecar response onto one of this module's codes.
*
* The statuses are the ones `respond_assets` produces, and the distinction that
* matters most to an operator is 403 vs 404: "you have not switched this on" and
* "your client does not have that file" are different jobs, and both are things
* they can fix.
*/
function describeFailure(res, what) {
const reason = res?.data?.reason || res?.error || `sidecar responded ${res?.status}`
switch (res?.status) {
case 403:
return new ClilocBridgeError(
`The shard is refusing to serve client assets (Bridge.AssetsEnabled is off): ${reason}`,
'DISABLED',
)
case 404:
return new ClilocBridgeError(`The shard has no ${what}: ${reason}`, 'NO_SOURCE')
case 409:
return new ClilocBridgeError(
`The sidecar refused the protocol version this build declares: ${reason}`,
'PROTOCOL',
)
case 422:
return new ClilocBridgeError(`The shard could not read its own ${what}: ${reason}`, 'UNREADABLE')
case 425:
return new ClilocBridgeError(
'The shard is busy serving another asset request and stayed busy',
'BUSY',
)
case 503:
case 504:
return new ClilocBridgeError(`The shard did not answer: ${reason}`, 'SHARD_DOWN')
default:
return new ClilocBridgeError(reason, 'UNAVAILABLE')
}
}
/**
* Stage 1: the fingerprint of the shard's own cliloc file.
*
* Returns `{ file, size, mtime, sha256, extractorVersion, hashing, complete }`.
*
* `sha256` may be **null** — the shard reports hashes only once it has computed
* them off the request path, because hashing the client files it also serves
* (343 MB of art and animation) cannot fit inside a 10 s reply. A null hash means
* "not yet", never "changed", and `sameSource` below compares (size, mtime) in
* that case, which is the same gate the shard itself uses.
*/
async function fingerprint() {
const res = await uoLinkClient.getAssetSources()
if (!res.ok) throw describeFailure(res, 'client file manifest')
// Since protocol 8 phase 7 this call answers when EITHER plane is enabled, so
// a 200 no longer means the client files are on offer. Without this check an
// operator who switched client-file extraction off would read "your UO client
// has no cliloc.enu" and go looking at their client install for a setting that
// lives on their shard.
if (res.data?.assetsEnabled === false) {
throw new ClilocBridgeError(
'The shard is refusing to serve client assets (Bridge.AssetsEnabled is off)',
'DISABLED',
)
}
const files = Array.isArray(res.data?.files) ? res.data.files : []
const entry = files.find((f) => String(f?.name || '').toLowerCase() === SOURCE_FILE)
if (!entry) {
throw new ClilocBridgeError(
`The shard's UO client has no ${SOURCE_FILE} (it reported ${files.length} client file(s))`,
'NO_SOURCE',
)
}
return {
kind: 'bridge',
file: entry.name,
path: entry.path ?? null,
size: Number(entry.size) || 0,
mtime: Number(entry.mtime) || 0,
sha256: entry.sha256 ?? null,
extractorVersion: Number(res.data?.extractorVersion) || 0,
hashing: Boolean(res.data?.hashing),
complete: Boolean(res.data?.complete),
}
}
/**
* True when two fingerprints describe the same client file.
*
* Hash first when both sides have one, because a hash is the only thing that
* catches a file rewritten with the same length and timestamp. Falls back to
* (size, mtime) when either side's hash is missing, which is the case on the
* first poll after a shard restart and the reason `hashing` exists at all.
*/
function sameSource(a, b) {
if (!a || !b) return false
if (a.extractorVersion !== b.extractorVersion) return false
if (a.sha256 && b.sha256) return a.sha256 === b.sha256
return a.size === b.size && a.mtime === b.mtime && a.size > 0
}
/** One page, with the 425 backoff. */
async function fetchPage({ lang, cursor }) {
for (let attempt = 0; ; attempt++) {
const res = await uoLinkClient.getClilocTable({ lang, cursor })
if (res.ok) return res.data
if (res.status === 425 && attempt < BUSY_RETRIES) {
await sleep(BUSY_BACKOFF_MS[Math.min(attempt, BUSY_BACKOFF_MS.length - 1)])
continue
}
throw describeFailure(res, `cliloc.${lang}`)
}
}
/**
* Walk the whole table.
*
* Returns `{ entries, source }` where `entries` is `[{ number, flag, text }]` in
* the shape `clilocParse` produces, so the merge in `shardClilocs.model` does not
* care which source an entry came from.
*
* Blanks are already gone: the shard drops the ~56,000 empty strings a stock
* table carries before they reach the wire, since the site would drop them at
* import anyway. Nothing downstream changes — `db.replaceAll` still filters, and
* still would if a source ever sent one.
*/
async function readCliloc({ lang = DEFAULT_LANGUAGE } = {}) {
const started = Date.now()
const entries = []
let cursor = null
let pages = 0
let first = null
let finished = false
let total = null
while (pages < MAX_PAGES) {
const page = await fetchPage({ lang, cursor })
pages++
if (!page || !Array.isArray(page.rows)) {
throw new ClilocBridgeError('The shard sent a cliloc page with no rows array', 'MALFORMED')
}
if (first === null) {
first = { size: Number(page.size) || 0, mtime: Number(page.mtime) || 0 }
total = Number.isFinite(Number(page.total)) ? Number(page.total) : null
} else if (Number(page.size) !== first.size || Number(page.mtime) !== first.mtime) {
// The client was patched (or a different one mounted) between two pages.
// Refusing is the only honest answer: half of what we hold is from a file
// that no longer exists, and nothing later can tell which half.
throw new ClilocBridgeError(
'The shard\'s cliloc file changed while it was being read; nothing was imported',
'SOURCE_CHANGED',
)
}
for (const row of page.rows) {
const number = Number(row?.n)
if (!Number.isInteger(number)) continue
entries.push({ number, flag: Number(row?.f) || 0, text: String(row?.t ?? '') })
}
if (entries.length > MAX_ROWS) {
throw new ClilocBridgeError(
`The shard sent more than ${MAX_ROWS} cliloc rows; refusing to keep reading`,
'TOO_LARGE',
)
}
if (!page.more) {
// `cut` is the field that says WHY a page was the last one, and only one of
// its values means the table ended. A shard that stopped for its own limit
// has not finished, and importing what arrived would silently drop the tail.
if (page.cut !== 'end') {
throw new ClilocBridgeError(
`The shard stopped sending cliloc rows after ${entries.length} (cut: ${page.cut || 'unknown'})`,
'INCOMPLETE',
)
}
finished = true
break
}
if (!page.cursor || page.cursor === cursor) {
// Either would loop forever: no cursor to advance with, or the same one
// back again.
throw new ClilocBridgeError(
`The shard asked for another cliloc page without advancing its cursor (${page.cursor || 'none'})`,
'STUCK',
)
}
cursor = page.cursor
}
if (!finished) {
throw new ClilocBridgeError(
`The cliloc table did not end within ${MAX_PAGES} pages; nothing was imported`,
'TOO_LARGE',
)
}
log.info('cliloc table read from the shard', {
lang,
entries: entries.length,
pages,
ms: Date.now() - started,
})
return {
entries,
source: {
kind: 'bridge',
lang,
file: `cliloc.${lang}`,
size: first?.size ?? 0,
mtime: first?.mtime ?? 0,
pages,
// What the shard said it holds, kept beside what actually arrived. They
// agree or the walk is wrong, and an operator seeing them disagree in the
// panel learns more than a single number would tell them.
reported: total,
received: entries.length,
},
}
}
module.exports = {
ClilocBridgeError,
SOURCE_FILE,
DEFAULT_LANGUAGE,
MAX_PAGES,
MAX_ROWS,
fingerprint,
sameSource,
readCliloc,
}

View File

@@ -6,6 +6,24 @@
// - the server, which refreshes the table on boot (`shardClilocs.model.js`)
// - the admin panel, which can force a reimport without a restart
//
// ── What protocol 8 took away, and what it left ───────────────────────────
//
// The BASE table no longer comes from here on a shard that has uo-link
// configured: `clilocBridge.js` asks the shard for it, because the shard has the
// operator's client files already and, since phase 2, the decompressor to read
// them (docs/link/v8.md §9). Nobody converts a file by hand any more.
//
// Two things keep this module alive rather than deleting it:
//
// - **Overlays.** Shard-added items carry cliloc ids no client table has, and
// ServUO has no server-side notion of a custom cliloc — there is nothing on
// the shard to ask for. `custom/` is still a directory the site reads, and
// `readOverlays` below is the entry point the bridge path uses.
// - **Installs with no shard link**, and development. A site that has never
// configured uo-link can still be pointed at a converted file; that path is
// deprecated, not removed, and it stays the whole of this module's base-table
// behaviour.
//
// The files are the OPERATOR'S (see docs/website/CLILOCS.md). Nothing derived
// from them is committed: the repo holds no string table, exactly as it holds no
// map snapshot and no artwork. That rule is why this module reads a configured
@@ -194,6 +212,63 @@ function readSources(configured) {
return { root, files }
}
/**
* Read the OVERLAY files only, with no base table.
*
* The bridge path needs exactly this: the base arrives from the shard and the
* `custom/` directory beside the configured path still has to be merged over it.
* `readSources` cannot answer it, because resolving a base is the first thing it
* does and there may not be one — an operator on the bridge is entitled to point
* this setting at a directory that holds nothing but `custom/`.
*
* **Never throws.** A path that is blank, missing or unreadable is reported as a
* `problem` string and an empty file list, because none of those may stop a base
* table that arrived perfectly well from being imported. The model decides what
* to do about it — and it has a real decision to make, since an overlay that was
* loaded last time and is missing now is the vanished-source hazard, not a
* config typo.
*/
function readOverlays(configured) {
const target = String(configured ?? '').trim()
if (target === '') return { root: null, files: [], problem: null }
let root
try {
const stat = fs.statSync(target)
root = stat.isFile() ? path.dirname(target) : target
} catch {
return { root: null, files: [], problem: `Cliloc path does not exist: ${target}` }
}
let overlays
try {
overlays = listCustom(root)
} catch (err) {
return { root, files: [], problem: err.message }
}
const files = []
for (const file of overlays) {
let buffer
try {
buffer = fs.readFileSync(file)
} catch {
return { root, files: [], problem: `Cliloc overlay is not readable: ${file}` }
}
files.push({
label: path.relative(root, file).split(path.sep).join('/'),
kind: 'custom',
file,
buffer,
sha256: sha256(buffer),
bytes: buffer.length,
compressed: isCompressedCliloc(buffer),
})
}
return { root, files, problem: null }
}
/**
* A fingerprint of every source: `{ "<label>": "<sha256>" }`, plus the base's
* details for the admin panel.
@@ -244,6 +319,25 @@ function missingSources(current, loaded) {
return Object.keys(loaded).filter((label) => !Object.hasOwn(current, label))
}
/**
* The same question asked of OVERLAYS only.
*
* Needed because the base table moved to the bridge. An install upgraded from the
* file pipeline carries a base label (`clilocs.plain`, say) in its loaded
* fingerprint, and that label is *supposed* to disappear when the base starts
* arriving from the shard — reporting it as a vanished source would make every
* first import after the upgrade demand an approval for a change the upgrade
* itself made. Overlay labels are the ones whose absence is genuinely ambiguous,
* and they are exactly the labels under `custom/`.
*/
function missingOverlays(current, loaded) {
if (!loaded) return []
const prefix = `${CUSTOM_DIR}/`
return Object.keys(loaded).filter(
(label) => label.startsWith(prefix) && !Object.hasOwn(current, label),
)
}
/**
* Read and parse every source, merged into one entry list.
*
@@ -309,8 +403,10 @@ module.exports = {
resolveBase,
listCustom,
readSources,
readOverlays,
hashSources,
sameSources,
missingSources,
missingOverlays,
readCliloc,
}

View File

@@ -662,6 +662,53 @@ const MAPPERS = {
}
},
// Protocol 6. A boss defeat, which until now could only be GUESSED at from
// `champ.update` losing its `bossUp` — a signal that also fires when a spawn is
// reset by a GM, when a boss despawns, and when the sweep simply reconnects.
// This one fires on the death itself.
//
// **The subject is the SPAWN, so it matches `uo.champ.boss_up`'s.** A rule with
// a cooldown on one altar therefore counts a boss going up and that same boss
// coming down as the same subject, which is what an operator writing "not more
// than once an hour about Destard" means. A kill the shard could not attribute
// to an altar carries no spawn, so the boss's own serial stands in — it is a
// subject that exists exactly once, which is all a cooldown needs of it.
//
// **Damagers are not surfaced as variables.** The table is on the frame and it
// is `staff` in the visibility config; putting names into a trigger's data
// would route them into mail an operator can address to `subscribers`, which is
// the field rule undone one layer up. `damagerCount` is a number and says the
// thing worth saying: how many took part.
'champ.boss.killed': (ev, tracker, out) => {
const spawnSerial = ev.serial == null ? null : String(ev.serial)
const bossSerial = ev.bossSerial == null ? null : String(ev.bossSerial)
const subject = spawnSerial || bossSerial
if (!subject) return
// The board no longer has a boss on this altar. Kept in step with the sweep's
// own view so the next `champ.update` carrying `bossUp: true` is read as a
// transition rather than as more of the same.
if (spawnSerial) tracker.champBossUp.set(spawnSerial, false)
const damagers = Array.isArray(ev.damagers) ? ev.damagers : []
out.push({
triggerId: 'uo.champ.boss_killed',
data: defined({
spawnSerial: subject,
champsUrl: PATHS.champs,
bossName: ev.boss || ev.bossType || 'the champion',
category: ev.category || undefined,
location: place(ev),
atPlace: trailing(place(ev), (p) => ` at ${p}`),
killerName: actorName(ev.killer),
damagerCount: damagers.length || undefined,
damagerNote: trailing(damagers.length || null, (n) =>
n === 1 ? ' One player fought it.' : ` ${n} players fought it.`),
}),
})
},
'champ.remove': (ev, tracker) => {
if (ev.serial == null) return
tracker.champActive.delete(String(ev.serial))

View File

@@ -17,7 +17,7 @@ const shardStateModel = require('../model/shardState/shardState.model')
const shardLinksModel = require('../model/shardLinks/shardLinks.model')
const shardMarketModel = require('../model/shardMarket/shardMarket.model')
const uoLinkConfigModel = require('../model/uoLinkConfig/uoLinkConfig.model')
const { settings: settingsModel } = require('../core')
const { settings: settingsModel, events: coreEvents } = require('../core')
const broadcaster = require('./shardBroadcast')
const shardPush = require('./shardPush')
const shardEngagement = require('./shardEngagement')
@@ -103,11 +103,12 @@ async function resolveShardName(shard, deps) {
// Apply the state-change side effect for a kind (if any). Returns a promise.
async function applyStateChange(event, deps) {
const { shardState, uoLinkConfig, log } = deps
const { shardState, uoLinkConfig, eventsReconcile, fromBackfill, log } = deps
switch (event.kind) {
case 'server.hello': {
const incoming = event.bootId || null
if (incoming && state.bootId && incoming !== state.bootId) {
const restarted = Boolean(incoming && state.bootId && incoming !== state.bootId)
if (restarted) {
log.warn('shard restarted (bootId changed) — clearing online roster', {
from: state.bootId,
to: incoming,
@@ -116,6 +117,31 @@ async function applyStateChange(event, deps) {
}
if (incoming) state.bootId = incoming
await uoLinkConfig.recordStatus({ pluginConnected: true, bootId: incoming, lastEventAt: event.t })
if (restarted && !fromBackfill) {
// EVENTS.md F: core has no concept of the game being up, so the module
// says when a ledger of live shard resources has become a claim about a
// world that no longer exists. This is that moment, and a changed
// `bootId` is the only thing that distinguishes it from a sidecar
// reconnect — which changes nothing in the game and must not orphan a row.
//
// **After `recordStatus`, and that ordering is load-bearing.** Every
// action's `reconcile()` decides what is still in force by comparing its
// stamp against the CURRENT boot id, which it reads back out of this
// row. Asking first would have every resource compared against the boot
// that has just ended, and every one of them would look live.
//
// **And never on a backfill replay**, which is the same rule the
// engagement fan-out and the SSE broadcast state below and is far more
// expensive to break here. A reconnect replays the last several
// `server.hello` frames in order — this rig saw three, each with a
// different `bootId` — so every replayed frame looks like a restart, and
// the intermediate ones would compare a resource stamped with the CURRENT
// boot against a boot that ended hours ago. The row is then `orphaned`:
// a live crier line core will never take down again, lost to nothing
// worse than the website reconnecting. The website-was-down case is not
// missed by skipping these — core asks every module at its own boot.
eventsReconcile()
}
return
}
case 'server.shutdown':
@@ -292,6 +318,15 @@ function resolveDeps(deps) {
broadcast: deps.broadcast || broadcaster.broadcast,
pushDispatch: deps.pushDispatch || shardPush.fromShardEvent,
engagement: deps.engagement || shardEngagement.fromShardEvent,
// MODULE_API 1.10.0 (EVENTS.md F, Phase 8). Injectable for the same reason
// every member above is: a test that asserted a shard restart triggers a
// reconcile must be able to see the call without a live event engine behind
// it.
eventsReconcile: deps.eventsReconcile || (() => coreEvents.reconcile()),
// Not injectable — it is the caller's statement about this frame rather than
// a dependency. It reaches `applyStateChange` because the reconcile below is
// the one state change that must not act on a replay; see the note there.
fromBackfill: Boolean(deps.fromBackfill),
log: deps.log || defaultLog,
}
}

View File

@@ -83,7 +83,27 @@ const FEATURES = {
// ── Shipped before v3. Defaults reproduce the previous hardcoded behavior. ──
status: { audience: 'anonymous', fields: {} },
activity: { audience: 'anonymous', fields: {} },
champs: { audience: 'anonymous', fields: {} },
// Protocol 6 adds `champ.boss.killed` to this feature, and with it the first
// field on a champs frame that is about PEOPLE rather than about an altar.
//
// `damagers` is the ranked table of who fought the boss and for how much. It is
// the honest basis for "who slew the champion" and it is also a performance
// record of named players that nobody consented to publish, which is precisely
// the tension the ladder exists to let a shard resolve for itself. It defaults
// to `staff`: the kill is public (a champion falling is announced in-world and
// is the content the board is for), the roll of who did the damage is not. A
// shard that wants a public board lowers one rule.
//
// Nested for the same reason `market.fees` and `houses.schedule` are: one rule
// covers the whole table rather than a rule per column, and the columns here
// are actor objects whose `acct`/`webId` remain admin-only by the locked-field
// rule regardless of what this is set to.
//
// `killer` is deliberately NOT listed. It is the single actor whose blow landed
// last, it is announced in-game to everyone present, and it is the same shape
// and the same disclosure `mob.killed` has published on the public activity
// feed since before this framework existed.
champs: { audience: 'anonymous', fields: { damagers: 'staff' } },
guilds: { audience: 'anonymous', fields: {} },
governors: { audience: 'anonymous', fields: {} },
// The public Houses page showed IDOC location only; owner/price were staff.
@@ -189,6 +209,10 @@ const KIND_FEATURE = new Map(
// boards
'champ.update': 'champs',
'champ.remove': 'champs',
// Protocol 6. Without this line rule 2 would fail the new kind closed to
// admin-only — correct as a default, and wrong as an outcome: a champion
// falling is exactly what the public board is for.
'champ.boss.killed': 'champs',
'guild.update': 'guilds',
'guild.remove': 'guilds',
'guild.join': 'guilds',
@@ -222,6 +246,14 @@ const KIND_FEATURE = new Map(
// needs it live. An admin can turn it on.
'vendor.listing': 'market',
'vendor.listing.remove': 'market',
// Protocol 6 part b's `lease.applied` and `lease.expired` are deliberately NOT
// here, on the same reasoning that keeps `account.login.result` off it. They are
// operational frames about the WEBSITE changing this shard's configuration --
// which key, from what to what, on whose run, and whether the shard's own
// deadline had to put it back because nobody asked. Rule 2 fails an unmapped
// kind closed to admin-only, which is where an audit trail of the site's writes
// belongs; mapping them would mean choosing a feature an operator could then
// widen, and there is no rung below admin these frames belong on.
}),
)

View File

@@ -351,10 +351,17 @@ function tagValue(block, name) {
* ~40 fields on every one of ~6,500 records to keep 14 of them. The records are
* flat, so a per-record regex sweep is both correct and cheap.
*
* Only the fields the site can actually show are kept. Everything to do with
* triggering, refractory windows, proximity, sequential spawning, sounds and
* `UniqueId` is dropped here rather than downstream — that is what holds the
* committed artifact under 1 MB.
* Only the fields the site can actually use are kept. Everything to do with
* triggering, refractory windows, proximity, sequential spawning and sounds is
* dropped here rather than downstream, which is what keeps the parsed atlas
* small.
*
* **`UniqueId` was on that list until Phase 12b and is now kept**, because a
* property lease has to name one particular spawner and this is the only name
* for one that exists off-shard. The line that justified dropping it cited a
* committed artifact; there is no committed artifact — `spawnAtlasSource.js`
* says so in its own header ("nothing is precomputed and committed") — so the
* only real cost was ~37 bytes a row in a table, and it bought a dropdown.
*
* NOTE: the facet comes from each record's own `<Map>`, never from the file
* name. `Eodon.xml`, `GravewaterLake.xml` and the other named-area files all
@@ -389,6 +396,14 @@ function parsePoints(source) {
points.push({
name: tagValue(block, 'Name'),
// **Kept from Phase 12b, having been discarded since the atlas shipped.**
// It is `XmlSpawner.UniqueId` — the shard writes it into the spawn files
// and carries it on the live spawner — so it is the ONE way an authoring
// form can name a particular spawner without the shard being up. A serial
// cannot do that job: serials are assigned when the world is built and
// nothing off-shard knows them, which is why a property lease that could
// only be addressed by serial could have no dropdown at all.
uniqueId: tagValue(block, 'UniqueId'),
facet,
x: toInt(tagValue(block, 'X')),
y: toInt(tagValue(block, 'Y')),
@@ -548,6 +563,48 @@ function walkLocations(node, facet, path, out) {
* A spawn with no `type` is randomised on every activation, which the site must
* render as "random" rather than as an empty type.
*/
/**
* Item types a shard uses as decoration, from one `Data/Decoration/*.cfg`.
*
* The format is a header line naming a type and an item id, optionally followed
* by a parenthesised property list, and then one `x y z` line per placement:
*
* ```
* # switch
* Static 0x108F
* 5552 1864 11
* ```
*
* Only the header matters here. The properties are decoration-authoring details
* (`Hue=`, `Facing=`, `Name=`) and the coordinates are where the SHARD put its
* own scenery, neither of which an event author is choosing — they pick a type
* and a place of their own.
*
* Returns one entry per header line, not per distinct type: the same type
* appears under many item ids (a `BarredMetalDoor` for each facing), and how
* often a shard reaches for something is worth keeping. `spawnAtlasSource`
* aggregates.
*/
function parseDecoration(source) {
const out = []
if (!source) return out
for (const raw of String(source).split(/\r?\n/)) {
const line = raw.trim()
// A coordinate line starts with a digit or a minus (z is often negative),
// so the type test is not merely "not a comment".
if (line === '' || line.startsWith('#')) continue
const match = /^([A-Za-z_][A-Za-z0-9_]*)\s+0x([0-9A-Fa-f]+)/.exec(line)
if (!match) continue
out.push({ type: match[1], itemId: parseInt(match[2], 16) })
}
return out
}
function parseChampions(source) {
const root = parseXml(source)
const champions = []
@@ -675,6 +732,7 @@ module.exports = {
parseRegions,
parseLocations,
parseChampions,
parseDecoration,
buildPlacementIndex,
resolveRegion,
facetKey,

View File

@@ -6,6 +6,15 @@
// - the server, which refreshes the atlas on boot (`shardAtlas.model.js`)
// - the CLI (`scripts/importSpawnAtlas.js`)
//
// **As of protocol 8 phase 7 it is no longer the only way in** (docs/link/v8.md
// §10). `treeBridge.js` reads the same five labelled groups off the SHARD, over
// the sidecar, and hands back files in exactly the shape `readSources` produces
// here — which is why `buildFromFiles` below is where the parse actually starts
// and both readers feed it. That closes the one place the platform's rule (only
// the sidecar bridges the shard) was broken, and broken by the component that
// faces the internet: this file's `SERVUO_PATH` required the WEBSITE to be able
// to read the shard's directories.
//
// The shard's own files are the single source of truth. Nothing is precomputed
// and committed, because a shard's maps change over its lifetime — facets get
// added, replaced or renamed — and a snapshot in the repo would silently go
@@ -23,6 +32,7 @@ const {
parseRegions,
parseLocations,
parseChampions,
parseDecoration,
buildPlacementIndex,
buildFacetIndex,
resolveFacetName,
@@ -35,6 +45,7 @@ const REGIONS_FILE = path.join('Data', 'Regions.xml')
const LOCATIONS_DIR = path.join('Data', 'Locations')
const SPAWNS_DIR = 'Spawns'
const CHAMPIONS_FILE = path.join('Config', 'ChampionSpawns.xml')
const DECORATION_DIR = path.join('Data', 'Decoration')
class AtlasSourceError extends Error {
constructor(message, code) {
@@ -46,8 +57,20 @@ class AtlasSourceError extends Error {
// ── Reading ────────────────────────────────────────────────────────────────
function sha256(text) {
return crypto.createHash('sha256').update(text, 'utf8').digest('hex')
/**
* The fingerprint of one source file, over its RAW BYTES.
*
* Bytes rather than the decoded string, so that this reader and the bridge
* reader cannot disagree. The shard hashes what it sends; a hash taken here over
* `text` would be a hash of a UTF-8 RE-ENCODING of what was read — identical for
* every valid UTF-8 file, and different for one that is not, because Node's utf8
* decode replaces each undecodable byte with U+FFFD and the re-encode never gets
* them back. A spawn file with one Latin-1 character in a creature name would
* then fingerprint differently depending on which end read it, and the drift gate
* would report a change on every single import, forever, with the tree untouched.
*/
function sha256(bytes) {
return crypto.createHash('sha256').update(bytes).digest('hex')
}
function listXml(dir) {
@@ -62,9 +85,36 @@ function listXml(dir) {
}
}
/**
* Every `.cfg` under `dir`, recursively, tree-relative and forward-slashed.
*
* Recursive because `Data/Decoration` nests two deep in places
* (`Magincia/Trammel`, `Stygian Abyss/Ter Mur`, `Old/Britannia`) and a flat read
* would silently index a third of what the shard actually has — the failure
* mode being a dropdown that is quietly missing whole expansions rather than an
* error anyone would notice.
*/
function listCfgTree(dir, prefix = '') {
let entries
try {
entries = fs.readdirSync(dir, { withFileTypes: true })
} catch (err) {
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') return []
throw err
}
const out = []
for (const entry of entries.sort((a, b) => a.name.localeCompare(b.name))) {
const rel = prefix ? `${prefix}/${entry.name}` : entry.name
if (entry.isDirectory()) out.push(...listCfgTree(path.join(dir, entry.name), rel))
else if (entry.name.toLowerCase().endsWith('.cfg')) out.push(rel)
}
return out
}
function readIfPresent(file) {
try {
return fs.readFileSync(file, 'utf8')
return fs.readFileSync(file)
} catch (err) {
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') return null
throw err
@@ -89,9 +139,14 @@ function readSources(root) {
const files = []
const push = (label, file) => {
const text = readIfPresent(file)
if (text === null) return false
files.push({ label, text, sha256: sha256(text), bytes: Buffer.byteLength(text, 'utf8') })
const bytes = readIfPresent(file)
if (bytes === null) return false
files.push({
label,
text: bytes.toString('utf8'),
sha256: sha256(bytes),
bytes: bytes.length,
})
return true
}
@@ -111,6 +166,12 @@ function readSources(root) {
push('Config/ChampionSpawns.xml', path.join(root, CHAMPIONS_FILE))
// Optional, like the champion file: a shard that has stripped its decoration still
// has a usable atlas, it just cannot offer the decoration verb anything to place.
for (const rel of listCfgTree(path.join(root, DECORATION_DIR))) {
push(`Data/Decoration/${rel}`, path.join(root, DECORATION_DIR, rel))
}
return { files }
}
@@ -140,8 +201,30 @@ function hashSources(root) {
*
* 2 — respawn delays normalised to seconds (they are per-record minutes OR
* seconds in the source, decided by `DelayInSec`).
* 3 — the decoration index, from `Data/Decoration/**\/*.cfg`.
* 4 — a spawn point keeps its `UniqueId`, which is what a property lease
* targets (Phase 12b). The bump is what re-reads a tree the boot path
* would otherwise skip on an unchanged hash — the source files have not
* changed, only what is kept from them.
* 5 — and it did NOT keep it: version 4 bumped the parser and the aggregator
* below still discarded the field, so the intent above shipped as a
* comment. This bump is what makes an already-imported tree re-read now
* that the mapping keeps it; without it `sameSources` sees an unchanged
* tree and every existing install stays empty. Released in v1.2.2.
* 6 — source files are parsed in one canonical label order (protocol 8 phase
* 7). The decoration index keeps the first item id it sees for a type, so
* the read order decided a preview graphic; it now cannot differ between a
* tree read off a disk and the same tree read over the bridge. Identical
* sources, and for a handful of types a different answer, which is exactly
* what this number exists to make reach an install.
*
* This was written as 5 on `edge` while 5 was being released from `main`
* meaning something else, so the cutover renumbered it: an install that
* imported under v1.2.2 already stores 5, and had the number not moved,
* `sameSources` would have called that tree current and this change would
* have reached nobody who was already running.
*/
const PARSER_VERSION = 2
const PARSER_VERSION = 6
/** True when two source fingerprints describe the same tree. */
function sameSources(a, b) {
@@ -211,8 +294,45 @@ function aggregateCreatures(points) {
* here writes. `shardAtlas.model.js` decides what to do with the result.
*/
function buildAtlas(root, options = {}) {
const { files } = readSources(root)
return buildFromFiles(readSources(root).files, options)
}
/**
* The parse itself, over files that have already been read.
*
* Split out in phase 7 so that a tree which arrived over the sidecar and a tree
* read off a local disk go through the SAME code from here on. The alternative —
* a second build for the bridge — would have been a second place for the facet
* reconciliation, the decoration case-folding and the disabled-spawner filter to
* be subtly different, and the difference would only ever show up as one install
* having a slightly wrong atlas.
*/
function buildFromFiles(files, options = {}) {
// **Sorted here, once, whatever order the reader handed them over in.**
//
// Order is not cosmetic in this parse: the decoration index keeps the FIRST
// item id it sees for a type and the first spelling of it, and `meta.source` is
// written in iteration order. Both readers happen to agree on a stock tree, and
// "happen to" is the problem — the filesystem reader walks each decoration
// directory with `localeCompare` while the shard sorts whole relative paths,
// and those two disagree the moment a directory mixes cases. A tree read over
// the bridge would then produce a subtly different atlas from the same tree read
// off a disk, in a way nothing reports and only a side-by-side diff would find.
//
// A plain ordinal comparison rather than `localeCompare`, because the answer
// must not depend on the host's ICU data either.
files = [...files].sort((a, b) => (a.label < b.label ? -1 : a.label > b.label ? 1 : 0))
const byLabel = new Map(files.map((file) => [file.label, file]))
if (!byLabel.has('Data/Regions.xml')) {
throw new AtlasSourceError('Missing required file: Data/Regions.xml', 'NO_REGIONS')
}
if (!files.some((file) => file.label.startsWith('Spawns/'))) {
throw new AtlasSourceError('No spawn files found in Spawns', 'NO_SPAWNS')
}
const source = {}
for (const file of files) source[file.label] = { bytes: file.bytes, sha256: file.sha256 }
@@ -264,6 +384,16 @@ function buildAtlas(root, options = {}) {
const place = resolveRegion(point.x, point.y, point.facet, placement, resolveOpts)
return {
name: point.name,
// **The field this whole `PARSER_VERSION` note was about, and it was
// dropped right here.** The parser has produced it since Phase 12b and
// the column and the query have both been waiting for it, but this
// mapping rebuilds each point from an explicit field list and `uniqueId`
// was not on it — so every row landed with `unique_id` NULL, and
// `listSpawners`, whose WHERE is `unique_id IS NOT NULL`, could only ever
// answer empty. That made `uo.options.spawners` an empty dropdown and
// every Phase 12b object-property lease unauthorable, with nothing on the
// form to say why. Found by the Phase 16b walk against a released bundle.
uniqueId: point.uniqueId,
facet: point.facet,
x: point.x,
y: point.y,
@@ -294,6 +424,43 @@ function buildAtlas(root, options = {}) {
}
})
// Decoration: what this shard already calls scenery, which is what makes the
// authoring dropdown the operator's own vocabulary rather than our taste.
//
// **Keyed case-INSENSITIVELY, because the decoration files disagree with
// themselves about casing.** Stock 57.4 names four types under two spellings
// each — `CheckerBoard`/`Checkerboard`, `ChessBoard`/`Chessboard`,
// `MetalChest`/`Metalchest`, `SpinningWheelEastAddon`/`SpinningwheelEastAddon`
// — and in every pair exactly one is a real class, the other a mis-cased line
// the shard's own loader resolves anyway. A case-sensitive Map keeps both, and
// then `shard_decor_types.type` (a PRIMARY KEY under MariaDB's default
// `..._ai_ci` collation, which folds case) rejects the second row and takes the
// WHOLE import transaction down with it. That is not a decoration bug: with no
// atlas, every option source answers empty and no world verb can be authored at
// all. The shard end of this feature already knew — `BridgeWorld.cs` resolves a
// decor type with `FindTypeByName(name, ignoreCase: true)` and says why — so
// folding here is the two ends agreeing rather than a new rule.
//
// The first spelling seen wins, exactly as the first item id does. Either
// spelling resolves on the shard, so which one survives is cosmetic.
const decorUses = new Map()
for (const file of files) {
if (!file.label.startsWith('Data/Decoration/')) continue
for (const entry of parseDecoration(file.text)) {
const key = entry.type.toLowerCase()
const seen = decorUses.get(key)
if (seen) {
seen.uses += 1
continue
}
// The FIRST item id wins, and it is only a preview: a type appears under
// as many ids as it has facings or variants, and picking one arbitrarily
// is honest in a way that picking "the most used" would not be.
decorUses.set(key, { type: entry.type, itemId: entry.itemId, uses: 1 })
}
}
const decor = [...decorUses.values()].sort((a, b) => a.type.localeCompare(b.type))
const creatures = aggregateCreatures(points)
const facets = [...new Set(points.map((point) => point.facet))].sort()
const unresolved = points.filter((point) => !point.region && !point.landmark).length
@@ -311,6 +478,7 @@ function buildAtlas(root, options = {}) {
regions: regions.length,
landmarks: landmarks.length,
champions: champions.length,
decor: decor.length,
unresolvedPoints: unresolved,
},
source,
@@ -321,9 +489,53 @@ function buildAtlas(root, options = {}) {
landmarks,
champions,
points,
decor,
}
}
// ── Backends ───────────────────────────────────────────────────────────────
//
// One descriptor, two readers (§10, phase 7). A source is `{ kind: 'fs', root }`
// or `{ kind: 'bridge' }`, and everything above this line belongs to the first.
//
// `treeBridge` is required lazily and INSIDE the functions rather than at the top
// of the file, because this module is also loaded by `scripts/importSpawnAtlas.js`
// and by tests that have no sidecar, no core logger and no intention of touching
// either. A top-level require would drag the whole client stack into both.
/** `{ files }` from whichever end this source names. */
async function readFrom(source) {
if (source?.kind === 'bridge') {
const { files } = await require('./treeBridge').readSources()
return { files }
}
return readSources(source?.root)
}
/**
* The `{ label: sha256 }` fingerprint, from whichever end.
*
* The bridge answers this from the MANIFEST alone — no file bytes cross the wire
* to answer "has anything changed", which is the whole reason the manifest is a
* separate call. A stock tree is one page and about 32 KB.
*/
async function hashFrom(source) {
if (source?.kind === 'bridge') {
const treeBridge = require('./treeBridge')
const listing = await treeBridge.manifest()
return treeBridge.fingerprintOf(listing.files)
}
return hashSources(source?.root)
}
/** The full atlas, from whichever end. */
async function buildFrom(source, options = {}) {
const { files } = await readFrom(source)
return buildFromFiles(files, options)
}
module.exports = {
AtlasSourceError,
PARSER_VERSION,
@@ -331,6 +543,10 @@ module.exports = {
hashSources,
sameSources,
buildAtlas,
buildFromFiles,
readFrom,
hashFrom,
buildFrom,
aggregateCreatures,
displayName,
}

418
server/utils/treeBridge.js Normal file
View File

@@ -0,0 +1,418 @@
// Spawn atlas sources — the SHARD half (docs/link/v8.md §10, protocol 8 phase 7).
//
// `spawnAtlasSource.js` reads a ServUO tree off a filesystem and has done since
// the atlas existed. That is the half this replaces, and it is worth being blunt
// about what was wrong with it: `SPAWN_ATLAS.md` required the WEBSITE to be able
// to read the shard's directories — "the same host, a bind mount, or a shared
// volume". Everything else about this platform holds that only the sidecar
// bridges the shard, and that one requirement broke the rule using the component
// that faces the internet.
//
// So the shard now serves its own files over the same request/reply path as
// every other shard read, and `SERVUO_PATH` becomes what it should always have
// been: the development and same-host convenience, not the design.
//
// **The parsers do not move.** `spawnAtlasParse.js` is pure, fs-free and covered
// by CI without a ServUO tree anywhere near it, and every quirk it handles — the
// two respawn delay units, `:OBJ=` splitting, facet-name reconciliation, the
// XmlSpawner directive stripping — stays exactly where it is. The shard sends
// bytes; the website still decides what they mean.
//
// ── Why a file arrives in pieces ──────────────────────────────────────────
//
// §10 said the shard would serve `tree/<label>` → bytes, and phase 7 measured
// that it cannot. A stock `Spawns/trammel.xml` is 4.03 MB; the sidecar discards
// any inbound line over 1 MiB; that file as a single base64 row is 5.4 MiB. It
// would never arrive — the reply would be dropped, the request would time out,
// and the import would retry forever with no error in it anywhere. Two files on
// a STOCK tree are in that state.
//
// A file therefore crosses as chunks, each gzipped:
//
// tree/Spawns/trammel.xml the manifest row — size, hash, chunk count
// tree/Spawns/trammel.xml/c0 the first 512 KiB of it, gzipped
//
// Measured on the stock 57.4 tree: 141 files, 11.34 MB, 158 chunks, three pages,
// 1.33 MB actually on the wire.
//
// ── The three checks below that are not decoration ────────────────────────
//
// Every one of them catches a way this can end in a tree that LOOKS imported:
//
// - **Each chunk re-declares its own address** and carries the hash of its own
// uncompressed bytes. A reassembly that put chunk 3 where chunk 4 belongs
// would produce XML that still parses — XML is forgiving about what it skips
// — and an atlas quietly missing spawns.
// - **The whole file is hashed after reassembly** against what the manifest
// said, which is also the fingerprint the drift gate stores.
// - **The catalog must not move mid-walk.** An operator editing a spawn file
// while this runs would otherwise produce one atlas stitched out of two
// trees, with nothing anywhere reporting a problem.
// Required as a namespace, not destructured: a test that stubs the sidecar
// replaces these on the module object, and a destructured copy taken at load
// time would keep calling the real one.
const zlib = require('zlib')
const uoLinkClient = require('./uoLinkClient')
const assetBridge = require('./assetBridge')
const log = require('../core').logger('tree-bridge')
/** The §5 key family the shard serves these under. */
const FAMILY = 'tree'
// How many chunk keys go in one `assets.fetch`. The shard cuts the PAGE by byte
// budget within whatever it is asked for, so this only bounds the request; a
// stock tree's 158 chunks fit in a single one.
const FETCH_CHUNK = 200
// Bounds on the walk. Neither is expected to be reached on any real tree — the
// stock one is 141 files and 158 chunks — and both exist so that a shard
// answering nonsense costs a bounded amount of memory rather than all of it.
const MAX_FILES = 20000
const MAX_BYTES = 256 * 1024 * 1024
class TreeBridgeError extends Error {
constructor(message, code) {
super(message)
this.name = 'TreeBridgeError'
this.code = code
}
}
/**
* Recast an asset-plane failure as one of ours.
*
* The distinction worth keeping is 403: on this family it does NOT mean the
* operator declined to serve their client files, it means they declined to serve
* their own configuration tree — a different switch (`Bridge.TreeEnabled`) with
* a different fix, and telling them to look at the wrong one costs them an
* afternoon.
*/
function rethrow(err, what) {
if (!(err instanceof assetBridge.AssetBridgeError)) return err
if (err.code === 'DISABLED') {
return new TreeBridgeError(
'The shard is refusing to serve its configuration tree (Bridge.TreeEnabled is off): '
+ err.message,
'DISABLED',
)
}
return new TreeBridgeError(`${what}: ${err.message}`, err.code)
}
/**
* Stage 1: every atlas source file the shard has, with its hash — and no bytes.
*
* Returns `{ catalog, chunkBytes, files: [{ key, label, bytes, mtime, chunks,
* sha256 }] }`.
*
* This is the whole of the drift gate. The website stores these hashes; the next
* import asks for this list again and fetches nothing at all when nothing moved,
* which on a shard whose maps are not being edited is every import. One page and
* about 32 KB on a stock tree.
*/
async function manifest() {
const started = Date.now()
const files = []
let cursor = null
let pages = 0
let catalog = null
let chunkBytes = 0
let finished = false
while (pages < assetBridge.MAX_PAGES) {
let page
try {
page = await assetBridge.withBusyRetry(
() => uoLinkClient.getAssetManifest({ family: FAMILY, cursor }),
'the shard configuration tree',
)
} catch (err) {
throw rethrow(err, 'reading the tree manifest')
}
pages++
if (catalog === null) {
catalog = page.catalog ?? null
chunkBytes = Number(page.chunkBytes) || 0
} else if (page.catalog !== catalog) {
throw new TreeBridgeError(
"The shard's configuration tree changed while it was being listed; nothing was imported",
'SOURCE_CHANGED',
)
}
for (const row of page.rows ?? []) {
const label = String(row?.label ?? '')
if (label === '') continue
files.push({
key: String(row?.key ?? `${FAMILY}/${label}`),
label,
bytes: Number(row?.bytes) || 0,
mtime: Number(row?.mtime) || 0,
chunks: Number(row?.chunks) || 0,
sha256: row?.sha256 ? String(row.sha256) : null,
})
}
if (files.length > MAX_FILES) {
throw new TreeBridgeError(
`The shard listed more than ${MAX_FILES} tree files; refusing to keep reading`,
'TOO_LARGE',
)
}
let state
try {
state = assetBridge.checkPage(page, { arrayName: 'rows', cursor, pages, noun: 'tree' })
} catch (err) {
throw rethrow(err, 'reading the tree manifest')
}
if (state.done) {
finished = true
break
}
cursor = state.cursor
}
if (!finished) {
throw new TreeBridgeError(
`The tree manifest did not end within ${assetBridge.MAX_PAGES} pages; nothing was imported`,
'TOO_LARGE',
)
}
log.info('tree manifest read from the shard', {
files: files.length,
catalog,
pages,
ms: Date.now() - started,
})
return { catalog, chunkBytes, files, pages }
}
/** A manifest as the `{ label: sha256 }` fingerprint the atlas model stores. */
function fingerprintOf(list) {
const hashes = {}
for (const file of list) hashes[file.label] = file.sha256
return hashes
}
/**
* Stage 2: the bytes.
*
* Returns `{ files: [{ label, text, sha256, bytes }] }` — deliberately the exact
* shape `spawnAtlasSource.readSources` returns from a filesystem, so that
* `buildAtlas` cannot tell which end a tree arrived from and nothing downstream
* has a second code path to be wrong in.
*/
async function readSources() {
const started = Date.now()
const listing = await manifest()
if (listing.files.length === 0) {
throw new TreeBridgeError(
'The shard served no atlas source files at all (is this a ServUO tree?)',
'NO_SOURCE',
)
}
const expected = listing.files.reduce((sum, file) => sum + file.bytes, 0)
if (expected > MAX_BYTES) {
throw new TreeBridgeError(
`The shard's tree is ${expected} bytes, over the ${MAX_BYTES} this will read`,
'TOO_LARGE',
)
}
const keys = []
for (const file of listing.files) {
// A zero-length file is still ONE chunk. An overlay that said zero would
// leave a manifest row nothing could ever fetch, and the walk below would
// report the import incomplete forever.
const chunks = Math.max(1, file.chunks)
for (let i = 0; i < chunks; i++) keys.push(`${file.key}/c${i}`)
}
const parts = new Map()
let pages = 0
let wire = 0
for (let i = 0; i < keys.length; i += FETCH_CHUNK) {
const batch = keys.slice(i, i + FETCH_CHUNK)
let cursor = null
let finished = false
let walked = 0
while (walked < assetBridge.MAX_PAGES) {
let page
try {
page = await assetBridge.withBusyRetry(
() => uoLinkClient.fetchAssets({ keys: batch, catalog: listing.catalog, cursor }),
'the shard configuration tree',
)
} catch (err) {
throw rethrow(err, 'reading the tree')
}
pages++
walked++
if (typeof page.catalog === 'string' && page.catalog !== '' && page.catalog !== listing.catalog) {
throw new TreeBridgeError(
`The shard's configuration tree changed mid-read (catalog ${listing.catalog} became ${page.catalog})`,
'SOURCE_CHANGED',
)
}
for (const row of page.rows ?? []) {
const key = String(row?.key ?? '')
if (row?.status !== 'ok') {
// Unlike an asset key, a tree key comes straight off a manifest this
// same walk just read. There is no such thing as an expected gap here:
// the shard listed the file, so a refusal means the tree moved or the
// two ends disagree about the key scheme, and importing the rest would
// silently drop whatever that file held.
throw new TreeBridgeError(
`The shard refused ${key || 'a tree chunk'} (${row?.status || 'unknown'}: `
+ `${row?.reason || 'no reason given'})`,
'INCOMPLETE',
)
}
const label = String(row.label ?? '')
const chunk = Number(row.chunk)
if (label === '' || !Number.isInteger(chunk) || chunk < 0) {
throw new TreeBridgeError(`The shard sent a tree chunk with no address (${key})`, 'MALFORMED')
}
let raw
try {
raw = zlib.gunzipSync(Buffer.from(String(row.gzip ?? ''), 'base64'))
} catch (err) {
throw new TreeBridgeError(`Could not decompress ${key}: ${err.message}`, 'MALFORMED')
}
const declared = Number(row.bytes)
if (Number.isFinite(declared) && declared !== raw.length) {
throw new TreeBridgeError(
`${key} declared ${declared} bytes and decompressed to ${raw.length}`,
'MALFORMED',
)
}
if (row.sha256 && assetBridge.sha256Of(raw) !== String(row.sha256)) {
throw new TreeBridgeError(`${key} does not match its own hash`, 'MALFORMED')
}
if (!parts.has(label)) parts.set(label, new Map())
parts.get(label).set(chunk, raw)
}
let state
try {
state = assetBridge.checkPage(page, { arrayName: 'rows', cursor, pages: walked, noun: 'tree' })
} catch (err) {
throw rethrow(err, 'reading the tree')
}
if (state.done) {
finished = true
break
}
cursor = state.cursor
}
if (!finished) {
throw new TreeBridgeError(
`A tree fetch did not end within ${assetBridge.MAX_PAGES} pages; nothing was imported`,
'TOO_LARGE',
)
}
}
const files = []
for (const file of listing.files) {
const chunks = Math.max(1, file.chunks)
const held = parts.get(file.label)
if (!held) {
throw new TreeBridgeError(`The shard sent nothing for ${file.label}`, 'INCOMPLETE')
}
const ordered = []
for (let i = 0; i < chunks; i++) {
const part = held.get(i)
// Indexed rather than appended in arrival order. The rows come back in the
// order they were asked for today, and a design that depends on that is one
// reordering away from an atlas that is wrong in a way nothing reports.
if (!part) {
throw new TreeBridgeError(`${file.label} is missing chunk ${i} of ${chunks}`, 'INCOMPLETE')
}
ordered.push(part)
}
const whole = Buffer.concat(ordered)
const sha256 = assetBridge.sha256Of(whole)
if (file.sha256 && sha256 !== file.sha256) {
throw new TreeBridgeError(
`${file.label} does not match the hash its manifest row carried`,
'MALFORMED',
)
}
files.push({
label: file.label,
text: whole.toString('utf8'),
sha256,
bytes: whole.length,
})
wire += whole.length
}
log.info('tree read from the shard', {
files: files.length,
bytes: wire,
chunks: keys.length,
pages,
ms: Date.now() - started,
})
return { files, catalog: listing.catalog, pages, chunks: keys.length }
}
module.exports = {
TreeBridgeError,
FAMILY,
FETCH_CHUNK,
MAX_FILES,
MAX_BYTES,
manifest,
fingerprintOf,
readSources,
}

View File

@@ -11,11 +11,48 @@
// `X-UOLink-Version: <protocol>` so a protocol mismatch is caught (409) rather
// than mis-parsed. Config is cached for a few seconds to avoid decrypting the
// token on every call.
//
// ── Protocol 6: `idempotencyKey` on a write ────────────────────────────────
//
// The three write helpers the event engine drives take an optional
// `idempotencyKey`, which the sidecar passes to the shard verbatim. The shard
// executes a key at most once and answers a repeat with the ORIGINAL reply, which
// is what makes retrying a world write safe — before it, a lost acknowledgement
// and a command that never applied were the same event seen from here.
//
// **A key is a function of the caller's unit of work, never of the attempt.** The
// event runner derives it from `sha256(runId|stepId)`, so every retry of one step
// carries the same key and a different step never collides with it. Passing a
// fresh value per call would satisfy the type and defeat the entire mechanism.
//
// **The DELETEs deliberately take no key.** Their idempotency is inherent — the
// second removal of a town-crier entry or a news article is a no-op the shard is
// already happy to perform — and the sidecar builds those commands from the path
// rather than from a body, so carrying one would be a protocol change bought for
// a guarantee that already holds.
//
// A caller that sends no key gets exactly the pre-protocol-6 behaviour, which is
// what leaves the admin screens (which send none, being driven by a human who can
// see whether the thing happened) unchanged.
//
// One new status can now come back from a keyed write: **425**, the sidecar's
// mapping of `bridge.busy` — a command under this key is still in flight on the
// shard. It is transient and retryable, and `shardAnnounce.classify` already
// treats it so by falling through to its retry case.
const uoLinkConfig = require('../model/uoLinkConfig/uoLinkConfig.model')
const log = require('../core').logger('uo-link-client')
const TIMEOUT_MS = 12000 // sidecar waits up to 10s on the shard before 504
// The sidecar waits up to 10s on the shard before answering 504, so this sits
// just above it — every call answers rather than being abandoned mid-flight.
//
// **Exported because the event actions are declared against it** (EVENTS_PLAN.md
// Phase 9). An action's `budgetMs` must exceed this or core's dispatch deadline
// fires first and classifies the step `retry` without asking the module, which
// for a broadcast means announcing twice. `config/uoEventActions.js` states that
// relationship and its test asserts it, and both need the number to come from
// here rather than from a copy that can drift.
const TIMEOUT_MS = 12000
const CONFIG_TTL_MS = 5000
let cachedConfig = null
@@ -141,6 +178,74 @@ const getPointsBoard = (system) => call(`/points/${encodeURIComponent(system)}`)
const getMarket = ({ limit = 200, offset = 0 } = {}) =>
call(`/market?limit=${encodeURIComponent(limit)}&offset=${encodeURIComponent(offset)}`)
// ── Protocol 8: the Asset Bridge (docs/link/v8.md) ────────────────────────
//
// The shard reads the operator's own UO client files and hands the results over
// this link, which is why nobody has to install UOFiddler any more.
// Stage 1 of the import gate: what those client files currently ARE — size, mtime
// and content hash of each, plus the version of the shard's extractor that would
// read them. No pixels and no strings cross on this call; its whole job is to let
// the site decide that nothing has changed and stop, which is the normal case on
// every restart.
//
// `sha256` comes back NULL for a file the shard has not hashed yet (anim.mul is
// 195 MB and hashing it cannot fit in a reply), with `hashing: true` alongside.
// That is "ask again in a moment", not "the file changed".
const getAssetSources = () => call('/assets/sources')
// The cliloc table out of the shard's own client, PAGED: each reply carries `rows`
// plus `more` / `cursor` / `cut`, and the caller echoes the cursor back until a
// reply says `more: false`. Only `cut: 'end'` means the table is finished — a short
// page can equally mean the byte budget was spent.
//
// `clilocBridge.js` is the thing that walks it; nothing else should call this
// directly, because a half-walked table is worse than none.
const getClilocTable = ({ lang, cursor } = {}) => {
const params = new URLSearchParams()
if (lang) params.set('lang', lang)
if (cursor) params.set('cursor', cursor)
const qs = params.toString()
return call(`/cliloc${qs ? `?${qs}` : ''}`)
}
// Stage 2 of the import gate, PAGED: every asset the shard could serve, with a
// hash and a size and no pixels. The website diffs it against what it holds and
// fetches only the keys whose hash moved — which on an ordinary restart is none
// of them, and is the whole difference between an Update and a re-download.
//
// This family pages on the shard's WALL CLOCK rather than on bytes: its rows are
// ~90 bytes, but building one means decoding a sprite, so a page ends when the
// shard's scan budget is spent (`cut: 'limit'`) far more often than when the byte
// budget is (`cut: 'budget'`). Neither means finished; only `cut: 'end'` does.
const getAssetManifest = ({ family, cursor } = {}) => {
const params = new URLSearchParams()
if (family) params.set('family', family)
if (cursor) params.set('cursor', cursor)
const qs = params.toString()
return call(`/assets/manifest${qs ? `?${qs}` : ''}`)
}
// The pixels, for keys the caller names. POST because the key list IS the request.
//
// `catalog` is the mid-import guard and should always be passed: it is an id the
// manifest derived from the client files themselves, and handing it back makes the
// shard refuse (422) if those files moved in between. Without it, an operator who
// patched their client halfway through an import gets one asset set stitched out
// of two, with nothing anywhere reporting a problem.
const fetchAssets = ({ keys, catalog, cursor } = {}) =>
call('/assets/fetch', { method: 'POST', body: { keys, catalog, cursor } })
// Slug → body id (docs/link/v8.md §8). The atlas knows a creature by its ServUO
// class name; the client knows it by a body id; nothing in the ServUO tree
// declares the mapping, so the shard answers it by constructing the creature and
// reading `Body.BodyID`.
//
// That runs on the shard's CORE THREAD, so the batch is small and the shard
// refuses an over-long list rather than truncating it. `assetBridge.js` chunks;
// nothing else should call this directly.
const resolveBodies = (types) => call('/assets/bodies', { method: 'POST', body: { types } })
// ── Commands ──────────────────────────────────────────────────────────────
const confirmLink = (code, websiteUserId) =>
call('/link/confirm', { method: 'POST', body: { code, websiteUserId: String(websiteUserId) } })
@@ -159,15 +264,18 @@ const createAccount = ({ actor, account, password, websiteUserId, ip }) =>
})
const unlinkAccount = ({ actor, account }) =>
call(`/link/${encodeURIComponent(account)}`, { method: 'DELETE', body: { actor } })
const postTownCrier = ({ id, lines, durationSec }) =>
call('/towncrier', { method: 'POST', body: { id, lines, durationSec } })
const postTownCrier = ({ id, lines, durationSec, idempotencyKey }) =>
call('/towncrier', { method: 'POST', body: { id, lines, durationSec, idempotencyKey } })
const deleteTownCrier = (id) => call(`/towncrier/${encodeURIComponent(id)}`, { method: 'DELETE' })
// Town Cryer News gump (Protocol 2.1). A full article (title/HTML body/image/URL)
// in the in-game News window; re-posting the same id REPLACES it. `announce`
// (default true on the sidecar) controls whether the criers proclaim the title.
const postNews = ({ id, title, body, image, url, announce }) =>
call('/news', { method: 'POST', body: { id: String(id), title, body, image, url, announce } })
const postNews = ({ id, title, body, image, url, announce, idempotencyKey }) =>
call('/news', {
method: 'POST',
body: { id: String(id), title, body, image, url, announce, idempotencyKey },
})
const deleteNews = (id) => call(`/news/${encodeURIComponent(id)}`, { method: 'DELETE' })
// ── Staff write plane (§6) ─────────────────────────────────────────────────
@@ -180,15 +288,166 @@ const adminBan = ({ actor, account, serial, durationSec, reason }) =>
call('/admin/ban', { method: 'POST', body: { actor, account, serial, durationSec, reason } })
const adminUnban = ({ actor, account }) =>
call('/admin/unban', { method: 'POST', body: { actor, account } })
const adminBroadcast = ({ actor, text, hue }) =>
call('/admin/broadcast', { method: 'POST', body: { actor, text, hue } })
const adminBroadcast = ({ actor, text, hue, idempotencyKey }) =>
call('/admin/broadcast', { method: 'POST', body: { actor, text, hue, idempotencyKey } })
// ── The event plane (protocol 6, EVENTS_PLAN.md Phase 11b) ─────────────────
//
// Leases and the run-scoped participation ledger. Both are gated on the shard by
// `Bridge.EventsEnabled`, which is deliberately NOT the admin plane's switch: an
// operator consenting to staff moderation from a screen has not thereby consented
// to the website changing their world on a schedule at four in the morning. A
// shard with the plane off answers 403, and the actions turn that into a refusal
// an author can read rather than a retry.
// Every lease this shard offers, with what each is worth right now and what is
// holding it. One read serves both questions core asks — `read()` wants the
// current value, `inForce()` wants to know whether the shard still has a record
// of the hold — so a lease costs one round trip, not two.
// **A targeted lease must name its target here** (protocol 7 part b). A key like
// `Spawner.MaxCount` is one capability over thousands of spawners, so it has no
// single `current` and the catalog walk cannot fill one in — while `read()` needs
// exactly one value for exactly one target before it applies anything. Naming both
// narrows the frame to that row and fills it.
//
// The frame also carries `holds`: every hold this shard has, whatever key or
// target. A catalog walk enumerates the KEYS but can never enumerate the holds on
// a targeted one — there is no list of spawners to walk — so `inForce()` reads
// that rather than the row's `held` flag.
const getLeases = ({ key, target } = {}) => {
const params = new URLSearchParams()
if (key) params.set('key', key)
if (target) params.set('target', target)
const query = params.toString()
return call(query ? `/lease?${query}` : '/lease')
}
// `holdMs` is authoritative and `untilMs` is display only. An absolute deadline
// computed here and honoured there is a deadline measured against two clocks, and
// a shard running ten minutes fast would restore a ten-minute lease the moment it
// took it. Values cross as TEXT whatever the lease's declared type: `1200` and
// `1200.0` are one number to a JSON parser and two strings to a compare-and-set.
const applyLease = ({ key, target, value, holdMs, untilMs, runId, idempotencyKey }) =>
call('/lease', {
method: 'POST',
body: { key, target, value: String(value), holdMs, untilMs, runId, idempotencyKey },
})
// `expected` is what this run applied and `baseline` is what to put back, both out
// of core's ledger rather than the shard's memory — so a release still works after
// a reconnect, and a shard that has forgotten the lease entirely (a restart, which
// reverts every config lease by design) answers honestly instead of refusing.
const releaseLease = ({ key, target, expected, baseline, idempotencyKey }) =>
call('/lease/release', {
method: 'POST',
body: {
key,
target,
expected: expected == null ? undefined : String(expected),
baseline: baseline == null ? undefined : String(baseline),
idempotencyKey,
},
})
// The participation ledger. The area is a map, a point and a radius rather than a
// region name, because protocol 6's own walk established that the most specific
// region containing an event is routinely anonymous.
const openParticipation = ({ runId, map, x, y, radius, holdMs, idempotencyKey }) =>
call('/participation', {
method: 'POST',
body: { runId: String(runId), map, x, y, radius, holdMs, idempotencyKey },
})
// A POST for a read, and the reason is the phase's headline: on a well-attended
// run the shard walks its members across Core ticks rather than in one inbound
// call, so a repeat arriving mid-walk is answered `bridge.busy` (425). A read that
// can legitimately be refused as a repeat in flight is not a GET.
const snapshotParticipation = ({ runId, idempotencyKey }) =>
call(`/participation/${encodeURIComponent(runId)}/snapshot`, {
method: 'POST',
body: { idempotencyKey },
})
const closeParticipation = ({ runId, idempotencyKey }) =>
call(`/participation/${encodeURIComponent(runId)}/close`, {
method: 'POST',
body: { idempotencyKey },
})
// ── The world verbs (protocol 7) ───────────────────────────────
//
// One endpoint for five author-facing verbs. `what` is the discriminator, and the
// per-verb fields ride alongside it: `type`/`name`/`hue`/`spread` for creatures and
// decoration, the three multipliers for a boss, `greeting`/`lines` for an oracle,
// `target`/`holdMs` for a gate.
//
// The shard registers every serial it places against the run and persists that
// registry, which is what makes `despawnWorld` below safe to point at a list of
// serials: it can only delete what the run actually owns.
const spawnWorld = (body) => call('/world', { method: 'POST', body })
// What the run still owns. A GET, unlike the participation snapshot: it carries no
// idempotency key and the shard answers it in one pass. An unknown run answers with an
// empty hand rather than a 404 — "owns nothing" and "never heard of it" are the same
// fact once the registry is the only record, and they stay the same fact across a
// restart, because the registry is written by the same world save as the objects it
// describes.
const ownedWorld = ({ runId }) => call(`/world/${encodeURIComponent(runId)}`)
// Give back what the run owns. No `serials` means everything, which is the call
// teardown makes. The reply splits three ways: `removed` was deleted, `gone` was
// already absent (a player killed it — an ordinary success), and `refused` was never
// this run's to delete.
//
// **It takes no idempotency key, and the parameter is gone rather than optional.**
// It used to accept one, and `revertOwned` passed the step's — the key the SPAWN
// went out under. The shard's at-most-once store is keyed on the key alone, so the
// despawn was answered with the spawn's stored reply and nothing was ever deleted.
// A repeat despawn needs no key: the second pass answers `gone`, which both ends
// already treat as a success. Removed from the signature so it cannot be handed
// one again by accident.
const despawnWorld = ({ runId, serials }) =>
call(`/world/${encodeURIComponent(runId)}/despawn`, {
method: 'POST',
body: { serials },
})
// ── Help-page (support) queue commands (§6) ────────────────────────────────
const respondPage = (pageId, { message, close }) =>
call(`/pages/${encodeURIComponent(pageId)}/respond`, { method: 'POST', body: { message, close } })
const closePage = (pageId) => call(`/pages/${encodeURIComponent(pageId)}/close`, { method: 'POST' })
// ── The one-shots (protocol 7 part b, EVENTS_PLAN.md Phase 12b) ────────────
//
// Neither owned nor borrowed: done is done. Both are gated on the shard by the
// same `Bridge.EventsEnabled` as the rest of the plane.
// What this shard will actually build, with the bounds it will build within. The
// module holds the same allowlist for its dropdown, so the form still works with
// the shard down; this is what is true when that copy is wrong.
const getGrantCatalog = () => call('/items')
// **The recipients are not sent.** The shard has held this run's participation
// ledger since it opened, keyed by the same character serials core stores as
// `member_key`, so the grant names a run and the shard resolves who was there.
// Sending a list would put the same list on the wire twice with a window in which
// the two disagree — and would have needed a core surface handing a module core's
// own participants.
const grantItem = ({ runId, item, amount, hue, name, where, idempotencyKey }) =>
call('/items/grant', {
method: 'POST',
body: { runId: String(runId), item, amount, hue, name, where, idempotencyKey },
})
// Starts a save. What actually happened rides `world.save.before`/`after` on the
// event stream, which have been there since protocol 2 — so this asserts only that
// the save was started, and a caller that needs the completion watches the feed it
// is already connected to.
const saveWorld = ({ idempotencyKey } = {}) =>
call('/world/save', { method: 'POST', body: { idempotencyKey } })
module.exports = {
TIMEOUT_MS,
invalidateConfig,
health,
getCharBySerial,
@@ -207,6 +466,11 @@ module.exports = {
getPoints,
getPointsBoard,
getMarket,
getAssetSources,
getClilocTable,
getAssetManifest,
fetchAssets,
resolveBodies,
confirmLink,
linkLookup,
createAccount,
@@ -215,6 +479,18 @@ module.exports = {
deleteTownCrier,
postNews,
deleteNews,
getLeases,
applyLease,
releaseLease,
openParticipation,
snapshotParticipation,
closeParticipation,
spawnWorld,
ownedWorld,
despawnWorld,
getGrantCatalog,
grantItem,
saveWorld,
adminKick,
adminBan,
adminUnban,

File diff suppressed because it is too large Load Diff