RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Compare commits

base: RunicGateway:b5233363137f8d77e2256c87ebae16c9e85e0487
Branches Tags
RunicGateway:main RunicGateway:chore/sync-installer-tree RunicGateway:docs/installer-phase-0.4 RunicGateway:docs/installer-plan RunicGateway:docs/android-v3-parity RunicGateway:docs/spawn-atlas RunicGateway:docs/ci-sonarqube RunicGateway:docs/android-m6-landed RunicGateway:chore/open-source-governance RunicGateway:feature/branding
..
compare: RunicGateway:docs/installer-phase-0.4
Branches Tags
RunicGateway:chore/sync-installer-tree RunicGateway:docs/installer-phase-0.4 RunicGateway:main RunicGateway:docs/installer-plan RunicGateway:docs/android-v3-parity RunicGateway:docs/spawn-atlas RunicGateway:docs/ci-sonarqube RunicGateway:docs/android-m6-landed RunicGateway:chore/open-source-governance RunicGateway:feature/branding

61 Commits
b523336313 ... docs/insta

Author SHA1 Message Date
wtclaude
d0363cd62d docs(installer): lead the remote-website case with a reverse proxy 
A TLS reverse proxy in front of the sidecar is a supported deployment already
running on a real domain, not the fallback the guide framed it as. Recommend it
first, keep [web] bind on loopback in that arrangement, and demote widen-the-
bind-and-firewall to the trusted-LAN alternative - on that path the token and
every event cross the network in the clear.

Adds the four things a proxy must actually do, checked against web.rs: forward
the WebSocket upgrade (/ws is the entire live feed, and losing it leaves REST
working with no events - a confusing half-working state); pass headers through
unmodified (auth is Authorization: Bearer or X-Api-Key, and a stripped
X-UOLink-Version silently skips the 409 mismatch check); no buffering and
long-lived connections (the sidecar pings every 30s, so a 60s+ read timeout is
safe as it stands); and no query-string logging, since ?token= is an accepted
auth form. Nothing needs X-Forwarded-For - the sidecar never uses the client IP
for authorization. Includes an nginx block that satisfies all four, and notes
Caddy and Traefik need no equivalent.

The website gets the proxied https:// / wss:// URLs, not the pair the installer
prints: those are composed from the sidecar's own bind address, which knows
nothing about what fronts it. PLAN records that the installer deliberately does
not try to detect a proxy - nothing visible from the sidecar's side says what is
in front of it, so guessing would print a confidently wrong URL.

Two new troubleshooting rows for the symptoms this causes: REST works but no
events (upgrade not forwarded), and a feed that drops every minute or two (read
timeout under the ping interval, or buffering).

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-04 12:12:46 -05:00
wtclaude
83bd4ec2d4 docs(installer): link the Phase 0.4 PR from the status table 2026-08-04 11:40:06 -05:00
wtclaude
5c8585fe75 docs(installer): add the operator install guide (Phase 0.4) 
Closes the last Phase 0 item. INSTALL.md is written before the installer
binary on purpose: everything it installs is already released (0.1-0.3), so
the guide is not speculation about a tool that might exist - it is the
specification of what the run asks, where it writes, what it prints, and what
the operator does next.

It is useful today. Appendix A is the same deployment done by hand - bundle
fetch, tarball verify and overlay copy, the optional patch tier,
--print-config provisioning, systemd unit / sc create - composed from the
released artifacts' actual contents and the sidecar's config and CLI source.
That appendix doubles as Phase 1's acceptance test.

Writing it settled four things the plan had left implicit, now recorded in
PLAN.md:

- The installer does not install itself; day-two commands run from the
  downloaded binary.
- The flag surface: --servuo, --patches/--no-patches, --host, --site-url and
  --yes join the --verify/--bundle/--purge the plan already named, so every
  prompt has a non-interactive equivalent.
- A modified Config/Bridge.cfg is reported, not overwritten - one deliberate
  deviation from deploy.ps1, whose overwrite-on-hash-differs rule is right for
  a developer and would silently revert an operator's whole shard config on
  update. install.json's recorded hashes are what make the distinction
  possible.
- Remote-website deployments: widen [web] bind, firewall it to the site's
  address, front it with TLS or a VPN off a trusted network. [shard] bind
  stays on loopback because that socket carries commands into the game.

Also adds the missing installer/ section to the docs index, and refreshes two
stale examples in PLAN.md (overlay file count, sidecar version).

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-04 11:39:02 -05:00
Colby Whitlock
fc79bb6ed0 Merge pull request 'docs(installer): record Phase 0.3 — the bundle CI and where bundles live' (#85) from docs/installer-phase-0.3 into main 
Reviewed-on: #85
 2026-08-04 16:20:05 +00:00
Colby Whitlock
af6036b9c7 Merge pull request 'docs(tree): sync link/PROJECT_TREE.md' (#83) from chore/sync-link-tree into main 
Reviewed-on: #83
 2026-08-04 16:17:52 +00:00
Colby Whitlock
e0245a209c Merge branch 'main' into chore/sync-link-tree 2026-08-04 16:17:39 +00:00
wtclaude
10ebce706f docs(installer): record Phase 0.3 — the bundle CI and where bundles live 
Documentation half of installer Phase 0.3. Code PRs: RunicGateway/installer#3,
RunicGateway/link#25, RunicGateway/servuo-plugins#9.

Phase 0 item 3 asked for a compose job, a nightly cron, and a dispatch step in
each component. All three landed, and building them settled questions §7 had
left open — so those sections are corrected rather than appended to.

Status: Phase 0 is now all but complete. 0.2 is merged and released as link
v1.1.0; 0.3 is in review; 0.4 (INSTALL.md) is the remaining item, and the shape
it was waiting on has settled.

Phase 0 item 3 gains an "As built" subsection matching items 1 and 2, recording
what was chosen rather than inherited: why gate 1 reads the sidecar's protocol
from source at the release tag instead of asking the binary via --print-config
(it only answers for v1.1.0+, and --bundle <tag> must be able to recompose an
older pair); why gate 2 records the hash CI computed itself and separately
checks for assets absent from SHA256SUMS; why release metadata is read
anonymously; why an unrecognized asset name is a hard failure; and why a run
that changes nothing writes nothing.

§7.1 is corrected in two places. The manifest shape now shows link.assets as a
map keyed by platform — the single sha256 this section sketched could only ever
have described one of the two binaries link publishes — plus schema/generated
and the servuo block. And it gains a subsection naming where bundles are
published: committed under bundles/ in the installer repo, NOT one Gitea release
per bundle, because that repo's own releases are the installer binaries and
/releases/latest returns whichever release is newest regardless of kind.

§7.2 records that the dispatch steps now exist, and that a failed dispatch is a
warning rather than a failed release — which is what keeps the installer repo's
token out of the components' hard requirements.

§7.3 records that the releasable-commit rule must also exclude merge commits,
whose subject quotes the real one: without that, merging a docs: branch whose
title mentions a fix: would re-dispatch a declining release workflow nightly.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-04 11:17:28 -05:00
Colby Whitlock
d1cb3e9511 Merge pull request 'docs(installer): record Phase 0.2 — the sidecar's CLI and settled data paths' (#84) from docs/installer-phase-0.2 into main 
Reviewed-on: #84
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
 2026-08-04 15:55:52 +00:00
wtclaude
c1957f0bfb docs(installer): record Phase 0.2 — the sidecar's CLI and settled data paths 
Phase 0.2 landed in link#24: the sidecar gained a four-flag CLI, --print-config,
and config-anchored data paths. Three sections of the installer plan asserted
facts that change as a result, so they are corrected rather than appended to.

- Status table: 0.1 merged (servuo-plugins#7/#8, overlay v0.1.1 released), 0.2 in
  review, repo bootstrap merged. 0.3 (bundle CI) is next and now unblocked — both
  components it composes exist.
- Phase 0 item 2 gains an "As built" subsection matching item 1's: why four
  hand-rolled flags rather than a parsing crate, why --print-config provisions
  instead of only reporting, why config_created/token_generated exist, and why no
  platform data directories are compiled into the binary.
- §2.3 (working-directory trap): half-closed in the sidecar — a relative
  [store].path now anchors to the config file's directory — while the service
  definitions still pin both env vars, and why that is not redundant.
- §2.4 (token handoff): the installer reads the handoff block out of one
  --print-config call and never parses the log, which is not a contract.
- §5 Phase 2 / Phase 4, §6: the ordering that follows (print-config before service
  registration), which doctor rows the CLI answers, that only the host is
  substituted into the printed URLs because web.bind is often 0.0.0.0, and that the
  printed token must not reach a log or support bundle.
- link/INTEGRATION.md §1: how to read the token back, replacing "the sidecar logs
  it" with the supported command and its output.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-04 10:48:44 -05:00
runic-docs-bot
b6343a0937 docs(tree): sync link/PROJECT_TREE.md from RunicGateway/link@654a08a [skip ci] 2026-08-04 15:48:24 +00:00
Colby Whitlock
e1608bb280 Merge pull request 'docs(installer): record Phase 0 progress and the overlay manifest' (#82) from docs/installer-phase0 into main 
Reviewed-on: #82
 2026-08-04 15:29:17 +00:00
wtclaude
f0975df598 docs(installer): record Phase 0 progress and the overlay manifest 
Tracks what actually landed while starting the installer plan, and corrects
the parts of the plan that the work proved wrong or stale.

Progress:

  A Phase 0 status table at the top, so the plan says where it is rather
  than needing a reader to reconstruct it from PR links.

  Phase 0 item 1 (§5) now records the release workflow as built, including
  its three deviations from link's copy — structural gates instead of build
  gates, no bump commit and therefore no push to main, and overlay.toml as
  the home for the declared protocol version. Plus the fixed tarball prefix
  and why: the installer would otherwise have to parse the version it is
  trying to read.

New §7.0 documents the overlay manifest as generated, and states plainly the
two things about it that carry weight: `protocol` is hand-maintained and has
to be (nothing in CI can derive it, which is exactly why §7.1's gate 1 has
something to compare), and `files` is what lets `doctor` distinguish
"operator edited a deployed file" from "the overlay moved on".

Corrections:

  §2.6 the plugin's protocol version now has a home (overlay.toml), and
        servuo-plugins now has a release workflow.
  §7.2  the dispatch step is deliberately deferred to Phase 0 item 3.
  §7.4  no longer "open risk" — the v3 cutover merged. The rule it motivated
        (never hardcode a protocol version) is restated as permanent rather
        than as a workaround for a mid-flight cutover.
  §8    open question 4 (branch targeting) resolved: servuo-plugins#6 merged,
        main == edge, everything targets main.

Version examples in §3, §5 and §7.1 said uo-link v3.x.y / 3.0.1, conflating
the release version with the protocol version. link is actually at v0.3.0 —
the two are independent, and the bundle names release versions, so an example
implying they track each other is actively misleading. Now uses the real
values (link 0.3.0, overlay 0.1.0, 30 overlay files).

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-04 09:28:23 -05:00
Colby Whitlock
8322e8318c Merge pull request 'docs(installer): add release orchestration and the bundle manifest' (#81) from docs/installer-release-orchestration into main 
Reviewed-on: #81
 2026-08-01 21:42:33 +00:00
wtclaude
25a5734107 docs(installer): add release orchestration and the bundle manifest 
The installer needs CI that reacts when a component publishes a release. Adds
that as section 7, folded into version tracking because the bundle IS the compat
matrix -- which closes the "where does the compat matrix live" gap section 7
previously left open.

- 7.1 Bundle manifest: CI publishes an exact, protocol-checked combination of
  component versions; the installer resolves against it at run time and
  --bundle <tag> pins one. A link release regenerates JSON and leaves the
  installer binary untouched, so operators don't re-download the installer for a
  sidecar patch and the repo doesn't accumulate releases with identical code.
  Two compose-time gates: sidecar PROTOCOL_VERSION must equal the overlay
  manifest's declared version, and every asset's SHA256 must match.
- 7.2 Triggers: each component's release job POSTs to the installer's
  workflow-dispatch endpoint (link's release.yml already declares
  workflow_dispatch and already holds a write:repository token), plus a nightly
  cron so a missed dispatch self-heals. repository_dispatch avoided -- support
  is uncertain on this Gitea version.
- 7.3 Stale overlay: dispatch, don't wait. Components self-release on merge to
  their own main, so the release normally already exists. If main is ahead with
  *releasable* commits (docs:/chore: correctly cut nothing), fire that repo's
  workflow, compose from what exists now, warn loudly, and let the nightly fold
  in the result. Dispatching another repo's workflow is fine -- it still runs
  its own gates -- but polling it is not, since Gitea's dispatch endpoint
  returns no run handle.

Bundle CI becomes a Phase 0 deliverable, since Phase 1 resolves what to install
from the bundle. `update` now moves between checked combinations rather than two
independently-latest artifacts.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-01 06:11:08 -05:00
Colby Whitlock
c583ddb77f Merge pull request 'docs(installer): plan the Runic Gateway installer' (#80) from docs/installer-plan into main 
Reviewed-on: #80
 2026-08-01 10:46:13 +00:00
wtclaude
29056ba996 docs(installer): plan the Runic Gateway installer 
Design of record for a deployment tool that takes a stock ServUO install and
configures it for Runic Gateway. Supersedes the informal overview it grew from,
which described a ServUO integration that does not match how servuo-plugins
actually ships.

Corrections that change the design:

- There is no RunicGateway.dll and no Plugins/ dir. The plugin ships as C#
  source compiled by ServUO at boot, so the step is a hash-compare sync of
  overlay/ -- but a successful copy does not mean a working bridge, because
  ScriptCompiler.Compile() ignores dotnet build's exit code and reloads the
  stale Scripts.dll.
- Stock ServUO files ARE modified, by three diffs in patches/. Made an opt-in,
  skippable tier: git apply against a hand-modified shard will fail, and the
  EventSink.cs patch needs a full core solution rebuild.
- Config paths collided with what the sidecar actually reads. Split ownership:
  sidecar.toml stays the sidecar's schema, install.json is the installer's.
  Service definitions pin UOLINK_CONFIG and UOLINK_DB_PATH, since the sidecar
  writes relative to CWD and would land in VirtualStore under Program Files.
- The token handoff was missing entirely -- the largest "installed it and
  nothing happened" failure mode.
- deploy.ps1 cannot be the cross-platform deployer; it stays the developer tool.

Decisions: public audience, unsigned binaries anchored on SHA256SUMS, Rust,
release-tarball plugin distribution, printed token handoff, new installer repo,
warn-and-skip on non-57.4 ServUO, and an uninstall that never edits the shard
tree -- it prints the files to delete and the hunks to revert.

Phases 0-5, with Phase 0 (a release workflow for servuo-plugins, which has
none today) gating everything else. Two open questions remain in section 8.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-08-01 05:44:44 -05:00
Colby Whitlock
186f057bc0 Merge pull request 'docs(tree): sync android/PROJECT_TREE.md' (#79) from chore/sync-android-tree into main 
Reviewed-on: #79
 2026-08-01 07:50:46 +00:00
runic-docs-bot
33a013ca98 docs(tree): sync android/PROJECT_TREE.md from RunicGateway/Android-app@5eaf5d2 [skip ci] 2026-08-01 07:22:23 +00:00
Colby Whitlock
5e2bc22a94 Merge pull request 'docs(tree): sync link/PROJECT_TREE.md' (#77) from chore/sync-link-tree into main 
Reviewed-on: #77
 2026-08-01 07:21:29 +00:00
Colby Whitlock
04cd64b838 Merge branch 'main' into chore/sync-link-tree 2026-08-01 07:21:09 +00:00
Colby Whitlock
916c11ee92 Merge pull request 'docs(tree): sync website/PROJECT_TREE.md' (#78) from chore/sync-website-tree into main 
Reviewed-on: #78
 2026-08-01 07:20:49 +00:00
runic-docs-bot
ea3755760d docs(tree): sync website/PROJECT_TREE.md from RunicGateway/website@5103b74 [skip ci] 2026-08-01 07:19:52 +00:00
runic-docs-bot
54b3002701 docs(tree): sync link/PROJECT_TREE.md from RunicGateway/link@295defb [skip ci] 2026-08-01 06:50:30 +00:00
Colby Whitlock
9d98109628 Merge pull request 'docs!: Protocol 3.0 cutover — the 3.0 documentation set' (#73) from edge into main 
Reviewed-on: #73
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
 2026-08-01 06:29:36 +00:00
Colby Whitlock
d0808b766b Merge branch 'main' into edge 2026-08-01 06:28:21 +00:00
Colby Whitlock
eab0a83f26 Merge pull request 'docs(link): the shard-name fallback, the atlas places shape, and two traps' (#76) from docs/protocol-3-smoke-findings into edge 
Reviewed-on: #76
 2026-08-01 06:02:59 +00:00
wtclaude
1444c77413 docs(link): the shard-name fallback, the atlas places shape, and two traps 
Records what the live Protocol 3.0 smoke test (ServUO + sidecar + website +
AVD) turned up, so none of it has to be rediscovered.

v3.md §5.3 — the ruleset `shard` field now falls back to the instance's own
name when the shard publishes ServUO's stock "My Shard", why that is done at
ingest rather than on read (the frame is also broadcast live), and why the
backfill snapshot must go through the dispatcher instead of writing state
directly: a direct call made it a second writer that skipped the
normalization.

v3.md §7.4 — an unscored board renders a placeholder row rather than a blank
card, and why it is deliberately not shaped like a real entry.

PLAN.md §9 M11 — `places` is a list of {facet,label,spawners,maxAlive} OBJECTS,
not of place-name strings, and typing it `List<String>` makes the whole detail
route fail to decode while the request itself returns 200. Adds the rule that
came out of it: decode tests must feed real captured JSON, because the fakes
build DTOs in Kotlin and can never catch a wire mismatch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP
 2026-08-01 00:58:53 -05:00
Colby Whitlock
2e24427032 Merge pull request 'docs(website): an operator runbook for extracting from your own UO client' (#75) from docs/operator-uofiddler-guide into edge 
Reviewed-on: #75
 2026-07-30 09:51:31 +00:00
wtclaude
afcdb373ec docs(website): an operator runbook for extracting from your own UO client 
CLILOCS.md and SPAWN_ATLAS.md each explain WHY the operator has to supply
something out of their own client, but neither says how. UOFIDDLER.md is the
missing procedure: where to get UOFiddler, which two files in the zip matter,
which runtime it needs, where Cliloc.enu actually lives, the conversion, how to
point the site at the result, and how to confirm it took.

Verified end to end on a stock Windows box: UOFiddler 4.22.2 (Ultima.dll is
net10.0), .NET SDK 9.0.312 building the net8.0 converter, RollForward carrying
it onto runtime 10.0.8, and the site's own parser reading the output back.

Corrects one claim while doing it. CLILOCS.md said a UOFiddler GUI export
"works equally well"; it does not. Its Cliloc tab writes `Number;Text;Flag` --
three columns, flag LAST -- and parseClilocText splits on the first separator
only, so the flag is absorbed into the name and every item renders as
`quarter staff;0`. The parser already handles `number,flag,text` with the flag
in the middle, but a trailing `;0` is indistinguishable from a name that
genuinely ends that way, so this stays a documented `sed` on the operator's
side rather than a heuristic that would corrupt real names.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-30 03:34:29 -05:00
Colby Whitlock
45fb4a3f15 Merge pull request 'docs(android): scope M11 — Protocol 3.0 shard parity for the app' (#74) from docs/android-v3-parity into edge 
Reviewed-on: #74
 2026-07-30 07:23:00 +00:00
wtclaude
5a091157d6 docs(android): scope M11 — Protocol 3.0 shard parity for the app 
The v3 work added four shard features and an admin-configurable visibility
framework the Android client knows nothing about. v3.md §10 deferred the app
side as a follow-up; re-examining it before the cutover found the gap is wider
than nav hiding:

  - no consumer for any of ruleset / leaderboards / market / atlas,
  - no `points` block on the character sheet (§7.3),
  - no cliloc-resolved item names (§8.6), and
  - shard nav gated on session role alone, so an admin who disables a feature
    or raises its audience leaves the app rendering entries that 404/403 into a
    generic error where the web client hides them.

Scoped as PLAN.md §9 M11 in two PRs (the visibility rules + read-model adds,
then the four screens), with the traps a real shard exposes recorded inline:
uncapped `maxPoints: 0`, cliloc-named boards with a null `nameString`, skill
caps in tenths, the required market staleness banner, the market stream being
off by default, atlas delays in seconds, and `points`-count vs `spawners`-list.

edge → main is held until both land so web and app surface the same shard on
the same day. Neither PR is coupled to the merge order — on a pre-v3 website
every new route and /public/shard/features 404s and the app falls back to
today's behavior — so holding the cutover is a schedule decision, not a
technical dependency.

Also records two things verified as already correct, so they are not
re-derived: the app's SSE request rides the authenticated client (same audience
rung as the same account on web), and every shard DTO is nullable-with-defaults
(field projection cannot cause a decode failure).

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-30 00:46:16 -05:00
Colby Whitlock
3a1bbdd165 Merge pull request 'docs(link): the Protocol 3.0 cutover (v3.md order 6)' (#72) from docs/protocol-3-cutover into edge 
Reviewed-on: #72
 2026-07-30 03:03:10 +00:00
wtclaude
32def88c4e docs(link): fill in the cutover PR numbers 
The order-6 row was written before the seven PRs existed. Same follow-up as the
cliloc row got.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-29 18:09:50 -05:00
wtclaude
71207cef16 docs(link): the Protocol 3.0 cutover (v3.md order 6) 
INTEGRATION.md was written for the window that just closed -- it told integrators
the version had NOT been bumped yet and that a sidecar on `edge` reports 2 while
already carrying v3 kinds. That guidance is now wrong in the direction that
matters, so the version section states 3 (header, /health, ws.hello, the 409
example and the §8 worked example) and replaces the "until then" paragraph with
what a v2 integration actually has to do to upgrade: change the constant it
sends, and nothing else, because nothing that existed in v2 changed shape.

v3.md gains §4.1 for what the bump touches and, more importantly, WHY the
website's boot migration is gated on a marker row: schema.sql is re-run on every
boot and uo_link_config.protocol is admin-editable, so an ungated UPDATE would
silently un-pin an operator running an older sidecar. That is the one piece of
the cutover a reader could not infer from the code being one constant.

Progress tables: 5b done, 6 in review.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-29 18:04:00 -05:00
Colby Whitlock
cdea1aa7cd Merge pull request 'docs(link): the player-vendor marketplace (Protocol 3.0 §8)' (#71) from docs/vendor-listing into edge 
Reviewed-on: #71
 2026-07-29 20:02:31 +00:00
wtclaude
6ce60a82c3 docs(link): the player-vendor marketplace (Protocol 3.0 §8) 
Documents order 5b across the four repos, and records what building it changed
about §8 as designed.

- NEW website/MARKETPLACE.md — the operator guide: what the pages must say out
  loud and why, the privacy contract (the player's in-game Vendor Search toggle
  wins, and no admin setting overrides it), the Bridge.cfg knobs and how they
  trade against each other, and the measured sweep costs.
- INTEGRATION.md — catalog entry for vendor.listing / vendor.listing.remove with
  its six consumer gotchas, and the GET /market REST section (the sidecar's only
  paged read, and why it orders by serial rather than shop name).
- BACKEND_DESIGN.md — shard_vendors / shard_vendor_items, the routes, and the
  marketplace search as the only rate-limited public read.
- SHARD_VISIBILITY.md — why the market's fields default to Everyone (the in-game
  gump already shows exactly that set), why location is one setting covering
  four things, and why hiding the owner name without the owner id achieves
  nothing.
- PLAN.md — the amortized round-robin as the one sweep pattern the bridge did not
  previously have, and an update to §7's cliloc note: pushing name resolution to
  the plugin was never an option, because ServUO cannot read a modern client's
  compressed cliloc files either.
- v3.md §8.8 — the four things the build settled differently, chief among them
  that §8.1's FLAT location payload would have made Part A's pre-wired
  market.location rule inert, exactly like the characterName miss one part
  earlier.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-29 09:52:14 -05:00
Colby Whitlock
70d49b7792 Merge pull request 'docs(link): the cliloc table, and why §8.6's recommendation was not implementable' (#70) from docs/cliloc-table into edge 
Reviewed-on: #70
 2026-07-29 11:58:26 +00:00
wtclaude
ee0c146d7a docs(link): cliloc overlays for shard-added and shard-edited items 
Shards edit items and add new ones, carrying cliloc ids no stock client table
has. The cliloc table now reads a SET of sources rather than one file — a base
plus every operator-maintained overlay under `custom/`, hash-gated together and
re-read on every boot, exactly as §6 reads the ServUO tree. Matching docs for
website #115.

website/CLILOCS.md gains a "Shard-added and shard-edited items" section: the
directory layout, merge precedence, the per-source breakdown an operator uses
to confirm an overlay took effect, and why `custom/` is a convention we chose
rather than one discovered from the shard — ServUO has no server-side notion of
a custom cliloc, they live in the patched client a shard distributes and
nothing in the tree declares them. Scale recorded: the live shard's script tree
references 16,434 cliloc ids and only 37 are absent from stock, which is why
this is an overlay and not a second table.

"Why there is no staged-approval flow" is replaced by "Two ways a refresh is
refused", because the set brings back the hazard a single file did not have. A
corrupt source fails the parse loudly; a source that has VANISHED parses
perfectly and imports a table quietly missing everything it contributed. That
is the same ambiguity §6 stages a facet removal for, so it is staged here too
(`needsReview`, `{approve:true}` to accept) — as a flag rather than §6's
approve/reject pair, because the atlas stores a pending decision SO THAT
approving re-parses, and here nothing is stored.

Two more traps recorded in §8.6 (now five), both found by running a shard-style
overlay rather than another stock-table fixture:

- Tidying punctuation unconditionally corrupts real names — a custom
  "Runic Gateway Sigil (v2)" rendered as "(v2". Stripping leftover brackets is
  right after a placeholder is removed and wrong otherwise, the same condition
  the `%` rule already had.
- Source labels must be forward-slashed and root-relative or the same directory
  fingerprints differently on Windows and Linux and every boot looks like a
  change — the identical bug §6 records.

BACKEND_DESIGN.md: the source set, the two refusals, and the updated admin
route notes (`approve`, `missingSources`, overlays found beside a file path).

Documentation only. Every claim was written after the behaviour was observed:
the counts come from the live shard's script tree, and the refusal/approve flow
and per-source breakdown are copied from real runs in website #115.

PROJECT_TREE.md files are deliberately untouched — CI-generated by the
sync-project-tree workflow.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-29 06:46:43 -05:00
wtclaude
e3aabf9e3e docs(link): fill in the cliloc PR numbers, correct the cliloc file count 
The progress table and CLILOCS.md carried #TBD placeholders until the PRs
existed; they now point at website #115 and docs #70.

Also corrects "all four Cliloc.* files" to eight (chs, cht, deu, enu, esp,
fra, jpn, kor) in both v3.md §8.6 and CLILOCS.md — the compression marker was
checked against every one of them, and understating the sample weakens the
claim it supports.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-29 04:25:06 -05:00
wtclaude
be9f5019fa docs(link): the cliloc table, and why §8.6's recommendation was not implementable 
Protocol 3.0 §8.6 resolved as its own website-only change, landing ahead of
§8 so the marketplace ships with real item names. Matching documentation for
website #TBD.

NEW website/CLILOCS.md — operator-facing: why the conversion step exists, how
to convert, how to configure the path, the refresh contract, what gets stored
and how names are applied.

link/v3.md §8.6 rewritten. Two things in the original recommendation turned
out to be wrong, and both are recorded because the reasoning generalises:

1. The committed db/data/clilocs.json artifact predates the Part C
   corrections (§6.1) and violates both — no committed snapshot of derived
   content, and nothing EA-derived ever shipped. UO's strings are EA's,
   exactly as the creature sprites are.
2. "scripts/buildClilocs.js reads the UO client's Cliloc.enu" is not
   possible. EVERY current client ships its cliloc files compressed (first
   DWORD's high byte 0x8E, the Mythic container); the plain layout is what
   those files looked like before that change, and parsing one as the other
   does not fail cleanly — it yields ~19k records with negative ids, 1,722
   distinct keys out of 19,508, one 62 KB "string", and a truncation
   somewhere in the middle. ServUO's own Ultima.StringList cannot read it
   either, so VendorSearch.GetItemName is already inert on such a shard and
   the work could not be pushed to the plugin.

That second point also retires an open question in §8.2: the warning never to
call GetItemName in the market sweep costs us nothing we could otherwise have
had, because the in-game Vendor Search gump has the same gap.

Three traps found by building it are recorded: StringList.SaveStringList
RE-COMPRESSES on save (its output is byte-identical to its compressed input,
because its purpose is round-tripping a file back into the client); trimming a
text line before splitting silently drops the ~half of a table that is empty
strings; and Number('') is 0, not NaN.

Also updated:
- Progress and §9 sequencing tables: order 5 split into 5a (this, website
  only) and 5b (the four-repo wire change).
- website/BACKEND_DESIGN.md — shard_clilocs / shard_cliloc_meta, the three
  admin routes, and why there is no staged-approval flow and no public route.
- link/INTEGRATION.md — the char.profile field note now says explicitly not to
  expect the shard to resolve clilocs, and points at CLILOCS.md.
- §10 documentation obligations list CLILOCS.md.

Documentation only. Every claim was written after the corresponding behaviour
was observed running: the compressed-format finding and the parse failures
come from the real client files on this machine, and the counts (123,490
parsed → 67,496 stored) and timings from importing them into the live MariaDB.

PROJECT_TREE.md files are deliberately untouched — they are CI-generated by
the sync-project-tree workflow and say so in their header.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-29 04:22:05 -05:00
Colby Whitlock
f715323aa0 Merge pull request 'docs(link): points.board, the leaderboards API, and what a real shard changed' (#69) from feat/points-board into edge 
Reviewed-on: #69
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
 2026-07-29 07:52:25 +00:00
wtclaude
8e857a9c8d docs(link): points.board, the leaderboards API, and what a real shard changed 
Protocol 3.0 §7 lands across servuo-plugins, link and website; this is the
matching documentation.

INTEGRATION.md
  - points.board in the event catalog: one frame per system, never a delta, no
    points.remove (the shard's system set is fixed at startup). Four gotchas
    called out, all of them things a consumer gets wrong by default:
      1. maxPoints: 0 means UNCAPPED, not "zero allowed" — and on a real shard
         it is the COMMON case, so anything rendering points/maxPoints must
         special-case it or divide by zero on the happy path.
      2. nameString is usually null, with a cliloc in nameNumber — so
         humanising the system key is the primary display path, not a fallback.
      3. players counts players actually holding points, not table size: ten of
         the ~25 systems keep a zero-point row per character ever created, so
         the raw count would report the shard's whole census.
      4. Entries carry serial + name only, never acct/webId.
  - The char.profile `points` block, and why `rank` is absent by default.
  - GET /points and /points/:system, including why 404 (never published) and
    200-with-empty-top (published, nobody scored) are different answers.

v3.md
  - B/2 marked done in both the progress table and §9.
  - NEW §7.5, "what the run against a real shard changed" — the same record §6.1
    and §6.2 keep. Four corrections the plan could not have anticipated from
    reading PointsSystem.cs, the sharpest being that (long)double.MaxValue is an
    unchecked conversion yielding long.MinValue, which published
    "maxPoints": -9223372036854775808 on the first live sweep. Also records that
    GetEntry/GetPoints mutate the world on AutoAdd systems and so cannot be used
    in a read model, and the one deliberate deviation from §7.4: the visibility
    field rule must key on the wire's `name`, not the descriptive
    `characterName`, or it is silently inert.

BACKEND_DESIGN.md — shard_points_boards (including why the top-N list stays in
the payload and why listing orders by COALESCE(name, system)), plus the two new
public routes.

PLAN.md — 3.0 phasing brought current: the spawn atlas and points.board added to
what has shipped, and the Points* keys noted in the config-key paragraph.

PROJECT_TREE.md files are deliberately untouched — they are CI-generated and say
so.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-28 21:05:04 -05:00
Colby Whitlock
64fb7edc3e Merge pull request 'docs(website): the spawn atlas API, the admin panel, and the delay-unit trap' (#68) from docs/spawn-atlas-api into edge 
Reviewed-on: #68
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
 2026-07-29 00:53:54 +00:00
wtclaude
be7e1a69ce docs(website): the spawn atlas API, the admin panel, and the delay-unit trap 
Docs half of website #113 (Protocol 3.0 Part C, second website PR).

Carries the atlas rewrite that missed #67: that PR merged before the "derive
from the tree on every boot" commit was pushed, so `edge` currently describes
the build/import-artifact design that was rejected in review, not what shipped
in website #112. It lands here.

New in SPAWN_ATLAS.md: the six public routes and five admin ones, and three
behaviours that read as bugs unless they are written down — an unreadable tree
answers 200 with status "unavailable" rather than 500 (refresh reports outcomes
so boot is never blocked by a bad tree, and the contract is preserved at the
API), setting the ServUO path deliberately does not import, and `points` is a
count while `spawners` is the list.

Also the delay-unit trap: XmlSpawner stores MinDelay/MaxDelay in minutes OR
seconds per record, decided by that record's own DelayInSec flag, so a `5` is
five minutes on one spawner and five seconds on the next. Both are plausible
respawn times, which is what makes it silent. 170 of 6,455 stock spawners are
second-flagged. And PARSER_VERSION, which exists because hashing the tree alone
would strand an install whose maps never change on whatever an older parser
derived.

BACKEND_DESIGN.md gains the routes, the router-map entry, and the parser-version
rule. v3.md marks Part C done and records in 6.3 what the API half found.

api-route-inventory.json refreshed from the live manifest — it had drifted to
200 routes before this PR (real count was 204) and is now 215.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP
 2026-07-28 19:51:50 -05:00
Claude
1b7da860b5 Merge remote-tracking branch 'origin/edge' into docs/spawn-atlas 2026-07-28 16:47:54 -05:00
wtclaude
ff1c2064a5 docs(website): the atlas reads the shard's tree on every boot, not a snapshot 
Follows the redesign in website #112. Two decisions from the original §6 were
rejected in review and replaced; the docs now describe what was actually built.

**The committed artifact is gone.** A shard's maps change over its life, so a
snapshot in the repo silently drifts from the world players actually see. The
ServUO tree is the single source of truth and the atlas is re-derived on every
server boot, hash-gated so an unchanged tree costs one read pass and no write.

**Nothing may name a facet.** The first implementation carried a lookup table of
the six stock UO facets. A shard may add facets, replace them outright, or rename
them when its maps are updated, and a built-in list mishandles all three
silently. Reconciliation is now by matching against the facet set discovered from
the shard's own data, with an unmatched name keeping its own rather than being
forced into a wrong bucket.

## Changes

- **`website/SPAWN_ATLAS.md`** rewritten: the two ideas that shape the design,
  how to configure the tree path, the boot flow as a decision tree, the
  approve/reject flow, and the code layout. The artwork policy is unchanged and
  still explicit — no art ever ships, operators extract their own from their own
  client files.
- **`link/v3.md` §6.1 (new)** records the two rejected decisions plus the two
  boot-path contracts. The old "what real data changed" list becomes §6.2. §6's
  now-superseded passages — the artifact bullet, the payload budget, the operator
  re-run story — are marked rather than deleted, so the reasoning stays legible.
- **`website/BACKEND_DESIGN.md`** documents `shard_atlas_pending` and the two
  contracts that make it safe: a facet removal is staged for a human, and the
  boot refresh can never block startup.

The two contracts are the part worth reviewing. Losing a facet is
indistinguishable at boot from a half-copied or mid-update tree, so it is staged
rather than applied; and no failure mode of the atlas — missing path, unreadable
mount, malformed file, database error — is allowed to stop the site coming up.

---

- [x] AI-assisted: written with **Claude Code** (Claude Opus 5), reviewed before opening.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP
 2026-07-28 16:44:39 -05:00
Colby Whitlock
10ae129b94 Merge pull request 'docs(website): record the spawn atlas pipeline and what real data changed' (#67) from docs/spawn-atlas into edge 
Reviewed-on: #67
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
 2026-07-28 21:16:06 +00:00
wtclaude
3fb3f63f25 docs(website): record the spawn atlas pipeline and what real data changed 
Protocol 3.0 order 3 (Part C), docs half of website #112. Part C is website-only
— no plugin, no sidecar, no new kinds, no wire change.

## New: website/SPAWN_ATLAS.md

The operator-facing reference: the build/import split and why it exists (build
needs a ServUO tree, import does not, and the container has the artifact but not
the tree), the re-run story, the artifact format, the placement transform, and
the three quirks in the source data that are silent when unhandled.

Also documents the artwork policy explicitly: **the project ships no creature
art and no extraction tooling.** Sprites live in the operator's own client
.mul/.uop files and are theirs, not ours to redistribute. `art` is nullable and
NULL on every fresh import; an operator who wants art extracts it themselves into
the gitignored uploads/atlas/ and maps slugs in a gitignored art map. Text-only
is the normal, supported state — not a degraded one.

## New: v3.md §6.1 — what the build against real data changed

Six corrections, kept as a diff rather than edited into §6 in place, because
each is a trap the next person would otherwise re-enter:

1. **Six facets, not thirteen.** Eodon.xml and the other named-area files carry
   TerMur/Trammel points; the facet comes from each record's `<Map>`.
2. **The XML dependency call resolved: hand-rolled, zero deps.** §6 left
   fast-xml-parser vs a tokenizer open.
3. **Facet names disagree between sources** — Locations says `Ter Mur`, `<Map>`
   says `TerMur`. Unreconciled the landmark fallback never fires there and every
   unregioned Ter Mur/Tokuno spawn silently reads "Wilderness".
4. **Spawn type tokens carry XmlSpawner directives** (`Fairy,{RND,4,8}`,
   `alchemist/z/-50`). Taken literally they invent creatures that do not exist
   and split real ones in two. 71 of 845 affected; 800 remain after stripping.
5. **The artifact is 1.41 MB, not "well under 1 MB"** — down from 4.40 MB via
   three encodings. Getting under 1 MB would mean dropping the spawner name.
6. **DELETE, not TRUNCATE** — TRUNCATE is DDL in MariaDB and implicitly commits,
   which would defeat the all-or-nothing reload the design asked for.

§6 also now records that Part C ships as two website PRs: the parsing half is
where the correctness risk lives and should not be reviewed inside a 10k-line
diff alongside routes and React.

## BACKEND_DESIGN.md

The seven atlas tables, the import-owned contract, the four column choices that
are traps (`spawn_range`/`grp` reserved words, DELETE vs TRUNCATE, explicit point
ids, plain INDEX not FULLTEXT), and the distinction between the configured
champion roster and the live champ.update feed.

PROJECT_TREE.md is left alone — it is auto-generated by the sync-project-tree
workflow.

---

- [x] AI-assisted: written with **Claude Code** (Claude Opus 5), reviewed before opening.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U7CBg11prhLimL9iHSX1bP
 2026-07-28 16:13:30 -05:00
Colby Whitlock
e9ecdc0ecb Merge pull request 'docs(link): record world.ruleset and mark Protocol 3.0 progress' (#66) from docs/world-ruleset into edge 
Reviewed-on: #66
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
 2026-07-28 20:34:38 +00:00
wtclaude
09467c67b0 docs(link): link the world.ruleset PRs from the v3 progress table 
Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-28 14:41:54 -05:00
wtclaude
b0a2207c6a docs(link): record world.ruleset and mark Protocol 3.0 progress 
Protocol 3.0 order 2 (v3.md §5) is built across all four repos; this is its
documentation half, plus the running progress record the plan was missing.

v3.md
  - A progress table at the top and a State column on §9's sequencing table, so
    "what has landed" is answerable without reading four git logs. Part A (order
    1) and world.ruleset (order 2) are marked done; the spawn atlas is next.
  - §5 gains the implementation notes worth keeping, chiefly: where a system's
    on/off state is DERIVED rather than configured, read the system's own static
    instead of inventing a .cfg key (Shadowguard has no Enabled key — it's the
    TOL expansion gate; Factions is `!ViceVsVirtueSystem.Enabled` by
    construction in stock ServUO). Also that the plugin CAN be compile-verified
    despite the "no standalone build" caveat, and how.

INTEGRATION.md
  - The world.ruleset catalog entry and GET /ruleset, with the two things
    consumers get wrong: caps are in TENTHS (1000 = 100.0), and `connect` exists
    only if the operator set Bridge.PublicConnectAddress — the shard's real
    listen address is never published.
  - §2 now says plainly that v3 has NOT been bumped yet and what that means:
    sidecars on `edge` report 2 while already carrying some v3 kinds, so do not
    infer feature availability from the version during this window.

PROTOCOL_2.md §10.4
  - The deferred "which PvP system does this shard run?" is answered (VvV on,
    Factions off — and mutually exclusive by construction), and world.systems is
    marked superseded by world.ruleset, which carries the systems block it asked
    for. No orphan kind is left behind.

BACKEND_DESIGN.md — the shard_ruleset table (why it is stored whole rather than
normalized, and why no row means null rather than {}) and the public route.

PROJECT_TREE.md is deliberately untouched: sync-project-tree regenerates it on
push to main, so it updates itself at the v3 cutover.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-28 14:39:53 -05:00
Colby Whitlock
bd9718a859 Merge pull request 'docs(shard): record the REST projection gap the Part A smoke test found' (#65) from docs/shard-visibility-rest-projection into edge 
Reviewed-on: #65
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
 2026-07-28 15:56:20 +00:00
wtclaude
4c0ceb1c41 chore(docs): drop an unrelated working-tree file committed by mistake 
android/TRUSTED_DEVICES_APP_HANDOFF.md was untracked in the working tree
before this branch and was swept in by a `git add -A`. It is not part of
this change; untracked here and left on disk.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-28 10:53:01 -05:00
wtclaude
35ad440bad docs(shard): record the REST projection gap the Part A smoke test found 
The live five-rung smoke test of the visibility framework found that Part
A enforced it on the SSE path and on /guilds + /governors, but not on the
remaining public REST reads - so one event was projected live and served
verbatim from stored history.

link/v3.md gains 3.6.1 with the full list (the anonymous acct/webId leak
on /feed, the flattened ownerAcct on /idoc, the dead `houses` field
rules, /feed ignoring live config, the empty-allowlist fall-through, and
the Date-to-{} projection bug), plus the rule it leaves behind: a read
path that returns shard data and does not project is a bug, and every new
Part B/C surface must gate its kind set on live config rather than on
PUBLIC_KINDS.

3.5 also corrected: the table is NOT seeded on boot. An absent row means
"use the compiled default", which keeps the defaults in one place instead
of duplicating them into a seeder that could drift.

BACKEND_DESIGN.md 6.5 records the same as a security contract: rule 1
locks a field by meaning rather than spelling; PUBLIC_KINDS is a
module-load constant and must not answer per-caller questions;
projectFeature walks arrays and plain objects only.

SHARD_VISIBILITY.md gets the admin-facing version - that stored history
answers the same way the live stream does, and that turning live updates
off stops the push, not the reading.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-28 10:52:13 -05:00
Colby Whitlock
5cb77595aa Merge pull request 'docs(website): record the shard visibility framework' (#64) from docs/shard-visibility into edge 
Reviewed-on: #64
 2026-07-28 15:06:56 +00:00
Colby Whitlock
8b4fc439ee Merge branch 'edge' into docs/shard-visibility 2026-07-28 15:06:39 +00:00
wtclaude
bf41105ec0 docs(website): record the shard visibility framework 
Protocol 3.0 Part A. Admin-configurable, per-feature and per-field
audience control over every shard-derived surface, replacing the static
PUBLIC_KINDS allowlist that used to be the whole boundary.

- SHARD_VISIBILITY.md (new): the admin-facing guide - the ladder, what
  each of the ten features exposes, the defaults, the two rules that are
  code rather than configuration, and worked examples.
- BACKEND_DESIGN.md 6.5 (new): the same thing as a security contract -
  the ladder and how viewerLevel resolves it, the locked acct/webId rule,
  the fail-closed kind map, the asymmetric ladder fallbacks, and the
  three enforcement points. Plus the shard_feature_visibility schema, the
  /public/shard/features route, and the adminOnly tier on
  /admin/shard/visibility.

Defaults reproduce pre-3.0 behavior everywhere, with one deliberate
exception which is the leak Part A was written to close: guilds and
governors previously returned the raw stored payload, whose leader and
governor actors carry acct and webId, to anonymous callers.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-28 10:04:29 -05:00
Colby Whitlock
06b4a06baa Merge pull request 'docs(tree): sync website/PROJECT_TREE.md' (#59) from chore/sync-website-tree into main 
Reviewed-on: #59
 2026-07-28 15:02:41 +00:00
Colby Whitlock
7e8cbe1916 Merge pull request 'docs(link): add the Protocol 3.0 design' (#63) from docs/link-v3-plan into edge 
Reviewed-on: #63
Reviewed-by: Colby Whitlock <whitlocktech@gmail.com>
 2026-07-28 14:41:26 +00:00
wtclaude
6622afe4bd docs(link): add the Protocol 3.0 design 
Surveys the live ServUO tree against everything the bridge already
surfaces and records the full gap list (17 items), then specs the four
features scoped for 3.0.

3.0 has three scope areas:

- A: the visibility framework. Admin-configurable, per-feature and
  per-field audience control over all ten shard-derived surfaces (the
  four new ones plus the six that already ship), on an
  anonymous -> logged_in -> player -> staff -> admin ladder. Every
  default reproduces today's behavior, so the retrofit is a no-op until
  an admin changes something. Two rules an admin cannot override: acct
  and webId are admin-only always, and an unmapped event kind is never
  broadcast below admin. This also fixes a verified leak - guild leader
  acct/webId are readable today on the anonymous /public/shard/guilds.
- B: three new wire streams - world.ruleset, points.board, and
  vendor.listing/vendor.listing.remove.
- C: the spawn atlas, built from static ServUO data files with no wire
  involvement.

Visibility lives entirely on the website; the sidecar stays a dumb
forwarder that defines no access parameters and advertises no
capabilities.

PROTOCOL_VERSION goes 2 -> 3 once, at the end: every part PRs into an
edge branch per repo, and the coordinated edge -> main merge is the
cutover. A schema migration moves uo_link_config.protocol so operators
don't have to.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-28 09:28:04 -05:00
runic-docs-bot
f2fa6abff7 docs(tree): sync website/PROJECT_TREE.md from RunicGateway/website@a3407ae [skip ci] 2026-07-28 06:12:43 +00:00
18 changed files with 4751 additions and 31 deletions
Expand all files Collapse all files

21
README.md
View File

@@ -7,10 +7,11 @@ so they live in one place, independent of either codebase.
## Layout
```
website/ docs from the shard website (Node/Express + MariaDB + React/Vite)
link/ docs from the ServUO bridge (C# plugin + Rust sidecar + Node WS)
android/ docs from the native Android client (Kotlin + Jetpack Compose)
ci/ cross-cutting CI/quality notes
website/ docs from the shard website (Node/Express + MariaDB + React/Vite)
link/ docs from the ServUO bridge (C# plugin + Rust sidecar + Node WS)
android/ docs from the native Android client (Kotlin + Jetpack Compose)
installer/ docs for the installer that deploys a shard's bridge components
ci/ cross-cutting CI/quality notes
```
### `website/`
@@ -19,6 +20,11 @@ ci/ cross-cutting CI/quality notes
| [BACKEND_DESIGN.md](website/BACKEND_DESIGN.md) | API contract, DB schema, security model |
| [HERO_EDITOR.md](website/HERO_EDITOR.md) | Hero canvas editor feature spec |
| [WIKI_UPGRADE.md](website/WIKI_UPGRADE.md) | Wiki subsystem upgrade notes |
| [SHARD_VISIBILITY.md](website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework |
| [SPAWN_ATLAS.md](website/SPAWN_ATLAS.md) | The bestiary / spawn atlas: what the shard contains, parsed from its own ServUO tree |
| [CLILOCS.md](website/CLILOCS.md) | UO's id → name table: converting one from your client so items have names |
| [UOFIDDLER.md](website/UOFIDDLER.md) | **Operator runbook** — step-by-step extraction from your own UO client (cliloc table, creature art) |
| [MARKETPLACE.md](website/MARKETPLACE.md) | The player-vendor index: how it is gathered, what it costs, how to tune it |
| [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) |
| [PROJECT_TREE.md](website/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout |
@@ -27,6 +33,7 @@ ci/ cross-cutting CI/quality notes
|---|---|
| [INTEGRATION.md](link/INTEGRATION.md) | How the website integrates with the uo-link sidecar |
| [PROTOCOL_2.md](link/PROTOCOL_2.md) | Protocol 2.0 / 2.1 design |
| [v3.md](link/v3.md) | Protocol 3.0 design — shard content/standings streams + the visibility framework |
| [ADMIN_CONTROLS.md](link/ADMIN_CONTROLS.md) | Staff write-plane (kick/ban/broadcast, page queue) |
| [SHARD_PREREQS.md](link/SHARD_PREREQS.md) | Shard-side prerequisites for the bridge |
| [PLAN.md](link/PLAN.md) | uo-link build plan |
@@ -44,6 +51,12 @@ ci/ cross-cutting CI/quality notes
| [TRUSTED_DEVICES_APP_HANDOFF.md](android/TRUSTED_DEVICES_APP_HANDOFF.md) | Trusted-devices app handoff notes |
| [PROJECT_TREE.md](android/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout |
### `installer/`
| Doc | What it covers |
|---|---|
| [INSTALL.md](installer/INSTALL.md) | **Operator guide** — installing Runic Gateway on a ServUO shard, connecting it to the website, and diagnosing it. Includes the by-hand path, which works today |
| [PLAN.md](installer/PLAN.md) | Installer design of record — phases, locked decisions, the bundle/compat-matrix model |
## Provenance
- `website/*` was extracted from `RunicGateway/website` via `git filter-repo`.

119
android/PLAN.md
View File

@@ -1,6 +1,6 @@
# Android App — Plan
Status: **M0–M7 landed; M7 (push notifications) both parts done — Part 1 backend (website#78) and Part 2 app (Android-app#15) plus a small `push.ntfyUrl` settings addition (website#79). Remaining: set the shard's `NTFY_*` deploy config so push lights up, and cut the v1 tag. M9 (native SSO login) is now underway backend-first — the Mobile SSO Authorization Bridge is being built in `website/` + `docs/` ahead of the app-side client (§4.2, §9 M9); custom-scheme callback only for now, App Links deferred (see [`APP_LINKS.md`](./APP_LINKS.md)).** This document is the
Status: **M0–M7 landed; M7 (push notifications) both parts done — Part 1 backend (website#78) and Part 2 app (Android-app#15) plus a small `push.ntfyUrl` settings addition (website#79). Remaining: set the shard's `NTFY_*` deploy config so push lights up, and cut the v1 tag. M9 (native SSO login) is now underway backend-first — the Mobile SSO Authorization Bridge is being built in `website/` + `docs/` ahead of the app-side client (§4.2, §9 M9); custom-scheme callback only for now, App Links deferred (see [`APP_LINKS.md`](./APP_LINKS.md)). **M11 (Protocol 3.0 shard parity)** is scoped and next: the app sees none of the four shard features v3 added (`ruleset`, `leaderboards`, `market`, `atlas`) and does not consult `GET /public/shard/features`, so it gates shard nav on session role alone while an admin can switch any of those surfaces off or raise its audience — the v3 `edge` → `main` cutover is held until it lands (§9 M11).** This document is the
design contract for the `RunicGateway/Android-app` repo. It was written before implementation so the
API changes it depends on could be landed in `website/` and `docs/` first. The authoritative API
reference is the committed OpenAPI spec at `website/server/swagger/swagger-output.json` (regenerated
@@ -631,6 +631,8 @@ not rank).
| News & content | everyone | `/public/posts/:category`, `/public/pages/:slug` |
| Wiki | everyone | `/public/wiki`, `/public/wiki/categories`, `/public/wiki/tags`, `/public/wiki/:slug` |
| Shard (live) | everyone | `/public/shard/*` + `/public/shard/stream` (SSE) |
| **Rules / Leaderboards / Market** | everyone, *if the shard publishes them* | `/public/shard/{ruleset,points,market}` (M11) |
| **Atlas** (bestiary) | everyone, *if the shard publishes it* | `/public/atlas/*` (M11) |
| Contact | everyone | `/public/contact` |
| **My Account** | signed-in | `/player/account/*` (or `/admin/account/*` for staff — see §6.4) |
| **My Characters / Vendors / Houses** | `player` (linked) | `/player/shard/*` |
@@ -639,6 +641,12 @@ not rank).
Guidelines:
- The menu is **declarative + data-driven**, not a pile of `if role ==` checks — one list of entries
with a `minAccess`/`requiredCapability` field, filtered by the session.
- **Session role is not the only gate on shard surfaces (M11).** Every shard-derived feature is
*admin-configurable* — it can be switched off or raised to a higher audience rung — so a shard entry
is filtered by the session role **and** by `GET /public/shard/features`, which reports the features
the caller may actually reach. While that answer is unknown (in flight, or the lookup failed) the app
shows everything: the server gates regardless, and a nav that flickers in on every load is worse than
a link that briefly `403`s.
- Never hide the fact that more exists behind auth in a way that misleads; anonymous users see public
groups and a "Sign in" affordance.
- The server is the source of truth: a hidden/greyed item is a UX convenience; every gated call still
@@ -663,9 +671,15 @@ Guidelines:
### 6.2 Public shard (live)
- Status/online/feed/economy/champs/guilds/governors(+history)/presence/houses/idoc — the
`/public/shard/*` GETs.
- **Live updates** — subscribe to `GET /public/shard/stream` (SSE, safe kinds only) and patch the
in-memory boards in place (champ/guild/city/house/presence update+remove frames). Reconnect with
backoff; fall back to poll if SSE drops.
- **Live updates** — subscribe to `GET /public/shard/stream` (SSE) and patch the in-memory boards in
place (champ/guild/city/house/presence update+remove frames). Reconnect with backoff; fall back to
poll if SSE drops. What arrives on the stream is **resolved from the caller's audience rung at
subscribe time**, not from a fixed allowlist (Protocol 3.0 §3.6) — the stream request carries the
bearer like every other call, so a signed-in app session sees exactly what the same account sees on
the web.
- **Visibility + the Protocol 3.0 surfaces (M11)** — `GET /public/shard/features` drives which of these
the menu offers; `GET /public/shard/{ruleset,points,points/:system,market,market/meta,market/vendors/:serial}`
and `GET /public/atlas/*` are the new reads. Full contract and traps in §9 M11.
### 6.3 Player self-service & game data (bearer)
- **Account** — `GET /player/account`; `PATCH /player/account/username`;
@@ -675,6 +689,11 @@ Guidelines:
- **My game data** — `GET /player/shard/roster/:account`, `/char/:serial`, `/vendors/:account`,
`/sales`, `/houses`. All ownership-checked server-side; a `503` means shard/sidecar down → show an
"offline, retry" state (see §7).
- **The character sheet carries two things the app does not yet read (M11):** the `points` block
(per-character loyalty/points standings, Protocol 3.0 §7.3) and the server-resolved cliloc names on
`equipment[].clilocName` / `titles.rewardResolved` (§8.6). Both are served **ungated** on this route —
a character's own standings are self-service data and do not depend on the public `leaderboards`
feature being visible, which is the behavior the app must mirror rather than re-gate.
- **Presentation is text-only for v1.** Character sheets and vendor listings render as data/text — no
item icons or paperdoll art. A richer "pretty paperdoll" view is a **future** enhancement (pending the
art/asset work on the platform side) and is explicitly out of the first release.
@@ -883,10 +902,98 @@ push, and Play (M6–M8) follow the designed app.
shard-write actions degrade gracefully when the sidecar is offline. Excluded: hero/CMS block
editor, Discord-bot config, uo-link config, OAuth-provider setup.
12. **M11 — Protocol 3.0 shard parity** (post-v1; scoped 2026-07-30). The website's Protocol 3.0 work
added four shard features and, with them, an **admin-configurable visibility framework** the app
knows nothing about. `link/v3.md` §10 deferred the app side as a follow-up; it is now scoped
deliberately, and **the v3 `edge` → `main` cutover is held until both parts land** so web and app
surface the same shard on the same day (decided 2026-07-30).
Neither part is coupled to the cutover *merge order*, which is what makes holding it a schedule
decision rather than a technical one: against a pre-v3 website every new route and
`/public/shard/features` simply `404`s, and each consumer below falls back to exactly today's
behavior. The app declares no protocol version and never talks to the sidecar.
- **Part 1 — the visibility rules + the read-model adds.** The security-shaped half, reviewed on
its own:
- `GET /public/shard/features` → `{ level, features[] }`: the features **this caller** may reach.
A new singleton cache mirrors the web client's (`lib/useShardFeatures.js`): per-viewer but
stable for a session, invalidated on sign-in/out and on a server switch.
- `MenuEntry` gains `feature: String?` beside its existing `access`, so the one declarative menu
(§5) filters on the session role **and** the shard's live feature config. While the lookup is
in flight or has failed, **show everything** — the same deliberate fail-open the web client
takes, because the server gates regardless and a nav that flickers in on every load is worse
than a link that briefly `403`s. The gate is server-side; hiding is presentation.
- **`404` and `403` mean different things here** and neither is a generic error:
`requireFeature` `404`s a *disabled* feature (deliberately not disclosing that it exists) and
`403`s a viewer *below its audience*. Both render "not available on this shard", alongside the
existing `503` = shard offline (§7).
- **The `level` from `/features` is authoritative — do not re-derive the rung from the role.**
The server's ladder is `anonymous → logged_in → player → staff → admin`, where `player` means
*a linked game account* and staff always satisfy `player` (the same superset rule `Menu.kt`
already encodes as `isPlayer || isStaff`).
- **`char.profile.points`** → the "Loyalty & Points" block the web character sheet gained:
`CharProfileDto.points[{system, nameString, points, maxPoints, rank?}]`. Three traps, all of
them things a real shard does and a fake one does not (`v3.md` §7.5): `maxPoints == 0` means
**uncapped** and is the *common* case, so nothing may divide by it; `nameString` is usually
`null` because most systems name themselves with a cliloc, making the humanise-the-`system`-key
path the **primary** one rather than a fallback; and `rank` is absent unless the shard runs
`PointsProfileRank=true` — absent and "unranked" are different answers.
- **Cliloc-resolved names** (`v3.md` §8.6, already live on the website): `EquipmentDto` gains
`name` + `clilocName` and `TitlesDto` gains `rewardResolved`, so equipment stops rendering as a
layer or a bare id. Precedence is `name → clilocName → layer`: a player-given name outranks the
resolved type name, and the server applies the same order. A shard with no cliloc table
configured sends neither field and the sheet renders exactly as it does today.
- `ActorDto` keeps its `acct` / `webId` fields (nullable, so nothing breaks) but its KDoc stops
describing them as available: they are **locked to the admin rung**, always, and stripped from
every response below it.
- **Part 2 — the four new screens**, each hidden by its feature name in the menu:
- **Rules** — `GET /public/shard/ruleset` (`ruleset`). A `null` body means "the shard has not
published its ruleset yet", which is a different state from the feature being disabled. Every
block is optional and omitted when its system is off. **`caps.skill` / `caps.totalSkill` are in
tenths** (1000 = 100.0) and must be converted — the raw number is actively misleading, not
merely unhelpful. Live via the `world.ruleset` frame, which is on the public stream by default.
- **Leaderboards** — `GET /public/shard/points`, `/points/:system` (`leaderboards`). The same
`maxPoints`/`nameString` traps as the profile block. Live via `points.board`.
- **Market** — `GET /public/shard/market` (`q`, `minPrice`, `maxPrice`, `itemId`, `map`, `region`,
`sort`, `limit`, `offset`), `/market/meta` for the filter options + staleness, and
`/market/vendors/:serial` (`market`). Four things this screen must get right: it is the site's
first **rate-limited** public endpoint, so handle `429` the way the contact form does; the
*"prices last refreshed N minutes ago"* banner is **required, not decoration** — the shard
sweeps vendors round-robin, so a listing can legitimately be a full cycle stale and a page
implying live prices sends people to an item that sold twenty minutes ago; a `truncated` shop
must say so; and `location` is a **nested object** that an admin may gate away entirely, which
the vendor screen renders as "hidden by the shard" (a real answer) rather than as blank
coordinates — same for `ownerName` / `ownerSerial`. **The `market` SSE fan-out is off by
default** (a live firehose of vendor inventories would be the site's biggest bandwidth
consumer), so the screen is a plain paginated read and must never depend on live frames.
- **Atlas** — `GET /public/atlas/{creatures,creatures/:slug,regions,landmarks,champions,meta}`
(`atlas`). Note the path: `/public/atlas`, **not** `/public/shard` — the atlas is static shard
*content*, not live shard *state*, and unlike `/shard/*` it **is** `siteMode`-gated like
`/posts` and `/wiki`, so a site in maintenance mode withholds it independently of the sidecar.
Three traps. Two are units/naming, from `v3.md` §6.3: respawn delays are **seconds**
throughout, and `points` is a *count* on the search route while `spawners` is the *list* on
the detail route. The third is a **shape**: `places` is a list of
`{facet, label, spawners, maxAlive}` **objects**, not of place-name strings — it is the
aggregate the screen exists to show ("Shrines, Isamu-Jima, Yew"), it arrives only on the
detail route, and typing it `List<String>` makes that whole route fail to decode while the
request itself returns `200`.
- **Verification** — the five-rung walk (`anonymous`, `logged_in`, `player`, `staff`, `admin`)
against a local website on the cutover branch, per
[`../link/v3.md`](../link/v3.md) §11 and the shard-visibility smoke harness; plus one pass with
**every feature disabled** in Admin → Shard Visibility, confirming the app *hides* each surface
instead of erroring on it. Unit tests cover the menu filter (role × feature set), the
`404`/`403`/`503` mapping, and DTO decode for each new shape. **Decode tests must feed real
captured JSON**, not DTOs built in Kotlin: the fakes under `data/api/fake/` construct objects
directly, so they can never catch a wire/type mismatch — which is how the `places` shape above
shipped past a green suite.
- **Excluded**, in the same class as M10's exclusions: the admin *configuration* panels — Shard
Visibility, Spawn Atlas and Cliloc import — alongside the hero/CMS block editor, Discord-bot
config, uo-link config and OAuth-provider setup.
### Deferred (not a milestone)
- **`/api/mobile` facade migration + app-version floor** — briefly planned as M11 (2026-07-22), now
**deferred with no app work scheduled**. The website's router refactor is being done in place with
- **`/api/mobile` facade migration + app-version floor** — briefly planned as its own milestone
(2026-07-22), now **deferred with no app work scheduled**. The website's router refactor is being done in place with
every URL byte-identical and `/api/v1` is not being retired, so the app's ~70 hardcoded `api/v1/…`
endpoints, its SSE path, and its SSO URLs keep working untouched. If the mobile contract ever needs
to diverge from web, the migration comes back — starting from a one-line alias mount on the server,

19
android/PROJECT_TREE.md
View File

@@ -85,6 +85,7 @@ android-app/
│ │ │ │ │ │ │ ├── PlayerShardDto.kt
│ │ │ │ │ │ │ ├── PostDto.kt
│ │ │ │ │ │ │ ├── PublicDto.kt
│ │ │ │ │ │ │ ├── ShardContentDto.kt
│ │ │ │ │ │ │ ├── ShardDto.kt
│ │ │ │ │ │ │ ├── SsoDto.kt
│ │ │ │ │ │ │ └── WikiDto.kt
@@ -106,6 +107,7 @@ android-app/
│ │ │ │ │ ├── NotificationsRepository.kt
│ │ │ │ │ ├── PlayerShardRepository.kt
│ │ │ │ │ ├── SettingsRepository.kt
│ │ │ │ │ ├── ShardFeaturesRepository.kt
│ │ │ │ │ ├── ShardRepository.kt
│ │ │ │ │ └── WikiRepository.kt
│ │ │ │ ├── di/
@@ -171,6 +173,8 @@ android-app/
│ │ │ │ │ ├── session/
│ │ │ │ │ │ └── SessionViewModel.kt
│ │ │ │ │ ├── shard/
│ │ │ │ │ │ ├── AtlasScreen.kt
│ │ │ │ │ │ ├── AtlasViewModel.kt
│ │ │ │ │ │ ├── ChampsScreen.kt
│ │ │ │ │ │ ├── ChampsViewModel.kt
│ │ │ │ │ │ ├── FrameFields.kt
@@ -180,7 +184,13 @@ android-app/
│ │ │ │ │ │ ├── GuildsViewModel.kt
│ │ │ │ │ │ ├── HousesScreen.kt
│ │ │ │ │ │ ├── HousesViewModel.kt
│ │ │ │ │ │ ├── LeaderboardsScreen.kt
│ │ │ │ │ │ ├── LeaderboardsViewModel.kt
│ │ │ │ │ │ ├── LiveBoard.kt
│ │ │ │ │ │ ├── MarketScreen.kt
│ │ │ │ │ │ ├── MarketViewModel.kt
│ │ │ │ │ │ ├── RulesScreen.kt
│ │ │ │ │ │ ├── RulesViewModel.kt
│ │ │ │ │ │ ├── ShardComponents.kt
│ │ │ │ │ │ ├── ShardEventText.kt
│ │ │ │ │ │ ├── ShardScreen.kt
@@ -284,6 +294,7 @@ android-app/
│ │ │ │ │ ├── PlayerShardDtoTest.kt
│ │ │ │ │ ├── PublicDtoTest.kt
│ │ │ │ │ ├── ShardBoardDtoTest.kt
│ │ │ │ │ ├── ShardContentDtoTest.kt
│ │ │ │ │ ├── ShardDtoTest.kt
│ │ │ │ │ ├── SsoDtoTest.kt
│ │ │ │ │ └── WikiDtoTest.kt
@@ -294,7 +305,8 @@ android-app/
│ │ │ │ └── FakeShardStream.kt
│ │ │ └── repository/
│ │ │ ├── AccountTrustedDevicesTest.kt
│ │ │ └── ConnectionVersionGuardTest.kt
│ │ │ ├── ConnectionVersionGuardTest.kt
│ │ │ └── ShardFeaturesRepositoryTest.kt
│ │ ├── ui/
│ │ │ ├── admin/
│ │ │ │ ├── AdminContentViewModelTest.kt
@@ -304,7 +316,8 @@ android-app/
│ │ │ ├── contact/
│ │ │ │ └── ContactViewModelTest.kt
│ │ │ ├── navigation/
│ │ │ │ └── MenuAccessTest.kt
│ │ │ │ ├── MenuAccessTest.kt
│ │ │ │ └── MenuFeatureGatingTest.kt
│ │ │ ├── notifications/
│ │ │ │ └── NotificationRoutingTest.kt
│ │ │ ├── player/
@@ -315,6 +328,8 @@ android-app/
│ │ │ │ ├── FrameFieldsTest.kt
│ │ │ │ ├── LiveBoardTest.kt
│ │ │ │ ├── ShardBoardViewModelTest.kt
│ │ │ │ ├── ShardContentHelpersTest.kt
│ │ │ │ ├── ShardContentViewModelTest.kt
│ │ │ │ └── ShardEventTextTest.kt
│ │ │ ├── theme/
│ │ │ │ └── BrandColorTest.kt

742
installer/INSTALL.md Normal file
View File

@@ -0,0 +1,742 @@
# Installing Runic Gateway on your shard
Operator guide for the **Runic Gateway installer** — the tool that takes a working ServUO
installation and connects it to a Runic Gateway website.
> **Status: the installer binary is not released yet.**
>
> Everything it installs *is* released and published — the sidecar, the plugin overlay, and the
> [bundle manifest](https://gitea.whitlocktech.com/RunicGateway/installer/src/branch/main/bundles/current.json)
> that names the checked combination of the two. This guide is the operator-facing contract those
> phases build to, and it is written first on purpose: it is the specification of what the run
> looks like, what it asks, where it writes, and what it prints.
>
> **You can install today without it** — [Appendix A](#appendix-a--installing-by-hand) is the same
> deployment done by hand, with the commands verified against the current releases. When the binary
> ships, Appendix A stays as the reference for what it does under the hood.
>
> Design of record: [PLAN.md](PLAN.md).
---
## What this installs
Three things, on the machine that runs your shard:
| # | Component | Where it comes from |
|---|---|---|
| 1 | **The plugin overlay** — C# source that ServUO compiles at boot, copied into your server tree | [`RunicGateway/servuo-plugins`](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins) release tarball |
| 2 | **The uo-link sidecar** — a small Rust service that the shard dials out to, and that your website reads from | [`RunicGateway/link`](https://gitea.whitlocktech.com/RunicGateway/link) release binary |
| 3 | **A record of what it did** — `install.json`, plus a cached copy of any patches it applied | Written by the installer |
```
ServUO shard ──loopback TCP 127.0.0.1:7788──► uo-link sidecar ──HTTP + WebSocket──► website
(1) overlay (2) binary + service (yours)
```
The shard **dials out**; it never listens for the website and is never reachable from the internet.
Only the sidecar is exposed, and only to your website.
### What it deliberately does not do
- **It never restarts or manages ServUO.** Your shard keeps starting the way it always has. The
installer refuses to run while ServUO is up, and tells you when a restart is required.
- **It never deletes anything from your server tree.** The overlay sync only adds and overwrites.
- **It never contacts your website.** It prints four values for you to paste into Admin → Shard.
- **It never edits stock ServUO files without asking.** That is the opt-in
[patch tier](#4-the-patch-tier-optional), and skipping it still leaves you with a working bridge.
---
## Before you begin
| Requirement | Detail |
|---|---|
| A working ServUO install | It must currently boot and compile scripts cleanly. The installer deploys onto a healthy shard; it does not repair a broken one. |
| ServUO **57.4** *(patch tier only)* | The base install works on any reasonably current ServUO. The patch tier is verified against stock 57.4 only, and is skipped with a warning on anything else. |
| ServUO **stopped** | `ServUO.exe` holds a lock on `Scripts.dll` and writes `Saves/` on exit. The installer refuses to deploy under a running shard. |
| Administrator / root | It writes into system directories and registers a service. |
| Outbound HTTPS | To `gitea.whitlocktech.com`, to fetch the bundle and the two artifacts. Nothing inbound is needed, and no Gitea account or git client is required. |
| The sidecar on the **same host** as the shard | The shard connects to `127.0.0.1:7788`. Splitting them is not supported — the loopback socket *is* the trust boundary for inbound commands. |
| Admin access to your Runic Gateway site | The last step is pasting four values into Admin → Shard. |
**Back up first.** The overlay overwrites `Scripts/Scripts.csproj` (a stock file), and the patch
tier edits stock sources. A copy of `Scripts/` and `Config/` before you start costs nothing.
---
## 1. Download and verify
Releases are **unsigned**. There is no code-signing certificate and no notarization, so the
`SHA256SUMS` file published beside every artifact is the whole trust anchor — check it.
Download the installer for your OS, plus `SHA256SUMS`, from the
[installer releases page](https://gitea.whitlocktech.com/RunicGateway/installer/releases):
```
runicgateway-installer-linux-x86_64
runicgateway-installer-windows-x86_64.exe
SHA256SUMS
```
**Linux**
```bash
sha256sum -c SHA256SUMS --ignore-missing
chmod +x runicgateway-installer-linux-x86_64
```
**Windows** (PowerShell)
```powershell
(Get-FileHash .\runicgateway-installer-windows-x86_64.exe -Algorithm SHA256).Hash
Get-Content .\SHA256SUMS # compare the line for this file, case-insensitively
```
Windows will show a **SmartScreen "Windows protected your PC"** prompt on first run, because the
binary is unsigned and unknown. Once you have verified the checksum above: *More info* →
*Run anyway*. If you would rather not, Appendix A's manual path uses no unsigned binary except the
sidecar itself, which you verify the same way.
The installer applies the same standard to everything **it** downloads: each artifact's SHA256 is
checked against the value recorded in the bundle manifest — which CI computed after verifying it
against the publishing repo's own `SHA256SUMS` — and a mismatch aborts the run.
### What it installs is a bundle, not "latest"
The three components version independently but must agree on one wire protocol, so CI publishes a
[**bundle**](https://gitea.whitlocktech.com/RunicGateway/installer/src/branch/main/bundles/README.md):
one exact, protocol-checked pair of sidecar + overlay versions. The installer resolves that at run
time rather than hardcoding versions or blindly taking each repo's newest release.
Consequences worth knowing:
- A sidecar patch release does **not** mean re-downloading the installer. The bundle is data.
- `--bundle <tag>` (e.g. `--bundle 2026.08.04`) pins an exact past combination, so a reinstall six
months from now reproduces today's install rather than tomorrow's.
---
## 2. Run it
```bash
sudo ./runicgateway-installer-linux-x86_64 install
```
```powershell
# Windows: from an elevated PowerShell
.\runicgateway-installer-windows-x86_64.exe install
```
Run `install --verify` first if you want to see exactly what would change and write nothing — the
same idea as `deploy.ps1 -Verify`, which developers of the plugin use.
The installer **does not install itself.** Keep the binary somewhere sensible on the host (it is
one file); `doctor`, `update` and `uninstall` are run from it later. Examples below shorten it to
`runicgateway`.
### What it asks
1. **Your ServUO root** — detected if the installer is run from inside it or from an obvious
sibling, otherwise prompted. A directory qualifies only if it contains `ServUO.exe`, `Scripts/`
and `Config/`.
2. **Whether to apply the patch tier** — off unless you say yes, and not offered at all if your
ServUO is not 57.4. See [§4](#4-the-patch-tier-optional).
3. **The hostname your website should use to reach this machine** — used only to compose the two
URLs it prints at the end. The sidecar's bind address is frequently `127.0.0.1` or `0.0.0.0`,
neither of which is something to hand to a website.
4. **Your site's URL** — used only to print a clickable link to its Admin → Shard page. The
installer never contacts your website.
### An illustrative run
```
Runic Gateway installer — bundle 2026.08.04 (protocol 3)
ServUO /opt/ServUO (57.4)
Shard process not running
Overlay servuo-plugins v0.1.1 protocol 3
Sidecar uo-link v1.1.0 protocol 3
✓ overlay tarball verified sha256 75dc6d6c…
✓ sidecar binary verified sha256 27d491ef…
Overlay sync
ADD Config/Bridge.cfg
ADD Scripts/Custom/Bridge/*.cs (22 files)
CHANGE Scripts/Scripts.csproj
deployed. add=23 change=1 unchanged=0
Patch tier skipped (not selected)
Without it: no vendor.sale events, no in-game moderation audit forwarding.
uo-link
binary /usr/bin/runicgateway-link
config /etc/runicgateway/sidecar.toml (created)
database /var/lib/runicgateway/uo-link.db
service runicgateway-link.service enabled, running
Recorded /etc/runicgateway/install.json
Scripts.csproj changed — ServUO rebuilds Scripts.dll on next boot.
Start your shard when ready; the installer does not start it for you.
```
Then the [token handoff](#5-connect-the-website).
### Commands and flags
The surface this guide specifies. Each command is idempotent: a second run with nothing new to do
reports "unchanged" and writes nothing.
| Command | What it does |
|---|---|
| `install` | The full run above. |
| `doctor` | Diagnoses an existing deployment end to end — see [§7](#7-day-two). |
| `update` | Re-resolves the bundle; updates the sidecar (replace + restart) and the overlay (re-sync + tell you to restart ServUO). |
| `uninstall` | Removes only what the installer exclusively owns; prints — never performs — anything inside your ServUO tree. |
| Flag | Applies to | Meaning |
|---|---|---|
| `--verify` | `install`, `update` | Dry run. Report every change that would be made; write nothing. |
| `--servuo <path>` | `install`, `doctor`, `update` | Name the ServUO root instead of detecting or prompting. |
| `--bundle <tag>` | `install`, `update` | Pin an exact published bundle instead of the current one. |
| `--patches` / `--no-patches` | `install` | Decide the patch tier non-interactively. `--patches` still refuses on a non-57.4 tree. |
| `--host <name>` | `install` | The hostname to print in the website URLs. |
| `--site-url <url>` | `install` | Your site's base URL, for the Admin → Shard link. |
| `--yes` | all | Assume the default answer to every prompt. Combine with the flags above for an unattended run. |
| `--purge` | `uninstall` | Also delete `sidecar.toml` and `uo-link.db`, which are otherwise kept. |
---
## 3. Where everything lands
**Linux**
| Path | What |
|---|---|
| `/usr/bin/runicgateway-link` | The sidecar binary |
| `/etc/runicgateway/sidecar.toml` | Sidecar config, including the auth token |
| `/etc/runicgateway/install.json` | What the installer deployed: versions, commit, per-file hashes, applied patches, timestamps |
| `/etc/runicgateway/patches/` | Copies of any patches applied, so `uninstall` can print the exact hunks long after the release tarball is gone |
| `/var/lib/runicgateway/uo-link.db` | The sidecar's SQLite store (event history, cached profiles, link map) |
| `/etc/systemd/system/runicgateway-link.service` | The service unit, running as a dedicated user |
**Windows**
| Path | What |
|---|---|
| `%ProgramFiles%\RunicGateway\uo-link-sidecar.exe` | The sidecar binary |
| `%ProgramData%\RunicGateway\sidecar.toml` | Sidecar config, including the auth token |
| `%ProgramData%\RunicGateway\install.json` | As above |
| `%ProgramData%\RunicGateway\patches\` | As above |
| `%ProgramData%\RunicGateway\uo-link.db` | The sidecar's SQLite store |
| Service `RunicGatewayLink` | Automatic start, restart on failure |
**Inside your ServUO tree** (added by the overlay sync — 24 files):
```
Config/Bridge.cfg every bridge setting, heavily commented
Scripts/Custom/Bridge/*.cs 22 files: the plugin itself
Scripts/Scripts.csproj OVERWRITES a stock file (see below)
```
Both service definitions pin `UOLINK_CONFIG` and `UOLINK_DB_PATH` explicitly. The sidecar's own
defaults are relative to its working directory, and a service manager's working directory is not
somewhere you want a database — on Windows it can be `%SystemRoot%\System32` or, under
`C:\Program Files\`, a silently redirected VirtualStore copy.
> **`Scripts.csproj` is overwritten deliberately.** The stock file omits `Scripts/Custom/`, so the
> plugin would sit in the tree and never compile — and ServUO would not tell you, because it
> ignores the script build's exit code and silently reloads the previous `Scripts.dll`. That
> failure mode is the reason [§6](#6-start-servuo-and-verify) exists.
---
## 4. The patch tier (optional)
Most of the plugin ships as **added** files, which is why the base install is a safe file copy. Two
features cannot: they need edits to stock ServUO sources, because the events they depend on do not
exist.
| Patch | Edits | Gives you | Rebuild needed |
|---|---|---|---|
| `playervendor-sale-eventsink.patch` + `playervendor-sale-gump.patch` | `Server/EventSink.cs`, `Scripts/Gumps/PlayerVendorGumps.cs` | `vendor.sale` events — player-vendor purchases with buyer, owner, price and commission, which is what cheat detection needs | **Core solution rebuild** (`dotnet build ServUO.sln`) — the dynamic script build is not enough |
| `commandlogging-event.patch` | `Scripts/Commands/Logging.cs` | In-game moderation actions (`[ban`, `[kick`, `[bcast`) forwarded to the website's moderation log as `admin.audit` | Script build only — a shard restart is enough |
Each patch has a companion `.cs` file that is copied **only after** its patch applies, because it
references symbols the patch introduces. That is why they are not in the base overlay: shipping them
unconditionally would break the build on every unpatched install.
How the installer handles it:
- **Opt-in.** The base install completes without it, and declining is a supported outcome, not a
degraded one.
- **Dry-run first, always.** Every patch is checked (`git apply --check`) before anything is
applied, and reported per patch. Most real shards are hand-modified; a patch that does not apply
is expected, not alarming.
- **All or nothing per feature.** The two vendor-sale patches are one unit and are applied together
or not at all.
- **Skipped entirely on a ServUO that is not 57.4**, with a warning. Unverified diffs are never
applied to an unknown tree.
- **Recorded, and the `.patch` files cached**, so re-runs stay idempotent and `uninstall` can print
the exact hunks to revert.
If it is skipped or fails, you lose exactly two things — **`vendor.sale` events** and **in-game
moderation audit forwarding**. Everything else works. You can apply the patches later by hand (see
`patches/README.md` in the tarball) and re-run `install` to record it.
---
## 5. Connect the website
The installer ends a successful run by printing the one manual step it cannot do for you:
```
Runic Gateway is installed.
One manual step remains — connect the website to this sidecar:
Base URL http://shard.example.com:8080
WebSocket URL ws://shard.example.com:8080/ws
Protocol version 3
Auth token 4f9c… (also in /etc/runicgateway/sidecar.toml)
Paste these into Admin → Shard on your Runic Gateway site:
https://your-site.example/admin/shard
The token is write-only once saved — the site will never show it back to you.
```
Every value there comes from asking the installed sidecar itself (`--print-config`), not from a
log file or a guess, so it cannot drift from what the service actually runs.
On your site, sign in as an administrator and open **Admin → Shard (uo-link)**:
| Field on the page | Paste |
|---|---|
| Enable the shard integration | ✔ on |
| Base URL (REST) | the **Base URL** line |
| WebSocket URL (feed) | the **WebSocket URL** line |
| Auth token | the **Auth token** line |
| Protocol | the **Protocol version** line (`3`) |
Saving restarts the site's ingest client, so the change takes effect immediately. The token is
AES-GCM encrypted at rest and **never returned to any client** — losing it means reading it back
from `sidecar.toml` on the shard host, not from the website.
If the sidecar sits behind a reverse proxy, paste your **public** `https://` and `wss://` URLs
instead of the two the installer printed — it composes those from the sidecar's own bind address,
which knows nothing about what fronts it. Everything else on the page is unchanged.
### If your website is on a different machine
The sidecar binds `127.0.0.1:8080` by default, which is reachable only from the shard host. If your
website runs elsewhere, the recommended arrangement is a **TLS reverse proxy in front of the
sidecar** — this is a supported deployment and the one Runic Gateway itself runs, on a real domain
name.
**Leave `[web] bind` on `127.0.0.1:8080`** and let the proxy be the only thing that talks to it.
Widening the bind and firewalling the port is the alternative, not the default (see below).
Give the website the **proxied** URLs — `https://link.example.com` and
`wss://link.example.com/ws` — in place of the `http://` / `ws://` pair the installer prints. Those
values are the sidecar's own view of itself; the proxy is what the outside world sees.
What the proxy must do:
| Requirement | Why |
|---|---|
| **Forward the WebSocket upgrade** (`Upgrade` / `Connection` headers, HTTP/1.1 to the upstream) | `/ws` is the live event feed. Without it the site's REST calls work and events never arrive — a confusing half-working state. |
| **Pass request headers through unmodified** | Auth is `Authorization: Bearer` (or `X-Api-Key`), and the website sends `X-UOLink-Version`. A proxy that strips unknown headers turns into a `401`, and a stripped version header just silently skips the mismatch check. |
| **Do not buffer the WS connection, and allow long-lived ones** | The feed is idle between events. The sidecar sends a WebSocket **Ping every 30 s**, so a read timeout of 60 s or more is safe as it stands — but a proxy that buffers responses will hold events instead of streaming them. |
| **Do not log query strings** | The sidecar also accepts `?token=…` (for clients that cannot set headers). If anything in your stack uses that form, a default access-log format writes your auth token to disk on every request. |
Nothing needs `X-Forwarded-For`: the sidecar never uses the client's IP for authorization, and the
browser IP that account provisioning cares about is supplied by the website in the request body.
An nginx server block that satisfies all of the above:
```nginx
server {
listen 443 ssl;
server_name link.example.com;
# your certificate directives here
location / {
proxy_pass http://127.0.0.1:8080;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade; # "upgrade" for WS, "" otherwise
proxy_set_header Host $host;
proxy_buffering off;
proxy_read_timeout 300s;
}
}
```
(with the usual `map $http_upgrade $connection_upgrade { default upgrade; '' close; }` at `http`
level). Caddy and Traefik handle WebSocket upgrades automatically and need no equivalent stanza.
**Without a proxy**, on a trusted network only: set `[web] bind` to `0.0.0.0:8080` or a specific LAN
address, restart the service, and **firewall the port to your website's address**. The auth token is
always required, but the sidecar speaks HTTP — on that path the token and every event cross the
network in the clear. Do not do this over the public internet.
The `[shard] bind` line is a different matter entirely: leave it on `127.0.0.1:7788` and never proxy
it. That socket accepts *inbound commands* to the game, and being loopback-only is what makes that
safe.
---
## 6. Start ServUO and verify
Start your shard the way you always do. Then confirm the bridge is actually live — not merely
installed. **A successful file copy is not a working bridge**: ServUO shells out to `dotnet build`,
prints the output, ignores the exit code, and reloads the existing `Scripts.dll`, so a broken script
build looks exactly like a clean boot.
**a. Watch the boot output.** You want to see the build succeed *and* the bridge announce itself:
```
Core: Compiling scripts...
Build succeeded.
[Bridge] enabled=True endpoint=127.0.0.1:7788 queueCap=10000 sweeps(stat=30s decay=60s …
```
If you scrolled past it, force the question:
```bash
cd <servuo root>
dotnet build Scripts/Scripts.csproj -c Release -p:Platform=x64 # must be 0 errors
```
**b. Ask the shard, in game.** As an Administrator:
```
[bridge status
```
It reports the config plus `connected=True depth=0 sent=… dropped=0 …`. `connected=False` means the
shard cannot reach the sidecar; `dropped` climbing means the sidecar is wedged and the shard is
shedding events rather than stalling — which it is designed to do. `[bridge reload` re-reads
`Bridge.cfg` without a restart; `[bridge sweepnow` forces one pass of every stream.
**c. Ask the sidecar.** `/health` needs no auth, so it is safe to curl from a terminal:
```bash
curl -s http://127.0.0.1:8080/health
{"status":"ok","protocol":3,"plugin_connected":true,"database":"ok","uptime":"2m","last_event":"2026-08-04T18:22:10.412Z"}
```
`plugin_connected: true` is the one that matters — it is the only value in this whole guide that
distinguishes "files copied" from "the bridge works".
**d. Ask the website.** The public site should stop showing the shard as offline, and live events
should appear on the admin dashboard within seconds.
---
## 7. Day two
### `runicgateway doctor`
The command that makes this supportable. Run it before asking anyone for help — its output is the
first thing a maintainer will want.
```
✓ ServUO found /opt/ServUO (57.4)
✓ Overlay in sync 24 files, all hashes match install.json
⚠ Patch tier 1 of 3 applied — vendor.sale unavailable
✓ uo-link installed 1.1.0
✓ Service running, enabled
✓ Sidecar reachable 127.0.0.1:8080 /health ok
✓ Protocol sidecar 3 = overlay manifest 3
✗ Shard connected no shard has dialed in since boot
```
Three of those rows come from asking the installed sidecar (`--version`, `--print-config`) rather
than from reading `install.json`, so `doctor` reports what the binary would actually do — including
which config and database file the *service* resolves — rather than what the installer believes it
was told. The overlay row compares live file hashes against both `install.json` and the release
manifest, which is how it tells "you edited a deployed file" from "the overlay moved on".
### `runicgateway update`
Re-resolves the bundle and moves both halves to a combination whose protocol versions were checked
together — never to two independently-latest artifacts that may disagree.
- **Sidecar**: download → verify → replace binary → restart service. No shard downtime.
- **Overlay**: download → verify → re-sync → record the new commit → **tell you to restart ServUO.**
It does not restart your shard.
Your `sidecar.toml`, your `Bridge.cfg` edits and your database are not touched. `Bridge.cfg` is
overwritten only if you have not changed it; a modified copy is reported, not clobbered.
### `runicgateway uninstall`
Removes what it exclusively owns, and **prints** everything else. The installer cannot know what you
have changed in your own server tree since deployment, so an automatic revert risks silently eating
your work.
| | |
|---|---|
| **Removed** | The sidecar binary, its service entry, `install.json`, the cached patch set |
| **Kept** | `sidecar.toml` and `uo-link.db` — config and history survive (`--purge` drops them) |
| **Printed, not done** | Every overlay file deployed into your ServUO tree, by path, for you to delete |
| **Printed, not done** | The exact hunks each applied patch added to `EventSink.cs`, `PlayerVendorGumps.cs` and `Logging.cs`, for you to revert |
The report is also written to a file, so it survives the scrollback.
---
## Troubleshooting
| Symptom | Cause and fix |
|---|---|
| **"ServUO is running — stop it before installing"** | Correct, and not overridable. `ServUO.exe` locks `Scripts.dll` and rewrites `Saves/` on exit; deploying underneath it corrupts one or both. Stop the shard, install, start it again. |
| Shard boots clean but nothing reaches the site | The classic silent failure: ServUO ignores the script build's exit code and reloaded a **stale `Scripts.dll`**. Run `dotnet build Scripts/Scripts.csproj -c Release -p:Platform=x64` and read the errors it prints. |
| `[bridge status` says `connected=False` | The sidecar is not listening on `127.0.0.1:7788`. Check the service is running, and that `[shard] bind` in `sidecar.toml` matches `Host`/`Port` in `Bridge.cfg`. |
| `[bridge` is not a command | The plugin did not compile, or `Bridge.cfg` has the bridge disabled. See the row above. |
| Website says the shard is offline; `/health` is fine locally | The website cannot reach the sidecar — bind address, firewall, or proxy. See [§5](#if-your-website-is-on-a-different-machine). Note that the site is *designed* to render normally with the shard offline, so this fails quietly by design. |
| REST reads work but **no live events arrive** | The classic reverse-proxy symptom: the WebSocket upgrade is not being forwarded. Confirm the proxy sets `Upgrade`/`Connection` and speaks HTTP/1.1 upstream, and that the site's WebSocket URL is `wss://…/ws` — not `https://`. |
| The event feed connects, then drops every minute or two | A proxy read timeout below the sidecar's 30 s WebSocket ping interval, or response buffering. Raise the timeout and turn buffering off. |
| Website logs `409` from the sidecar | Protocol mismatch: the number in Admin → Shard does not match the sidecar's. The sidecar rejects rather than mis-parsing. Set the field to what `/health` reports (`protocol`). If the *sidecar* and *overlay* disagree, you have a hand-assembled pair — reinstall from a bundle. |
| `401` from the sidecar | Wrong or missing auth token. Read the live one back with `uo-link-sidecar --print-config --config <path>`; do not retype it from a screenshot. |
| A patch will not apply | Expected on a hand-modified shard. The base install is unaffected; you lose only the two features in [§4](#4-the-patch-tier-optional). Apply the hunks by hand if you want them. |
| `vendor.sale` events never arrive despite patching | The `EventSink.cs` patch is a **core** change. A shard restart is not enough — rebuild the solution (`dotnet build ServUO.sln`). |
| Sidecar writes its database somewhere unexpected | A relative `[store] path` resolves against the directory holding `sidecar.toml` — not the working directory. Run `--print-config` to see the absolute path it will actually use. |
| Token leaked into a log or a screenshot | Clear `[web] auth_token` in `sidecar.toml`, restart the service (a new token is generated and saved), read it back with `--print-config`, and re-save it in Admin → Shard. |
---
## Appendix A — installing by hand
This is what the installer automates. It works today, on the current releases, and is the fallback
whenever you would rather not run an unsigned binary.
Throughout: `<servuo>` is your ServUO root, and **the shard is stopped**.
### A1. Fetch the bundle (so you install a checked pair)
```bash
curl -s https://gitea.whitlocktech.com/RunicGateway/installer/raw/branch/main/bundles/current.json
```
It names the sidecar tag, the overlay tag, their agreed `protocol`, and the SHA256 of every asset.
Use those versions together; that pairing is the only thing CI has verified.
### A2. Deploy the plugin overlay
```bash
curl -LO https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/releases/download/v0.1.1/runicgateway-overlay-0.1.1.tar.gz
curl -LO https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/releases/download/v0.1.1/SHA256SUMS
sha256sum -c SHA256SUMS --ignore-missing # must say: OK
tar xzf runicgateway-overlay-0.1.1.tar.gz # → runicgateway-overlay/
cd runicgateway-overlay
cat manifest.json # version, commit, protocol, per-file hashes
cp -r overlay/. <servuo>/ # adds files; overwrites Scripts/Scripts.csproj
```
On Windows, `Expand-Archive` does not read `.tar.gz`; use `tar.exe` (shipped with Windows 10+) and
`Copy-Item -Recurse -Force`. Plugin developers have `deploy.ps1` in the source repo, which does the
same copy with a hash diff and a `-Verify` dry run — it is not shipped in the tarball.
The overlay only ever **adds or overwrites**. Nothing in your tree is deleted.
*Optional — the patch tier* (stock ServUO 57.4 only; see `patches/README.md` in the tarball for the
full explanation):
```bash
cd <servuo>
git apply --check patches/playervendor-sale-eventsink.patch patches/playervendor-sale-gump.patch
git apply patches/playervendor-sale-eventsink.patch patches/playervendor-sale-gump.patch
cp patches/BridgeVendorSale.cs Scripts/Custom/Bridge/
dotnet build ServUO.sln # REQUIRED — EventSink.cs is a core file
git apply --check patches/commandlogging-event.patch
git apply patches/commandlogging-event.patch
cp patches/BridgeModerationAudit.cs Scripts/Custom/Bridge/
```
`git apply` works in a plain directory — the shard does not need to be a git repo. If you use
`patch` instead, note the core files are CRLF: use `patch --binary`.
### A3. Install the sidecar
```bash
curl -LO https://gitea.whitlocktech.com/RunicGateway/link/releases/download/v1.1.0/uo-link-sidecar-linux-x86_64
curl -LO https://gitea.whitlocktech.com/RunicGateway/link/releases/download/v1.1.0/SHA256SUMS
sha256sum -c SHA256SUMS --ignore-missing
sudo install -m 0755 uo-link-sidecar-linux-x86_64 /usr/bin/runicgateway-link
sudo mkdir -p /etc/runicgateway /var/lib/runicgateway
```
Provision the config and read back the token in one step. `--print-config` writes the file if it is
missing, generates the auth token if there is none, and prints the resolved settings as JSON — it is
the supported alternative to scraping the startup log:
```bash
sudo UOLINK_DB_PATH=/var/lib/runicgateway/uo-link.db \
/usr/bin/runicgateway-link --print-config --config /etc/runicgateway/sidecar.toml
```
```json
{
"component": "uo-link-sidecar",
"version": "1.1.0",
"protocol": 3,
"config_path": "/etc/runicgateway/sidecar.toml",
"config_created": true,
"token_generated": true,
"shard": { "bind": "127.0.0.1:7788" },
"web": {
"bind": "127.0.0.1:8080",
"ws_path": "/ws",
"auth_required": true,
"auth_token": "4f9c…"
},
"store": { "path": "/var/lib/runicgateway/uo-link.db" }
}
```
`config_created` and `token_generated` tell you whether *this* run provisioned anything — the values
alone cannot distinguish a fresh install from a re-read. **The output contains the auth token in
clear text**: keep it out of shell transcripts, logs and support bundles.
### A4. Register the service
**Linux** — `/etc/systemd/system/runicgateway-link.service`:
```ini
[Unit]
Description=Runic Gateway uo-link sidecar
After=network.target
[Service]
Type=simple
User=runicgateway
Environment=UOLINK_CONFIG=/etc/runicgateway/sidecar.toml
Environment=UOLINK_DB_PATH=/var/lib/runicgateway/uo-link.db
ExecStart=/usr/bin/runicgateway-link
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
```
```bash
sudo useradd --system --no-create-home runicgateway
sudo chown -R runicgateway /var/lib/runicgateway /etc/runicgateway
sudo systemctl daemon-reload
sudo systemctl enable --now runicgateway-link
systemctl status runicgateway-link
```
**Windows** (elevated PowerShell) — binary under `%ProgramFiles%`, data under `%ProgramData%`:
```powershell
New-Item -ItemType Directory -Force "$env:ProgramFiles\RunicGateway", "$env:ProgramData\RunicGateway" | Out-Null
Copy-Item .\uo-link-sidecar-windows-x86_64.exe "$env:ProgramFiles\RunicGateway\uo-link-sidecar.exe"
& "$env:ProgramFiles\RunicGateway\uo-link-sidecar.exe" --print-config --config "$env:ProgramData\RunicGateway\sidecar.toml"
sc.exe create RunicGatewayLink binPath= "\"$env:ProgramFiles\RunicGateway\uo-link-sidecar.exe\"" start= auto
sc.exe failure RunicGatewayLink reset= 86400 actions= restart/5000
[Environment]::SetEnvironmentVariable('UOLINK_CONFIG', "$env:ProgramData\RunicGateway\sidecar.toml", 'Machine')
[Environment]::SetEnvironmentVariable('UOLINK_DB_PATH', "$env:ProgramData\RunicGateway\uo-link.db", 'Machine')
sc.exe start RunicGatewayLink
```
Machine environment variables are read at service start, so set them before starting — and never
leave the config path to the default, which is relative to the service's working directory.
### A5. Connect the website, start the shard, verify
Exactly as in [§5](#5-connect-the-website) and [§6](#6-start-servuo-and-verify): paste the four
values into Admin → Shard, start ServUO, then check `[bridge status` in game and `/health` on the
sidecar.
### A6. Updating by hand
Re-read `current.json`, and if either version moved: replace the sidecar binary and restart its
service; re-extract the overlay tarball over your tree and restart ServUO. Keep the two in step —
`current.json` is the only statement that a given pair speaks the same protocol.
---
## Appendix B — `sidecar.toml` reference
Written on first run with a generated token. Environment variables override the file; the file
overrides these defaults.
```toml
[shard]
bind = "127.0.0.1:7788" # where the SHARD dials in. Keep this on loopback.
[web]
bind = "127.0.0.1:8080" # where the WEBSITE connects. Widen only with a firewall in front.
auth_token = "…" # generated if blank; the website's Admin → Shard "Auth token"
[store]
path = "uo-link.db" # relative paths resolve against this file's directory, not the CWD
```
| Environment variable | Overrides |
|---|---|
| `UOLINK_CONFIG` | Which config file to read (`--config <PATH>` outranks it) |
| `UOLINK_SHARD_BIND` | `[shard] bind` |
| `UOLINK_WEB_BIND` | `[web] bind` |
| `UOLINK_WEB_TOKEN` | `[web] auth_token` |
| `UOLINK_DB_PATH` | `[store] path` |
| Sidecar command | Output |
|---|---|
| `uo-link-sidecar --version` | `uo-link-sidecar 1.1.0 (protocol 3)` |
| `uo-link-sidecar --print-config [--config PATH]` | The JSON in [A3](#a3-install-the-sidecar). Provisions on first run. **Contains the token.** |
| `uo-link-sidecar --help` | Usage. An unrecognized argument exits `2` rather than starting a sidecar you did not ask for. |
Authentication is **always on**: a blank token is generated and written back, so the web surface is
never unauthenticated. `/health` is the one unauthenticated route, so monitoring can reach it.
---
## Appendix C — `Config/Bridge.cfg` settings worth reviewing
The file is deployed heavily commented and every setting has a working default — you can leave it
entirely alone. These are the ones most shards want to look at once. Run `[bridge reload` after
editing; endpoint changes take effect on the next reconnect.
| Setting | Default | Why you might change it |
|---|---|---|
| `LinkUrl` | `https://yoursite/link` | Shown in game when a player runs `[link` to connect their account. **Set this to your site.** |
| `PublicConnectAddress` | *(blank)* | The one connection detail the bridge will publish, e.g. `play.myshard.com,2593`. Blank omits it; `Server.cfg`'s address is **never** published automatically. |
| `AdminWriteEnabled` | `false` | Opt-in staff write plane: kick/ban/broadcast from the website. Authorization is enforced on the website; `AdminAccessFloor` is the shard-side floor that even a compromised sidecar cannot cross. |
| `MarketEnabled`, `MarketSweepSeconds`, `MarketSweepBatch` | `true`, `60`, `25` | The player-vendor index. Coverage takes `ceil(vendors / batch) × seconds` — 500 vendors is one full pass every 20 minutes at the defaults. |
| `PointsLeaderboardEnabled`, `PointsTopN`, `PointsSystems` | `true`, `10`, *(all shown on the loyalty gump)* | Standings boards. One frame **per system**, and ServUO carries ~25 of them, so a large `TopN` multiplies. |
| `RulesetEnabled`, `RulesetIncludeSchedule` | `true`, `true` | Publishes your ruleset (expansion, caps, systems on/off) to the site's rules page. Turn the schedule off if you would rather not advertise a predictable restart window. |
| `SignupMode` | `hybrid` | Which side may mint accounts — `website`, `game`, or `hybrid`. Pair `website` with `Accounts.AutoCreateAccounts=false`, or an in-game login still creates accounts. |
| `QueueCap` | `10000` | Outbound queue cap. On overflow the plugin **drops oldest** and counts drops, because a stalled sidecar must never take the shard down with it. |
Sweep intervals (`StatSweepSeconds`, `DecaySweepSeconds`, `EconomySweepSeconds`, and the rest) trade
freshness against Core-thread time. The measured cost is small — a vitals sweep is 0.0015 ms per
character, so 1000 online players is ~1.5 ms per pass — but there is rarely anything to gain by
hurrying them.
---
## Where to go next
| Doc | What |
|---|---|
| [PLAN.md](PLAN.md) | The installer's design of record — phases, locked decisions, the bundle model |
| [`bundles/README.md`](https://gitea.whitlocktech.com/RunicGateway/installer/src/branch/main/bundles/README.md) | The compat matrix: what a bundle is and how it is composed |
| [link/INTEGRATION.md](../link/INTEGRATION.md) | The sidecar's HTTP/WS API — for anyone integrating something other than the website |
| [link/ADMIN_CONTROLS.md](../link/ADMIN_CONTROLS.md) | The staff write plane in detail, before you turn `AdminWriteEnabled` on |
| [website/SHARD_VISIBILITY.md](../website/SHARD_VISIBILITY.md) | Which shard data each audience sees, configured on the website |
| [link/SHARD_PREREQS.md](../link/SHARD_PREREQS.md) | A worked example of diagnosing a shard whose scripts silently stopped compiling |

717
installer/PLAN.md Normal file
View File

@@ -0,0 +1,717 @@
# Runic Gateway Installer — plan
Status: **Phase 0 complete.** Every prerequisite in another repo has landed, the installer repo
publishes the bundle manifest, and [`INSTALL.md`](INSTALL.md) now specifies the operator-facing run
— so *what* the installer installs and *what using it looks like* both exist ahead of the binary.
No installer code exists yet; **Phase 1 is next.** This document is the design of record; it
supersedes the informal overview it grew out of, which described a ServUO integration that does not
match how `servuo-plugins` actually ships (see
[Corrections](#corrections-to-the-original-overview)).
| Phase 0 item | State |
|---|---|
| 0.1 `servuo-plugins` release workflow | ✅ Merged — [servuo-plugins#7](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/7) + [#8](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/8); first overlay release is [`v0.1.1`](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/releases/tag/v0.1.1) |
| 0.2 `link` installable (data paths + `--print-config`) | ✅ Merged — [link#24](https://gitea.whitlocktech.com/RunicGateway/link/pulls/24) (docs half [docs#84](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/84)); released as [`v1.1.0`](https://gitea.whitlocktech.com/RunicGateway/link/releases/tag/v1.1.0) |
| 0.3 Bundle CI in the installer repo | ✅ Merged — [installer#3](https://gitea.whitlocktech.com/RunicGateway/installer/pulls/3), plus the dispatch step in each component ([link#25](https://gitea.whitlocktech.com/RunicGateway/link/pulls/25), [servuo-plugins#9](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/9)). First bundle: [`2026.08.04`](https://gitea.whitlocktech.com/RunicGateway/installer/src/branch/main/bundles/current.json) |
| 0.4 This file + `INSTALL.md` | 🟨 In review — [docs#87](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/87). [`INSTALL.md`](INSTALL.md) is the operator guide, written before the binary because it *is* the specification of the run |
| — Repo bootstrap (governance + CI) | ✅ [`RunicGateway/installer`](https://gitea.whitlocktech.com/RunicGateway/installer) created; workflows merged ([installer#1](https://gitea.whitlocktech.com/RunicGateway/installer/pulls/1), [#2](https://gitea.whitlocktech.com/RunicGateway/installer/pulls/2)) |
---
## 1. Purpose
Take a stock ServUO installation and configure it for Runic Gateway with minimal manual steps, while
keeping the components separated and independently maintainable.
The installer handles environment detection, ServUO overlay deployment, the optional stock-file
patch tier, uo-link installation and service registration, version tracking, diagnostics, and
updates from Gitea releases.
**It is a deployment tool, not a hosted bootstrapper.** There is no `curl | bash`, no installer
service, and no hosted bootstrap script. Artifacts are downloaded from a Gitea release page and run.
**It does not replace ServUO startup behavior.** ServUO keeps running through its existing
release/start scripts. The installer never writes a launcher.
### Decisions locked
| Question | Decision |
|---|---|
| Audience | **Public** — any ServUO operator, not just shards we run |
| Code signing | **Unsigned.** `SHA256SUMS` is the trust anchor; SmartScreen/Gatekeeper warnings are expected and documented, as with most self-hosted tooling |
| Language | **Rust** — single static binary per OS, reuses the cross-compile pattern already proven in `link/.gitea/workflows/release.yml` |
| Plugin source | **Release tarball artifact** — no git and no Gitea credentials on the shard host |
| Composition | **Published bundle manifest** (§7.1). CI names an exact, protocol-checked combination of component versions; the installer fetches it at run time and `--bundle <tag>` pins one. Component releases regenerate JSON, not the installer binary |
| Token handoff | **Print token + prefilled admin URL** at the end of the run |
| Repo | **New repo**, `RunicGateway/installer`. It deploys *both* other components, so living inside `link/` would invert the dependency |
| ServUO version | **Warn and skip.** Patches are verified against stock 57.4 only; on anything else the base install proceeds and the patch tier is skipped with a warning. Forks are the norm in a public audience — refusing outright would block most operators |
| Uninstall | **Never touches the ServUO tree.** Removes uo-link and its service entry, then *prints* the overlay files to delete and the patch hunks to revert. Reverting is the operator's call |
---
## 2. Corrections to the original overview
These are not wording nits — each one changes what the installer has to do.
### 2.1 There is no `RunicGateway.dll` and no `Plugins/` directory
The plugin ships as **C# source** and ServUO compiles it at boot. The real deployable is
`servuo-plugins/overlay/`, which mirrors the server root:
```
overlay/
├── Config/Bridge.cfg
└── Scripts/
├── Scripts.csproj # Phase 0 — whole-file overwrite of a stock file
└── Custom/Bridge/*.cs # 22 files
```
So the plugin step is a hash-compare file sync, not a DLL drop — mechanically easier than the
overview assumed. The sting is that **a successful copy does not mean a working bridge.** Per
`link/SHARD_PREREQS.md`, `ScriptCompiler.Compile()` shells out to `dotnet build`, prints the output,
**ignores the exit code**, and reloads the existing `Scripts.dll`. A broken script build is
invisible: the shard boots clean on stale code. Diagnostics must therefore verify *post-boot* state,
never treat "files copied" as success.
### 2.2 Stock ServUO files *are* modified — by an optional tier
`servuo-plugins/patches/` holds unified diffs against stock ServUO 57.4, plus two `.cs` files that
can only be copied *after* their patch lands (they reference symbols the patch introduces):
| Patch | Target | Companion file | Rebuild required |
|---|---|---|---|
| `playervendor-sale-eventsink.patch` | `Server/EventSink.cs` | `BridgeVendorSale.cs` | **Core** — `dotnet build ServUO.sln`; the dynamic script build is not enough |
| `playervendor-sale-gump.patch` | `Scripts/Gumps/PlayerVendorGumps.cs` | (same unit as above) | script build |
| `commandlogging-event.patch` | `Scripts/Commands/Logging.cs` | `BridgeModerationAudit.cs` | script build |
Plus `overlay/Scripts/Scripts.csproj`, which overwrites a stock file (Phase 0 — it fixes the silent
ServUO build bug above).
This is the hardest part of the installer. `git apply` against a hand-modified shard will fail, and
most real shards are hand-modified. Therefore:
- The patch tier is **opt-in and skippable**. The base install must complete without it.
- Always dry-run (`git apply --check`) before applying, and report per-patch.
- When skipped or failed, say plainly what is lost: **no `vendor.sale` events, no in-game moderation
audit forwarding**.
- The `EventSink.cs` patch must warn loudly that a **core solution rebuild** is required, not just a
shard restart.
- On any ServUO version other than stock **57.4**, skip the whole tier with a warning and continue
with the base install. Do not attempt to apply unverified diffs to an unknown tree.
- Record applied patches in `install.json`, **and cache the applied `.patch` files** next to it
(`/etc/runicgateway/patches/`, `%ProgramData%\RunicGateway\patches\`). Re-runs stay idempotent,
and uninstall can print the exact hunks offline long after the release tarball is gone (§5,
Phase 4).
### 2.3 Config paths collide with what the sidecar actually reads
The sidecar reads `$UOLINK_CONFIG`, else `sidecar.toml` in the **working directory**
(`link/sidecar/src/config.rs`), with keys `[shard].bind`, `[web].bind`, `[web].auth_token`,
`[store].path`. The overview proposed a `config.toml` with `[updates]`, `[link]`, `[servuo]` — keys
the sidecar cannot read.
Two files, two owners:
| File | Owner | Contents |
|---|---|---|
| `/etc/runicgateway/sidecar.toml` | uo-link | The sidecar's own schema, unchanged. Service sets `UOLINK_CONFIG` to this path |
| `/etc/runicgateway/install.json` | installer | Deployed versions, file hashes, applied patches, ServUO path, timestamps |
**Working-directory trap:** the sidecar wrote both `sidecar.toml` and `uo-link.db` relative to CWD.
Under `C:\Program Files\` that fails or silently lands in VirtualStore. Phase 0.2 fixed the second
half in the sidecar — a relative `[store].path` now resolves against the directory holding
`sidecar.toml`, so pinning the config alone is enough to put the database somewhere deterministic —
but the config path itself is still CWD-relative by default, and "deterministic" is not the same as
"where this install wants it". The service definitions therefore still pin `UOLINK_CONFIG` and
`UOLINK_DB_PATH` explicitly:
- Linux: config `/etc/runicgateway/sidecar.toml`, db `/var/lib/runicgateway/uo-link.db`, dedicated
service user
- Windows: binary under `%ProgramFiles%\RunicGateway\`, **data under `%ProgramData%\RunicGateway\`**
### 2.4 The token handoff was missing entirely
The whole point is the website reaching the sidecar, and today that is manual and undocumented in
the install flow: the sidecar generates a token on first run and logs it, then a human pastes base
URL, WS URL, token, and protocol version into Admin → Shard, where it is AES-GCM encrypted and
becomes write-only. This is the largest "I installed it and nothing happened" failure mode.
The installer closes it by printing a copy-paste block at the end of a successful run — see §6.
Phase 0.2 supplied the missing half of that: `uo-link-sidecar --print-config` provisions the config
if absent and prints the resolved settings — token, both binds, `ws_path`, protocol version, db
path — as JSON. The installer reads the block it prints out of that one call. **It never parses the
log**, which was the alternative and would have made the handoff depend on a log format that is not
a contract.
### 2.5 `deploy.ps1` cannot be the cross-platform deployer
It is PowerShell-only; a Linux ServUO host running .NET typically has no `pwsh`. It also hard-throws
when the ServUO process is running — correct behavior, and the installer must inherit it (detect and
refuse, rather than corrupt a live `Scripts.dll`). The installer reimplements the sync natively; it
is a short hash-compare-and-copy that never deletes.
`deploy.ps1` **stays** in `servuo-plugins` as the developer-facing tool. The installer is for
operators.
### 2.6 Prerequisites the overview assumed away
- **`servuo-plugins` had no release workflow.** Only `link` did. "Pull latest repository" is replaced
by a release tarball, which had to be built first — Phase 0 item 1, now in review
([servuo-plugins#7](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/7)).
- **arm64 is not buildable today.** `link/release.yml` cross-compiles only
`x86_64-unknown-linux-gnu` and `x86_64-pc-windows-gnu`. An arm64 `.deb` needs another cross
toolchain.
- **The compat matrix has no home.** `PROTOCOL_VERSION` lives in `link/sidecar/src/main.rs`. The
sidecar publishes it via `X-UOLink-Version` and `/health`, and the website stores an expected
value — but the *plugin's* protocol version is not queryable before boot. Phase 0 item 1 gives it
a home: `servuo-plugins/overlay.toml`, declared into the overlay manifest. See §7.0 / §7.1.
---
## 3. Distribution model
Components are published as Gitea release artifacts. Operators download from the release page
(browser, `curl`/`wget`, or `scp` to the server) and run the binary.
```
Runic Gateway Installer v1.0.0
├── runicgateway-installer-windows-x86_64.exe
├── runicgateway-installer-linux-x86_64
└── SHA256SUMS
uo-link v1.1.0 (existing release, extended)
├── uo-link-sidecar-windows-x86_64.exe
├── uo-link-sidecar-linux-x86_64
├── runicgateway-link_<ver>_amd64.deb (Phase 5)
└── SHA256SUMS
servuo-plugins v<ver> (new release, Phase 0)
├── runicgateway-overlay-<ver>.tar.gz # overlay/ + patches/ + manifest.json
└── SHA256SUMS
```
Binding those together is the **bundle manifest** (§7.1) — published by the installer repo's CI, not
by any component, and the thing the installer actually resolves against.
```bash
scp runicgateway-installer-linux-x86_64 user@server:/tmp/
chmod +x runicgateway-installer-linux-x86_64
sudo ./runicgateway-installer-linux-x86_64
```
### Unsigned-binary posture
Because releases are unsigned, trust is anchored on checksums and the operator's own verification.
The docs must state this up front rather than let users discover it as a scary dialog:
- Every release publishes `SHA256SUMS`; the install docs lead with the verification command for both
OSes.
- Windows will show a SmartScreen "unrecognized app" prompt. Documented, with the exact click path.
- The installer verifies the SHA256 of everything **it** downloads (overlay tarball, sidecar binary)
against the release's `SHA256SUMS` and refuses on mismatch. Self-verification is not optional just
because the installer itself is unsigned.
- Revisit signing if it ever becomes affordable; the release layout should not have to change.
---
## 4. Component architecture
```
Runic Gateway Installer (Rust, one binary per OS)
│
┌───────────────┴────────────────┐
▼ ▼
ServUO integration uo-link
│ │
┌────────┴────────┐ ┌────────┴────────┐
▼ ▼ ▼ ▼
overlay sync patch tier (opt-in) binary install service registration
(never deletes) (git apply + guard) + config + data (systemd / Windows SCM)
```
Each component keeps its own lifecycle. ServUO's existing startup process is untouched.
---
## 5. Phases
### Phase 0 — prerequisites (no installer code)
Repo work that must land before an installer can exist.
1. **`servuo-plugins`: add `.gitea/workflows/release.yml`.** Retarget the release *engine* half of
`link/release.yml` (its header comment explicitly anticipates this — the plan/release steps
consume only `{version, changelog, artifacts}`). The adapter half produces
`runicgateway-overlay-<ver>.tar.gz` containing `overlay/`, `patches/`, and a `manifest.json`
(version, commit, per-file SHA256, declared protocol version, minimum ServUO version).
As built ([servuo-plugins#7](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/7)),
with three deviations from `link`'s copy that each fell out of the repo rather than being chosen:
- **No build gates, structural gates instead.** Nothing in that repo can be compiled without
ServUO reference assemblies, so CI asserts what it honestly can: `Bridge.cfg` and the Bridge
scripts present, `Scripts.csproj` present (its absence ships code that never compiles while
ServUO reports success — §2.1), every `.patch` parseable via `git apply --stat`, and each
patch's companion `.cs` present.
- **No bump commit, so no push to `main`.** `link` writes the version into `Cargo.toml` because
the binary embeds it; the tarball embeds nothing but the generated manifest, so the tag *is*
the version. That workflow needs no branch-protection exception.
- **`overlay.toml` at the repo root** holds the declared `protocol` and the ServUO compatibility
values, read by CI into the manifest. It exists because the number needs one maintained home —
see §7 for why the plugin cannot simply be asked.
The tarball uses a **fixed** top-level directory, `runicgateway-overlay/`, not a versioned one:
the installer looks for `overlay/`, `patches/` and `manifest.json` at known paths rather than
parsing the version it is trying to read. Member order, mtime and ownership are pinned, so a
given tree yields a byte-identical tarball and its checksum moves only when its contents do.
2. **`link`: make the sidecar installable.** Confirm/settle default data paths, and add a way to
read back config non-interactively (e.g. `--print-config` emitting JSON: bind addresses, token,
protocol version, db path) so the installer does not have to scrape logs for the token.
As built ([link#24](https://gitea.whitlocktech.com/RunicGateway/link/pulls/24)) — the sidecar
had **no CLI at all** before this, so the shape was chosen rather than inherited:
- **Four flags, hand-rolled:** `--print-config`, `--config <PATH>`, `--version`, `--help`. No
argument-parsing crate — it would be larger than the code it replaced — and deliberately no
flags that duplicate a config key, so `sidecar.toml` stays the single place settings live.
An unrecognized argument exits `2`; silently ignoring a typo'd flag would start a sidecar that
is not the one the installer asked for.
- **`--print-config` performs first-run setup rather than only reporting.** It runs the same
load path a normal start does, so a missing config file is written and a blank token is
generated and saved. That collapses "provision the sidecar" and "find out its token" into one
non-interactive call — which is exactly the sequence §6 needs. `config_created` and
`token_generated` say whether *this* run did either, because the values alone cannot
distinguish a fresh install from a re-read of an existing one, and a re-run must not report a
token as newly minted.
- **The document is the whole of stdout.** The log subscriber writes to stdout, so it is not
started in this mode. `ws_path` is emitted from the same constant the route is registered
with, so the installer's WebSocket URL cannot drift from the server's.
- **Relative `[store].path` now anchors to the config file's directory, not the CWD** — see
§2.3, which this half-closes on the sidecar side. Absolute paths are used as written; parent
directories are created; `:memory:` and `file:` URIs are left alone.
- **The db path is handed to sqlx as a path, not a `sqlite://` URL.** The URL spelling is
parsed as one: it percent-decodes the path and splits it on `?`, so an installed path
containing `%20` opened a different file than the operator named.
- **No platform data directories are compiled in.** That is the "settle" half of this item, and
the answer is that the *installer* owns layout (§2.3) and pins `UOLINK_CONFIG` /
`UOLINK_DB_PATH` in the service definition. Baking `/etc` and `%ProgramData%` defaults into
the binary would give the same paths two owners and break `cargo run` in a working tree.
3. **Bundle CI in the installer repo** (§7). Compose job (read both repos' latest releases → run the
two gates → publish `bundle.json`), the nightly cron, and the dispatch step appended to each
component's release workflow. This must exist before Phase 1 is useful, since the installer
resolves what to install *from* the bundle.
As built ([installer#3](https://gitea.whitlocktech.com/RunicGateway/installer/pulls/3),
[link#25](https://gitea.whitlocktech.com/RunicGateway/link/pulls/25),
[servuo-plugins#9](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/9)) —
`installer/.gitea/workflows/bundle.yml`, with the decisions §7 had left open:
- **Bundles are committed to the installer repo, not published as releases** — see §7.1 for
where and why. That was the one genuinely open question here, and the deciding factor is that
this repo's *own* releases are the installer binaries.
- **Gate 1 reads the sidecar's protocol from source at the release tag**, not from the binary.
`--print-config` (Phase 0.2) would answer authoritatively, but only for releases from `v1.1.0`
onward, and `--bundle <tag>` has to be able to recompose a bundle from an older pair. Reading
`sidecar/src/main.rs` at the tag the release was built from works uniformly, needs no execution
of a downloaded artifact, and does not provision a throwaway config whose auth token would then
be sitting in a CI log. A constant that has moved or been renamed is a hard failure — treating
"could not read" as "matches" is exactly how a mismatched pair would ship.
- **Gate 2 records the hash CI computed itself**, after verifying the download against the
publishing repo's `SHA256SUMS`. It also asserts the reverse direction — an asset with *no*
`SHA256SUMS` entry — because `sha256sum -c` silently passes over a file the sums file does not
mention, which would put an unverified artifact in the bundle.
- **Release metadata is read anonymously**, on purpose: those are exactly the requests the
shipped installer makes on a host with no Gitea credentials, so a repo flipped to private
fails CI here instead of on an operator's machine.
- **An unrecognized asset name is a hard failure.** link's binaries are mapped onto platform keys
by suffix; adding a target (aarch64, macOS) to its release workflow therefore reddens this job
rather than silently omitting the new binary from every bundle.
- **A run that changes nothing writes nothing** — the comparison excludes `bundle` and
`generated`, which are metadata about the run. Without that the nightly cron would commit a
dated duplicate of the same matrix every morning.
The workflow's compose steps were run against the live releases before merge, producing the
first bundle (`2026.08.04`: link `v1.1.0` + overlay `v0.1.1`, protocol 3), which is committed so
the manifest exists ahead of the binary that reads it.
4. **`docs`: this file, plus `docs/installer/INSTALL.md`** (the operator-facing guide) once the
shape is settled.
As built ([`INSTALL.md`](INSTALL.md)) — written *before* the binary on purpose. Everything it
installs is already released (items 1–3), so the guide is not speculation about a tool that
might exist; it is the specification of what the run asks, where it writes, what it prints, and
what the operator does next. Phase 1–4 implement it.
- **It is useful before the installer exists.** Appendix A is the same deployment done by hand —
bundle fetch, tarball verify + overlay copy, the optional patch tier, `--print-config`
provisioning, and a systemd unit / `sc create` service — composed from the released artifacts'
actual contents and the sidecar's config and CLI source rather than from memory. That appendix
doubles as **Phase 1's acceptance test**: walking it end to end on a real shard is what proves
the automated path has nothing left to discover.
- **The installer does not install itself.** §5's `runicgateway doctor` sketch implied a name on
`PATH`; nothing places one there, and adding self-installation would give the tool a second
lifecycle to manage. The guide names the downloaded artifact, says to keep it, and shortens it
in later examples.
- **The flag surface got fixed here**, because a guide cannot describe a run in the abstract:
`--verify`, `--bundle`, `--purge` were already named by §5/§7; `--servuo`, `--patches` /
`--no-patches`, `--host`, `--site-url` and `--yes` are the remainder, chosen so every prompt
in §6's handoff has a non-interactive equivalent and an unattended install is expressible.
- **A modified `Bridge.cfg` must survive an update** — see Phase 1, where this changes the sync
rule inherited from `deploy.ps1`.
- **Remote-website deployments needed an answer, and it is a reverse proxy.** `[web] bind`
defaults to `127.0.0.1`, which only works when the site runs on the shard host. The guide's
recommended arrangement — already in production on a real domain — is to **leave the bind on
loopback** and put a TLS reverse proxy in front, giving the website the proxied `https://` /
`wss://` URLs in place of the pair the installer prints from the bind address. Four
requirements make that work and are stated with an nginx block that satisfies them: forward
the WebSocket upgrade (`/ws` is the whole live feed), pass headers through unmodified (auth is
`Authorization: Bearer`, and a stripped `X-UOLink-Version` silently skips the mismatch check),
do not buffer and allow long-lived connections (the sidecar pings every 30 s, so a ≥60 s read
timeout is safe), and do not log query strings (`?token=` is an accepted auth form). Widening
the bind and firewalling the port stays documented as the trusted-LAN alternative, not the
default, because on that path the token crosses the network in the clear. `[shard] bind` is
never proxied and never widened — that socket carries inbound commands *into* the game.
### Phase 1 — installer core
- ServUO root detection and validation (`ServUO.exe`, `Scripts/`, `Config/`), with version detection
and an explicit refusal when the ServUO process is running.
- Overlay sync: fetch tarball → verify SHA256 → hash-compare against the server tree → add/change,
**never delete**. Port of `deploy.ps1` semantics including its `-Verify` dry run (`--verify`).
- **One deviation from `deploy.ps1`: an operator-modified `Config/Bridge.cfg` is reported, not
overwritten.** `deploy.ps1` overwrites every file whose hash differs, which is right for a
developer redeploying their own tree and wrong for an operator who has set `LinkUrl`,
`PublicConnectAddress` and sweep intervals — an `update` would silently revert the shard's entire
configuration. `install.json` records the hash deployed, so the installer can distinguish "the
operator edited this" from "the overlay moved on" (§7.0) and act only on the second. The rule is
specific to `Bridge.cfg`: it is the only file in the overlay that is *meant* to be edited in
place, and it carries no code, so a stale copy cannot break the build. Every `.cs` file and
`Scripts.csproj` still overwrite unconditionally.
- Write `install.json`: component, version, source commit, per-file hashes, applied patches,
timestamp.
- Idempotent re-runs; a second run with no upstream change reports "unchanged" and writes nothing.
### Phase 2 — uo-link install and service
- Linux: binary → `/usr/bin/runicgateway-link`, config → `/etc/runicgateway/sidecar.toml`, db →
`/var/lib/runicgateway/`, systemd unit with a dedicated user, `enable` + `start`.
- Windows: `%ProgramFiles%\RunicGateway\`, data in `%ProgramData%\RunicGateway\`, service
registration with automatic start and restart-on-failure.
- Both: `UOLINK_CONFIG` and `UOLINK_DB_PATH` pinned in the service definition (§2.3).
- Token surfacing (§6): run the installed binary once as
`uo-link-sidecar --print-config --config <the pinned path>` **before** registering the service.
That both writes the config the service will read and returns the token to print, so the service
never starts against a config that does not exist yet.
### Phase 3 — patch tier (opt-in)
Everything in §2.2. Detect applicability, dry-run, apply, record, warn about the core rebuild, and
degrade loudly rather than silently.
### Phase 4 — diagnostics and updates
`runicgateway doctor` — the command that makes the whole thing supportable:
```
✓ ServUO found /opt/ServUO (57.4)
✓ Overlay in sync 24 files, all hashes match install.json
⚠ Patch tier 1 of 3 applied — vendor.sale unavailable
✓ uo-link installed 1.1.0
✓ Service running, enabled
✓ Sidecar reachable 127.0.0.1:8080 /health ok
✓ Protocol sidecar 3 = overlay manifest 3
✗ Shard connected no shard has dialed in since boot
```
The last check matters most: it is the only thing that distinguishes "files copied" from "the bridge
actually works" (§2.1).
Three of those rows are answered by the sidecar's own CLI rather than by inspecting the filesystem:
`--version` prints `uo-link-sidecar <ver> (protocol <n>)`, and `--print-config` gives the config and
db paths the *installed service* resolves — so `doctor` reports what the binary would actually do,
not what `install.json` believes it was told to do. The protocol row compares that number against
the overlay manifest's declared one (§7.0).
`runicgateway update` — resolves the current bundle (§7.1), then acts asymmetrically by component,
deliberately:
- **uo-link**: compare the bundle's version against what is installed → download → verify checksum →
replace binary → restart service.
- **plugin overlay**: download the bundle's overlay tarball → verify → re-sync (leaving a modified
`Bridge.cfg` alone — Phase 1) → record commit → tell the operator ServUO must restart (the
installer does not restart the shard).
Because both come from one bundle, an update always moves to a combination whose protocol versions
were checked together, rather than to two independently-latest artifacts that may disagree.
`runicgateway uninstall` — **removes only what it exclusively owns, and never edits the ServUO
tree.** The installer cannot know what the operator has changed in those files since deployment, so
a clever automatic revert risks silently eating their work. It removes and it reports:
| Action | Scope |
|---|---|
| Removed | uo-link binary, its service entry (systemd unit / Windows service), `install.json` and the cached patch set |
| Kept | `sidecar.toml` and `uo-link.db` (config and history survive; `--purge` to drop them) |
| **Printed, not done** | Every overlay file deployed into the ServUO tree, listed by path, for the operator to delete |
| **Printed, not done** | The exact hunks each applied patch added to `EventSink.cs`, `PlayerVendorGumps.cs`, `Logging.cs`, rendered from the cached `.patch` files, for the operator to revert by hand |
The printed report is also written to a file, so it survives the terminal scrollback of a long
uninstall.
### Phase 5 — packaging polish
`.deb` packaging, Windows MSI, arm64 cross build, and optional automated backup before upgrade.
Deliberately last: v1 can register services directly (`sc create` / a written systemd unit) and ship
plain binaries. Nothing in Phases 1–4 should have to change to add these.
---
## 6. Token handoff (the end of a successful run)
```
Runic Gateway is installed.
One manual step remains — connect the website to this sidecar:
Base URL http://<this-host>:8080
WebSocket URL ws://<this-host>:8080/ws
Protocol version 3
Auth token 4f9c... (also in /etc/runicgateway/sidecar.toml)
Paste these into Admin → Shard on your Runic Gateway site:
https://<your-site>/admin/shard
The token is write-only once saved — the site will never show it back to you.
```
Every value in that block except the host and the site URL comes from one
`uo-link-sidecar --print-config` call (§2.4): `web.auth_token`, `protocol`, and `web.bind` +
`web.ws_path` for the two URLs. Only the **host** is substituted — `web.bind` is frequently
`0.0.0.0`, which is not something to hand a website — so the installer composes the URLs from the
host it detects or prompts for, rather than echoing the bind address.
**It does not attempt to detect a reverse proxy**, which is the recommended arrangement for a
website on another host (`INSTALL.md` §5). Nothing visible from the sidecar's side says what fronts
it, so guessing would produce a confidently wrong `https://` URL. The two printed URLs always
describe the sidecar itself, and the guide tells the operator to paste their public `https://` /
`wss://` pair instead when there is a proxy. `--host` accepting a full origin later is a cheap
improvement if this proves annoying in practice.
The installer prompts for the site URL only to build that link; it never contacts the website. A
future "installer registers itself with the website" flow (claim code + authenticated endpoint) is
explicitly **out of scope** — it is real backend work in a security-sensitive area and can be added
later without changing anything here.
The printed token is a secret in transit: `--print-config` output must go to the operator's
terminal and the config file, never into an installer log file or a support bundle.
---
## 7. Version tracking, the bundle, and release orchestration
Three components version independently, bound by a protocol contract:
- **sidecar** — `PROTOCOL_VERSION` in `link/sidecar/src/main.rs`, exposed on `/health` and as
`X-UOLink-Version` on every response; a mismatch is rejected `409`.
- **website** — stores an expected protocol version in `uoLinkConfig` (admin-managed).
- **plugin overlay** — has no queryable version before ServUO boots. The overlay release
`manifest.json` declares it, and `install.json` records what was deployed.
### 7.0 The overlay manifest
Shipped inside every `runicgateway-overlay-<ver>.tar.gz`, generated by that repo's release workflow:
```json
{
"component": "servuo-plugins-overlay",
"version": "0.1.0",
"commit": "968b526…",
"repo": "RunicGateway/servuo-plugins",
"protocol": 3,
"servuo": { "min_version": "57.4", "patches_verified_against": "57.4" },
"files": { "overlay/Config/Bridge.cfg": "32718424…", "patches/…": "…" }
}
```
`version` and `commit` come from the release engine; `protocol` and the `servuo` block are read from
`servuo-plugins/overlay.toml`; `files` is a SHA256 per shipped file.
Two of these carry weight beyond documentation:
- **`protocol` is a hand-maintained declaration, and has to be.** The plugin announces no version on
the wire and none is queryable before ServUO boots, so nothing in CI can derive it — which makes
this line the only thing §7.1's gate 1 has to compare the sidecar against. The duty is stated in
`overlay.toml` and in that repo's README: **bump it in the same PR that changes the emitters**, the
way `link` bumps `PROTOCOL_VERSION`.
- **`files` is what makes `doctor` able to tell "the operator edited a deployed file" from "the
overlay moved on"** (§5, Phase 4). The installer copies these hashes into `install.json` at deploy
time; a later mismatch against *both* the manifest and `install.json` means upstream changed, a
mismatch against `install.json` alone means local edits.
`min_version` and `patches_verified_against` are separate on purpose. The base overlay only *adds*
files and is expected to work broadly; the patch tier diffs stock ServUO files and is verified
against exactly one version (§2.2).
### 7.1 The bundle manifest
**The bundle is the compat matrix.** Rather than the installer hardcoding versions or blindly
resolving "latest", CI publishes a small manifest naming an exact, checked combination:
```json
{
"schema": 1,
"bundle": "2026.08.04",
"generated": "2026-08-04T16:07:13Z",
"protocol": 3,
"link": {
"repo": "RunicGateway/link", "tag": "v1.1.0", "version": "1.1.0", "protocol": 3,
"assets": {
"linux-x86_64": { "name": "uo-link-sidecar-linux-x86_64", "url": "…", "sha256": "27d491ef…" },
"windows-x86_64": { "name": "uo-link-sidecar-windows-x86_64.exe", "url": "…", "sha256": "fbefd886…" }
}
},
"overlay": {
"repo": "RunicGateway/servuo-plugins", "tag": "v0.1.1", "version": "0.1.1",
"commit": "3a52abb…", "protocol": 3,
"servuo": { "min_version": "57.4", "patches_verified_against": "57.4" },
"asset": { "name": "runicgateway-overlay-0.1.1.tar.gz", "url": "…", "sha256": "75dc6d6c…" }
}
}
```
Note `link.assets` is a **map keyed by platform**, not the single `sha256` this section originally
sketched: link publishes a Linux binary and a Windows `.exe`, and the installer runs on both, so one
hash could only ever have described one of them. `schema` versions this document's shape and is
independent of `protocol` and of either component's release version — all three move separately.
The installer fetches the current bundle at run time; `--bundle <tag>` pins an older one for a
reproducible install. Because the bundle is data, **a new `link` release regenerates ~30 lines of
JSON and leaves the installer binary untouched** — operators do not re-download the installer to
pick up a sidecar patch, and the installer does not accumulate releases whose code is byte-identical.
Two gates run at compose time, both cheap and both worth it:
1. The sidecar's `PROTOCOL_VERSION` must equal the overlay manifest's declared protocol version.
This is the check that catches an `edge`/`main` protocol mismatch before it reaches an operator.
The two halves are read from different places because they *are* different: the overlay's from
`manifest.json` inside the tarball (the only statement of it that exists — §7.0), the sidecar's
from `sidecar/src/main.rs` at the release tag (see Phase 0 item 3 for why not from the binary).
2. Every referenced asset must exist and its SHA256 must match the publishing repo's `SHA256SUMS`.
The hash recorded in the bundle is the one CI computed from the asset it downloaded, *after* that
check — and the installer verifies every download against it. These artifacts are deliberately
unsigned (§3), so the checksum is the whole trust anchor; a hash copied from a file nobody
verified would make the chain decorative.
#### Where bundles are published
Committed to the installer repo under `bundles/`, so the installer's fetch is a plain anonymous
`GET` against a public repo — the shard host has no Gitea credentials (§1):
```
bundles/current.json → …/RunicGateway/installer/raw/branch/main/bundles/current.json
bundles/bundle-<tag>.json → …/raw/branch/main/bundles/bundle-2026.08.04.json (--bundle)
```
Every bundle is kept forever, so `--bundle` stays reproducible. Tags are UTC dates; a second bundle
on the same day — a sidecar release in the morning and an overlay release in the afternoon is the
normal way that happens — becomes `2026.08.04.2`, so one tag always names exactly one matrix.
**Not one Gitea release per bundle**, which was the obvious alternative. This repo's own releases
are the installer *binaries*, and `/releases/latest` returns whichever release is newest regardless
of kind — interleaving bundle releases would make "latest" intermittently resolve to a release
carrying no installer binary. Committing also yields a reviewable diff and a git history of the
compat matrix, and needs no new branch-protection exception: `release.yml`'s version-bump commit
already requires the CI user to be able to push to `main`.
### 7.2 What triggers a bundle
| Trigger | Why |
|---|---|
| `link` publishes a release | Its release job `POST`s to the installer repo's workflow-dispatch endpoint as its final step |
| `servuo-plugins` publishes a release | Same. Phase 0 item 1 gave it the release workflow; the dispatch step was left as a marked TODO until there was something to dispatch, and landed with the bundle CI it calls (item 3) — a step that `404`s on every release is worse than no step |
| Nightly cron on the installer repo | Recomputes from whatever the latest releases actually are, so a missed or failed dispatch self-heals instead of silently pinning operators to a stale sidecar |
`repository_dispatch` is deliberately avoided — support for it is uncertain on this Gitea version,
whereas dispatching an existing `workflow_dispatch` workflow via the API works today.
**A failed dispatch is a warning, never a failed release.** By the time that step runs the component
release is published and correct; failing the job would misreport it. This also keeps the dispatch
from becoming a new hard credential requirement — `REGISTRY_TOKEN` having write on the installer
repo is a nicety, and without it the nightly cron picks the release up anyway. A dropped dispatch
costs latency, not correctness, which is the whole reason the cron exists.
### 7.3 Stale-overlay handling: dispatch, don't wait
Each component **self-releases on merge to its own `main`**, using the same conventional-commit
engine. Note that "updated since the last release" must mean *releasable* commits — the engine sets
`RELEASE=false` when nothing but `docs:`/`chore:` has landed, so a docs typo correctly does **not**
cut an overlay release, and the bundle keeps using the existing one.
The compose job's copy of that rule additionally **excludes merge commits**, whose subject is
`Merge pull request '<the real subject>'`. Without that, every squash-free merge of a `feat:` branch
would be counted twice, and worse, a merge of a `docs:` branch whose *title* happens to quote a
`fix:` would be read as releasable — re-dispatching, every night, a release workflow that correctly
declines to run.
So by the time the installer's CI looks, the release normally already exists. If it finds
`servuo-plugins` main ahead of its latest release *with* releasable commits, it:
1. fires that repo's release workflow via workflow-dispatch and **does not wait for it**,
2. composes this bundle from the assets that exist right now,
3. writes a loud warning into the job summary.
The new overlay release lands minutes later on its own and the nightly cron folds it into the next
bundle. This gets the automation without the flaky part: dispatching another repo's workflow is
fine — that workflow still runs its own gates — but *polling* it is not, because Gitea's dispatch
endpoint returns no run handle, so the job would have to guess which run is its own and hold a
runner idle meanwhile. The warning exists so a genuinely broken release workflow surfaces once
rather than being silently retriggered every night forever.
### 7.4 Open risk
**Settled as of the v3 cutover.** Protocol work landed on `edge` branches and the `edge → main`
cutover has now merged, so `main` speaks protocol 3 consistently across the repos. The rule it
motivated stands regardless and is not a temporary measure: **the installer hardcodes no protocol
version anywhere.** It reads what the artifacts declare, and §7.1's gate 1 is what stops a
mismatched pair from being published as a bundle — which is the mechanism that will matter at the
*next* protocol bump, not just this one. See `docs/link/v3.md`.
---
## 8. Open questions
1. **Windows service mechanism** — `sc create` against the plain console binary (simplest, works
today), a bundled WinSW/NSSM shim, or a native `--service` mode in the sidecar using the
`windows-service` crate (cleanest, but changes `link`). Recommendation: `sc create` for v1,
revisit if restart semantics prove inadequate.
2. **Does the installer manage ServUO stop/start?** Currently it refuses while ServUO runs and tells
the operator to restart afterward. Offering to stop/start would be friendlier but means owning
another shard's process lifecycle, and the shard's own start scripts vary.
3. **Co-location assumption** — the shard dials out to the sidecar on loopback `127.0.0.1:7788`, so
sidecar and ServUO must share a host. Should the installer support installing only uo-link on a
different host, or hard-assume co-location?
Resolved and moved into §1 / §2.2 / §5: uninstall scope, and minimum ServUO version.
**Resolved — branch targeting for the new repo** (was question 4). The v3 cutover landed:
`servuo-plugins#6` merged, so that repo's `main` and `edge` agree at protocol 3. The release
workflow targets `main`, and the installer repo starts clean on `main`. §7.4's caution still applies
in principle — the installer hardcodes no protocol version, it reads what the artifacts declare —
but the specific `edge`/`main` disagreement that motivated it is gone.
---
## 9. Administrator experience
Before:
```
find plugins → copy files → edit ServUO → download bridge → start bridge
→ configure startup → find the token → troubleshoot paths
```
After:
```
download artifact → verify checksum → run installer → select ServUO directory
→ install components → paste 4 values into Admin → Shard → start ServUO normally
```

314
link/INTEGRATION.md
View File

@@ -23,7 +23,31 @@ Every route **except `GET /health`** requires the shared token from `sidecar.tom
| REST | `X-Api-Key: <token>` |
| WebSocket | `?token=<token>` in the connect URL (browsers can't set headers on a WS handshake) |
Missing or wrong token → **401** `{"error":"missing or invalid auth token"}`. The token is compared in constant time. It is generated automatically on first run (the sidecar logs it); rotate by editing `sidecar.toml` and restarting.
Missing or wrong token → **401** `{"error":"missing or invalid auth token"}`. The token is compared in constant time. It is generated automatically on first run; rotate by editing `sidecar.toml` and restarting.
To read it back afterwards, ask the sidecar rather than hunting through the startup log or the TOML:
```console
$ uo-link-sidecar --print-config --config /etc/runicgateway/sidecar.toml
{
"component": "uo-link-sidecar",
"config_created": false,
"config_path": "/etc/runicgateway/sidecar.toml",
"protocol": 3,
"shard": { "bind": "127.0.0.1:7788" },
"store": { "path": "/var/lib/runicgateway/uo-link.db" },
"token_generated": false,
"version": "0.1.0",
"web": {
"auth_required": true,
"auth_token": "c0f04ace…",
"bind": "127.0.0.1:8080",
"ws_path": "/ws"
}
}
```
That is the same set of values Admin → Shard asks for — base URL and WS URL are `web.bind` (substituting a reachable host if it is `0.0.0.0`) plus `web.ws_path`. The output **contains the token in clear text**, so treat it as a secret: it belongs in a terminal, not in a log or a CI artifact. `--print-config` also performs first-run setup, writing the config file and generating a token if there is none, and reports whether it did via `config_created` / `token_generated`.
---
@@ -31,18 +55,32 @@ Missing or wrong token → **401** `{"error":"missing or invalid auth token"}`.
The wire protocol is versioned so a mismatch is caught immediately instead of failing weirdly.
- Every response carries an **`X-UOLink-Version: 2`** header.
- `GET /health` and the WebSocket `ws.hello` frame include `"protocol": 2`.
- **Optionally**, send `X-UOLink-Version: 2` on your requests. If it disagrees with the sidecar, the request is rejected **409 Conflict**:
- Every response carries an **`X-UOLink-Version: 3`** header.
- `GET /health` and the WebSocket `ws.hello` frame include `"protocol": 3`.
- **Optionally**, send `X-UOLink-Version: 3` on your requests. If it disagrees with the sidecar, the request is rejected **409 Conflict**:
```json
{ "error": "protocol version mismatch", "sidecar_protocol": 2, "client_protocol": "1" }
{ "error": "protocol version mismatch", "sidecar_protocol": 3, "client_protocol": "2" }
```
Pin the version you built against and compare it to the header (or `/health.protocol`) at startup.
**v2 (Protocol 2.0)** added the account-provisioning surface (§6.x: `POST /accounts/create`, `DELETE /link/{account}`) and the `account.*` events. Outbound event kinds are **additive** — a v1 client that ignores unknown kinds keeps working against the live feed — but the new *endpoints* require a v2 sidecar. If you send `X-UOLink-Version: 1`, calls to the new endpoints are refused with the 409 above.
**v3 (Protocol 3.0)** adds `world.ruleset`, `points.board` and `vendor.listing` /
`vendor.listing.remove`, with the `GET /ruleset`, `/points` and `/market` reads that serve them from
the sidecar's store. Same shape as the v2 bump: the event kinds are additive, so a v2 client that
ignores unknown kinds keeps working against the live feed, but the three new endpoints require a v3
sidecar. There is deliberately **no feature-negotiation array** — v3 implies all three kinds, so the
version number alone tells you what is available.
**Upgrading a v2 integration.** The bump is an operator-visible hard break in one direction only: a
client still declaring `2` gets a 409 on every protected route and, on the WebSocket, a closed
connection on the `ws.hello` mismatch. So update the pinned version at the same time you deploy the
v3 sidecar. Nothing that existed in v2 changed shape, so that is the whole migration — the website
does it with a one-shot boot migration of its `uo_link_config.protocol` row ([`v3.md`](v3.md) §4.1);
a third-party client changes the constant it sends.
---
## 3. Health
@@ -54,7 +92,7 @@ GET /health (no auth)
```json
{
"status": "ok", // "ok" when plugin connected AND db reachable, else "degraded"
"protocol": 1,
"protocol": 3,
"plugin_connected": true, // is the shard link up right now?
"database": "ok", // "ok" | "error"
"uptime": "3d 12h",
@@ -77,7 +115,7 @@ A push-only stream of game events as they happen. You do **not** send commands o
**On connect**, the first frame is:
```json
{ "kind": "ws.hello", "protocol": 1 }
{ "kind": "ws.hello", "protocol": 3 }
```
**Then** a continuous stream of event frames, each with at least `t` (epoch ms) and `kind`. Route on `kind`.
@@ -315,6 +353,198 @@ The house registry — one row per house, complementing the `house.decay` *trans
Render from `GET /houses` (§6) on connect, then keep live with these events.
#### Shard ruleset (Protocol 3.0)
How the shard is actually configured, published by the shard itself. **Not a sweep** — it changes only
when an operator edits `Config/*.cfg`, so it is emitted once per shard↔sidecar connect (and on
`[bridge reload`), exactly like `server.hello`.
| kind | fields | notes |
|------|--------|-------|
| `world.ruleset` | `rev`, `shard`, `expansion`, `connect?`, `systems`, `caps`, `housing`, `accounts`, `vetRewards`, `loot`, `vendors`, `champions?`, `treasureMaps`, `vvv?`, `store`, `schedule?` | The whole ruleset, always complete — **never a delta**, so the latest frame replaces the previous one outright. Every block except `shard`/`expansion` is optional and is **omitted when its system is off**, so absence means "not applicable here", not "unknown". |
`rev` is the shard's FNV-1a of the body: identical `rev` means the ruleset is unchanged and this frame
is just a reconnect re-send, so a consumer can skip the write. It is deliberately **not**
`String.GetHashCode()`, which is seeded per process and would change on every shard restart.
```json
{"kind":"world.ruleset","rev":"1a2b3c4d","shard":"UOMysticmoon","expansion":"EJ",
"systems":{"cityLoyalty":true,"vvv":true,"factions":false,"siege":false,"chat":true,
"store":true,"dailyRares":true,"honesty":true,"shadowguard":true,
"treasureMaps":true,"vetRewards":true,"testCenter":false},
"caps":{"skill":1000,"totalSkill":7000,"stat":225,"str":125,"dex":125,"int":125,
"strMax":150,"dexMax":150,"intMax":150},
"housing":{"accountHouseLimit":1},
"accounts":{"perIp":3,"charSlots":7,"autoCreate":true},
"vetRewards":{"enabled":true,"rewardIntervalDays":30},
"loot":{"feluccaLuckBonus":1000,"feluccaBudgetBonus":100,"feluccaMaxProps":11},
"vendors":{"restockDelayMinutes":60,"maxSell":500,"economyStockAmount":500},
"champions":{"powerScrolls":6,"statScrolls":16,"scrollChance":0.1,
"transcendenceChance":50.0,"rankThresholds":[5,10,13]},
"treasureMaps":{"enabled":true,"lootChance":0.01,"resetDays":30},
"vvv":{"enabled":true,"startSilver":2000,"enhancedRules":false},
"store":{"enabled":true,"currencyName":"Sovereigns"},
"schedule":{"autoSaveEnabled":true,"autoSaveFrequencyMinutes":15,"autoRestartEnabled":false},
"t":1752489280000}
```
**Two things consumers get wrong.**
1. **`caps.skill` and `caps.totalSkill` are in tenths**, the way ServUO stores them: `1000` is `100.0`
skill and `7000` is `700.0` total. Rendering the raw number is actively misleading. The other caps
(`stat`, `str`, …) are plain integers.
2. **`connect` is present only if the operator set `Bridge.PublicConnectAddress`.** The shard's real
listen address (`Server.cfg`) is never published; nor are `Staff.cfg`, `Email.cfg`, `DataPath.cfg`,
`Bridge.cfg`, `Compiler.cfg`, `Reports.cfg` or `Client.cfg`. The frame is built from an explicit
allowlist in `BridgeRuleset.cs` — `Config.Entries` is never enumerated, because that would sweep in
every key on the server.
Absent entirely if the shard runs `Bridge.RulesetEnabled=false` or an older plugin. Render from
`GET /ruleset` (§6) on connect, then keep live with this event.
This **supersedes the `world.systems` frame** sketched in [`PROTOCOL_2.md`](PROTOCOL_2.md) §10.4 and
never implemented; the `systems` block above is what that asked for.
#### Points / loyalty leaderboards (Protocol 3.0)
ServUO carries ~25 separate point currencies — Queen's Loyalty, Void Pool, Casino, Clean Up Britannia,
the nine city loyalties, Blackthorn, the Doom / Khaldun / Kotl treasure systems — every one a standing
players accumulate over months, and none of them visible outside an in-game gump before 3.0.
A diff sweep (default 300 s), **one frame per system** rather than one large frame for all of them,
matching `champ.update` / `guild.update`. A system is emitted only when its top N or its participant
count actually changes.
| kind | fields | notes |
|------|--------|-------|
| `points.board` | `system`, `nameString`, `nameNumber`, `maxPoints`, `showOnGump`, `players`, `top[]` | One system's complete board — **never a delta**. The latest frame for a `system` replaces the previous one outright. `top[]` entries are `{rank, serial, name, points}`. |
`system` is the shard's own `PointsType` enum name (`QueensLoyalty`, `CleanUpBritannia`, …) and is the
board's stable key. There is deliberately **no `points.remove`**: the set of systems is fixed at startup
by `PointsSystem.Configure`, so a system cannot disappear at runtime — the same argument `city.update`
makes for cities.
```json
{"kind":"points.board","system":"QueensLoyalty",
"nameString":"Queen's Loyalty","nameNumber":1114938,
"maxPoints":15000,"showOnGump":true,"players":842,
"top":[{"rank":1,"serial":"0x1A2B","name":"Darrow","points":29500},
{"rank":2,"serial":"0x1A2C","name":"Mireille","points":21000}],
"t":1752489280000}
```
**Four things consumers get wrong.**
1. **`maxPoints` of `0` means UNCAPPED, not "zero points allowed".** ServUO's idiom for an uncapped
system is `double.MaxValue` (`DespiseCrystals`, `ShameCrystals` and `VoidPool` all use it), which
the plugin normalises to `0` rather than emitting a nonsense integer. On a real shard **most
systems are uncapped**, so a UI that renders `points / maxPoints` must special-case this or it will
divide by zero on the common path.
2. **`nameString` is usually `null`.** The shard's `Name` is a `TextDefinition`, which may carry a
literal *or* a cliloc id, and in practice most systems use the cliloc — so `nameNumber` is set and
`nameString` is `null`. Resolve clilocs consumer-side; failing that, humanising the `system` key
("CleanUpBritannia" → "Clean Up Britannia") reads better than showing a bare number. This is the
same contract `titles.reward` already documents.
3. **`players` counts players who actually hold points**, not the size of the system's table. Ten of
the ~25 systems have `AutoAdd = true` and therefore keep a zero-point row for every character that
has ever logged in, so the raw table size would report the shard's entire character census as that
system's participants.
4. **Entries carry `serial` and `name` only — never `acct` or `webId`.** A board is the widest-audience
surface the bridge has, so the account name of every ranked player deliberately does not cross the
wire; resolve serial → site user from your own link mirror if you need it.
Absent entirely if the shard runs `Bridge.PointsLeaderboardEnabled=false` or an older plugin. Render
from `GET /points` (§6) on connect, then keep live with this event.
##### `char.profile` gains a `points` block
Read-model enrichment on the existing kind — there is **no** request kind for one character's points,
the same precedent `titles` set in [`PROTOCOL_2.md`](PROTOCOL_2.md) §10.3:
```json
"points":[{"system":"QueensLoyalty","nameString":"Queen's Loyalty","nameNumber":1114938,
"points":29500,"maxPoints":15000}]
```
Systems where the character has no entry, or an entry at zero, are **omitted** — otherwise every sheet
would carry ~25 zeroes. `maxPoints` follows the same `0 == uncapped` rule as the board.
`rank` is **absent by default** and appears only when the shard runs `Bridge.PointsProfileRank=true`:
a points lookup stops at the character's own row, but a rank must count every row that beats them, in
every system, on every profile build. Derive rank from `points.board` instead for anyone in the top N.
#### Player-vendor marketplace (Protocol 3.0)
The shard-wide shop index: every player vendor's shop name, owner, location and priced inventory —
the same set the in-game **Vendor Search** gump reads, published so a site can offer the same search
from outside the game.
An **amortized round-robin diff sweep**, not a snapshot RPC, and the distinction is load-bearing:
`rpc.rs::try_route` correlates a reply on the FIRST frame carrying a matching `reqId`, so a chunked
reply sharing one `reqId` would deliver chunk 1 to the HTTP caller and leak chunks 2..N onto the
broadcast feed. A whole-world snapshot could not fit in one frame inside the 10 s reply timeout
either. The per-account `vendor.snapshot` RPC (§5) is unaffected and still serves the player portal.
Each tick inventories at most `Bridge.MarketSweepBatch` vendors (default 25) starting from a
persistent cursor, so **per-tick cost is bounded independently of world size**; full coverage takes
`ceil(vendors / batch) × MarketSweepSeconds`. A vendor is emitted only when its contents, prices,
shop name or location actually change.
| kind | fields | notes |
|------|--------|-------|
| `vendor.listing` | `serial`, `shopName`, `ownerSerial`, `ownerName`, `location{}`, `count`, `total`, `truncated`, `items[]` | One vendor's complete shop — **never a delta**. The latest frame for a `serial` replaces the previous one outright. |
| `vendor.listing.remove` | `serial` | The shop is gone from the index: dismissed, expired, or its owner switched off the in-game Vendor Search flag. |
```json
{"kind":"vendor.listing","serial":"0x40001234",
"shopName":"Darrow's Bargains","ownerSerial":"0x1A2B","ownerName":"Darrow",
"location":{"map":"Trammel","x":1421,"y":1699,"z":0,
"region":"Britain","house":"Darrow's Villa"},
"count":2,"total":2,"truncated":false,
"items":[{"serial":"0x40012ABC","itemId":3922,"hue":0,"amount":1,
"price":25000,"name":null,"cliloc":1023721},
{"serial":"0x40012ABD","itemId":7026,"hue":1157,"amount":3,
"price":500,"name":"a shard sigil","cliloc":1041243}],
"t":1752489280000}
```
**Six things consumers get wrong.**
1. **`name` is `null` for nearly every item; `cliloc` is the real label.** Items carry a
`LabelNumber`, not a name. The plugin deliberately never calls `VendorSearch.GetItemName`, which
builds an `ObjectPropertyList`, serialises it and byte-parses the packet **per item** — a
multi-hundred-millisecond stall across a full pass. (It would not work anyway: every current
client ships its cliloc files compressed and ServUO's bundled `Ultima.StringList` cannot read
them, so the in-game gump has the same gap.) Resolve clilocs consumer-side; a non-null `name` is a
player-set literal and is strictly more specific, so **prefer it over the cliloc**.
2. **`location` is one nested object, and it may be absent entirely.** It is nested so that a
consumer gating vendor whereabouts gates one field rather than five that can drift apart — the
website's `market.location` rule removes the whole object. Treat a missing `location` as "not
published", not as an error.
3. **`truncated` means the shop holds more than the frame carries.** `count` is what was published,
`total` is what the shop actually holds, capped by `Bridge.MarketMaxListings` (default 250). A
commodity reseller with thousands of stacked resources is real and an uncapped frame for one is
measured in megabytes. Say "showing 250 of 3,104" rather than presenting a partial shop as
complete.
4. **`child: true` means the price buys the ENCLOSING CONTAINER.** ServUO prices a container as a
unit and everything inside inherits that price with no `VendorItem` of its own; `DoSearch`
surfaces the same flag. A UI that prints the container's price against each item inside it is
lying about the shard.
5. **Opted-out vendors are absent, and that is a privacy control.** `pv.VendorSearch` is the player's
own in-game toggle and the sweep honours it — hide your vendor in game and it is hidden here too.
The same goes for `Map.Internal` and a null backpack, matching `DoSearch`. Process
`vendor.listing.remove` promptly: it is how a player *revoking* that consent reaches you.
6. **Prices are inherently stale, by design.** The round-robin sweep means a shop can be a full cycle
behind. Any UI over this must say how old the data may be — the website derives it from the oldest
vendor row.
Entries carry `ownerSerial`/`ownerName` and **never `acct` or `webId`**, the same rule `points.board`
follows. Absent entirely if the shard runs `Bridge.MarketEnabled=false` or an older plugin. Render
from `GET /market` (§6) on connect, then keep live with these events — though note that a live
firehose of whole vendor inventories is the largest stream the bridge produces, and a consumer that
only needs a browsable index (as the website does) is better served by the REST read plus the
periodic re-sweep.
---
## 5. REST — read queries
@@ -354,7 +584,7 @@ Full character sheet: stats, all trained skills, worn equipment with flattened i
Field notes:
- `skills[].base` is trained value, `value` includes item/temp bonuses, `cap` is the cap. **Do not assume `base <= cap`** — GM characters can exceed it.
- `equipment[].mods` is a flattened map of every non-zero AOS attribute on the item (weapon or armor). Empty `{}` for plain items.
- Item names are usually **clilocs**, not strings: use `name` when present, otherwise resolve `cliloc` against a UO cliloc table on the site.
- Item names are usually **clilocs**, not strings: use `name` when present, otherwise resolve `cliloc` against a UO cliloc table on the site. **Do not expect the shard to resolve them for you** — on any modern client ServUO's own `Ultima.StringList` cannot read the client's compressed cliloc files, so `VendorSearch.GetItemName` returns `item.Name` and the in-game Vendor Search gump has the same gap. Building that table is a consumer-side job; the website's is described in [`website/CLILOCS.md`](../website/CLILOCS.md).
- `titles` (Protocol 2.0): `selected` is the index into `reward` currently displayed (`-1` if none). `fameKarma`/`skill` are computed display titles, omitted when the character has none. `reward` entries may be a **cliloc number as a string** or a literal string — resolve numeric ones against your cliloc table, same as item names.
- Errors: unknown account → **404** `{"kind":"bridge.error","reason":"unknown account"}`; bad slot → **404**/**400** similarly.
@@ -658,6 +888,72 @@ GET /houses
Every house's latest snapshot — owner→houses map. Served from the sidecar's projection, kept current by the `house.*` stream (§4). Ordered by name. Survives a sidecar restart.
### Shard ruleset (Protocol 3.0)
```
GET /ruleset
→ { "ruleset": {"kind":"world.ruleset","rev":"1a2b3c4d","shard":"UOMysticmoon",
"expansion":"EJ","systems":{...},"caps":{...},"accounts":{...}, ... } }
```
The shard's published ruleset (§4 for the full frame and its two gotchas). Served from the sidecar's
store, so it **answers while the shard is down** — a rules page that goes blank during a restart is
worse than one that is briefly stale. Keep it current with the `world.ruleset` stream.
`{"ruleset": null}` means the shard has never published one — an older plugin, or
`Bridge.RulesetEnabled=false`. That is a real answer distinct from a published ruleset, and worth
rendering differently ("not published yet") rather than as an empty ruleset.
### Points / loyalty leaderboards (Protocol 3.0)
```
GET /points
→ { "boards": [ {"kind":"points.board","system":"QueensLoyalty","nameString":"Queen's Loyalty",
"nameNumber":1114938,"maxPoints":15000,"showOnGump":true,"players":842,
"top":[{"rank":1,"serial":"0x1A2B","name":"Darrow","points":29500}, ...],"t":...}, ... ] }
GET /points/{system} # e.g. /points/QueensLoyalty
→ {"kind":"points.board","system":"QueensLoyalty", ... }
```
Every system's latest board, or one by its `PointsType` name (§4 for the frame and its four gotchas).
Served from the sidecar's projection, kept current by the `points.board` stream, ordered by display
name. Survives a sidecar restart — which matters more here than for live state, since these are
standings built over months and blanking them during a restart reads as data loss.
`GET /points/{system}` returns **404** for a system the shard has never published (an unknown name, or
one excluded by `Bridge.PointsSystems`). That is distinct from a published board nobody has scored in
yet, which is **200** with an empty `top[]` — and the two are worth rendering differently.
### Player-vendor marketplace (Protocol 3.0)
```
GET /market?limit=200&offset=0
→ { "vendors": [ {"kind":"vendor.listing","serial":"0x40001234",
"shopName":"Darrow's Bargains","ownerSerial":"0x1A2B","ownerName":"Darrow",
"location":{"map":"Trammel","x":1421,"y":1699,"z":0,
"region":"Britain","house":"Darrow's Villa"},
"count":2,"total":2,"truncated":false,"items":[ ... ],"t":...}, ... ],
"total": 137, "limit": 200, "offset": 0 }
```
Every vendor's latest shop, exactly as `vendor.listing` published it (§4 for the frame and its six
gotchas). Served from the sidecar's projection, so it answers while the shard is down.
**This is the only PAGED read the sidecar serves**, because it is the only board that can be a whole
world's inventory. `limit` is clamped to 1..1000 (default 200); `total` is returned so a caller knows
when to stop rather than paging until it sees a short page, which would race a concurrent sweep.
Ordering is by **serial**, not by shop name — a serial is stable while a shop name is renameable, so
a rename mid-walk cannot make a vendor skip or repeat a page.
The route is `/market` and deliberately **not** `/vendors`: `/vendors/{account}` next door is the
per-account RPC (§5), and two routes a prefix apart meaning "this player's shops" and "every shop on
the shard" is a trap nobody wins.
Frames are served **verbatim**, owner names and coordinates included. That is not an oversight: the
sidecar defines no audiences. Deciding who may see what is the consuming site's job — see
[`v3.md`](v3.md) §3 for how the website does it.
---
## 7. Status codes
@@ -683,7 +979,7 @@ Every house's latest snapshot — owner→houses map. Served from the sidecar's
A typical character page:
```js
const H = { "Authorization": `Bearer ${TOKEN}`, "X-UOLink-Version": "2" };
const H = { "Authorization": `Bearer ${TOKEN}`, "X-UOLink-Version": "3" };
// 1. render the roster
const roster = await fetch(`${BASE}/roster/${account}`, { headers: H }).then(r => r.json());

45
link/PLAN.md
View File

@@ -284,6 +284,15 @@ Counts in `hello` are a live snapshot taken on the Core thread, not a cached val
`Item.Name` is frequently `null`; the display name is `LabelNumber`, a cliloc id. **There is no `Data/Cliloc.enu` in this repo** — `BRIDGE_FINDINGS.md` §IV.4 is wrong about this. Cliloc data lives in the client install, which `DataPath` resolves to `D:\Games\Electronic Arts\Ultima Online Classic\`. Ship **both** `name` (when non-null) and `cliloc`, and resolve the number **on the website** against a cliloc map. That avoids a server-side dependency on the client directory.
**Update (3.0).** That recommendation held, and the reason it had to hold turned out to be stronger
than "avoids a dependency": **ServUO cannot resolve clilocs either.** Every current client ships its
`Cliloc.*` files compressed, and the bundled `Ultima.StringList` reads only the older plain layout —
so `VendorSearch.StringList` is null and `VendorSearch.GetItemName` returns `item.Name` on any modern
shard. The in-game Vendor Search gump has the same gap, which is why `vendor.listing` never calls it.
Pushing name resolution to the plugin was never an option. See [`v3.md`](v3.md) §8.6 and
`docs/website/CLILOCS.md` for how the site gets a table instead (the operator converts one from their
own client, once).
---
## 8. Corrections to `BRIDGE_FINDINGS.md`
@@ -315,6 +324,35 @@ Counts in `hello` are a live snapshot taken on the Core thread, not a cached val
7. **Core edit: `PlayerVendorSale`** (§6). Then the cheat-detection feed.
8. **Cheat signals.** `FastWalk`, `OnPropertyChanged` audit, vendor-sale anomaly detection in the sidecar.
**Beyond 1.0.** Phases above are the 1.0 read/event plane. Protocol 2.0's phasing (provisioning +
world-state boards) is [`PROTOCOL_2.md`](PROTOCOL_2.md) §13; Protocol 3.0's (visibility framework,
shard content and standings) is [`v3.md`](v3.md) §9, which also tracks what has landed. Shipped from
3.0 so far: **Part A** — the visibility framework — **`world.ruleset`** ([`v3.md`](v3.md) §5),
`BridgeRuleset.cs`, the first bridge stream that is neither an event subscription nor a sweep (it is
emitted once per connect, like `server.hello`, because shard config changes only when an operator
edits a file) — the **spawn atlas** ([`v3.md`](v3.md) §6), which is website-only and touches no wire
at all — and **`points.board`** ([`v3.md`](v3.md) §7), `BridgePoints.cs`, the loyalty/points
leaderboards. `BridgePoints` is the widest read the bridge performs: ten of ServUO's ~25 point systems
keep a row for every character ever created, so it selects the top N in a single bounded pass rather
than sorting, and runs on a deliberately slow 300 s interval.
Also shipped: **`vendor.listing`** ([`v3.md`](v3.md) §8), `BridgeMarket.cs`, the shard-wide
player-vendor index. It introduces the one sweep pattern the bridge did not previously have — an
**amortized round-robin**. Every other sweep walks its whole collection per tick, which is fine for
tens of houses or a fixed set of point systems and is not fine for a world of shops whose inventories
recurse into containers. `BridgeMarket` inventories at most `MarketSweepBatch` vendors per tick from
a persistent cursor, so the per-tick cost is bounded by the batch rather than by world size, and full
coverage takes `ceil(vendors / batch) x MarketSweepSeconds`. Measured at **15.4 ms** for a cold tick
of 25 vendors x 40 listings and **0.3 ms** in steady state (the per-vendor diff), on a shard of 209k
items / 43k mobiles. It is also the first stream to honour a per-player privacy toggle: ServUO's own
`PlayerVendor.VendorSearch` flag, so a shop hidden in game is hidden on the site.
That completes 3.0's feature work, so the last step is the version itself: `PROTOCOL_VERSION` **2 →
3** and the coordinated `edge` → `main` merge across all four repos ([`v3.md`](v3.md) §4 and §4.1).
The bump is deliberately the *only* thing that happens at that moment — v3 adds kinds and endpoints
but changes nothing that already existed in v2 — so the operator-visible break is limited to
re-pinning the version, which the website does for itself in a one-shot boot migration.
### Config keys (`Config/Bridge.cfg`)
```ini
@@ -328,6 +366,13 @@ EconomySweepSeconds=300
Read in `Configure()` via `Config.Get<T>("Bridge.<Key>", default)`. Key scope is the filename: `Bridge.cfg` + `StatSweepSeconds` → `Bridge.StatSweepSeconds`.
The set above is the 1.0 sample, not the current one — every later phase added keys (sweep intervals
for each board, the town-crier/news caps, the admin write plane, account provisioning, and 3.0's
`RulesetEnabled` / `PublicConnectAddress` / `RulesetIncludeSchedule`, and the `Points*` and `Market*`
blocks).
**`servuo-plugins/overlay/Config/Bridge.cfg`
is the authoritative, commented list**; `BridgeConfig.cs` holds the defaults.
---
## 11. Phase 1 acceptance

2
link/PROJECT_TREE.md
View File

@@ -18,12 +18,14 @@ link/
│ ├── scripts/
│ │ └── gen_tree.py
│ ├── workflows/
│ │ ├── pr-checks.yml
│ │ ├── release.yml
│ │ ├── sonarqube.yml
│ │ └── sync-project-tree.yml
│ └── PULL_REQUEST_TEMPLATE.md
├── sidecar/
│ ├── src/
│ │ ├── cli.rs
│ │ ├── config.rs
│ │ ├── main.rs
│ │ ├── rpc.rs

17
link/PROTOCOL_2.md
View File

@@ -285,6 +285,18 @@ City titles and faction/VvV merchant titles (`CityLoyaltySystem.ApplyCityTitle`,
### 10.4 Factions / Vice vs Virtue
> **Status update (Protocol 3.0, 2026-07-28).**
>
> - **The deferred question is answered.** This shard runs **Vice vs Virtue** (`VvV.cfg Enabled=True`);
> old Factions is off, and in stock ServUO that is not a coincidence —
> `Services/Factions/Core/Faction.cs` sets `Settings.Enabled = !ViceVsVirtueSystem.Enabled`, so the
> two are mutually exclusive by construction. The `vvv.standings` / `vvv.battle` streams below are
> therefore unblocked, but are **not** scoped for 3.0 (see [`v3.md`](v3.md) §2 row 5).
> - **`world.systems` is superseded by `world.ruleset`** ([`v3.md`](v3.md) §5), which shipped in 3.0.
> It was never implemented under this name. `world.ruleset` carries the same
> `systems{cityLoyalty, vvv, factions, …}` sub-object this section asked for, plus the rest of the
> shard's published ruleset, so no orphan kind is left behind. Do not implement `world.systems`.
**Which system is live is a shard decision — verify before building.** Two exist:
- **Old Factions** (`Scripts/Services/Factions`): `Faction.Commander` (leader, `Faction.cs:160`), `Faction.Election`, `Faction.Members` (`List<PlayerState>`), and faction-controlled **Towns** (`Town.cs` — each town has an owning faction, a sheriff, and finance). Config-gated and, on most modern shards, **off**.
@@ -300,7 +312,10 @@ City titles and faction/VvV merchant titles (`CityLoyaltySystem.ApplyCityTitle`,
{"kind":"vvv.standings","order":142000,"chaos":138500,"leaderSide":"Order"}
```
> Start by detecting which system is enabled at boot and streaming only that one; emit a one-time `world.systems` frame (what's on: cityLoyalty, vvv, factions) so the website renders the right panels instead of guessing.
> Start by detecting which system is enabled at boot and streaming only that one. ~~emit a one-time
> `world.systems` frame (what's on: cityLoyalty, vvv, factions) so the website renders the right panels
> instead of guessing.~~ — **superseded: `world.ruleset` already carries that `systems` block** (see the
> status note at the top of this section).
## 11. Further integration points — a menu to pick from

1037
link/v3.md Normal file
View File

File diff suppressed because it is too large Load Diff

335
website/BACKEND_DESIGN.md
View File

@@ -99,8 +99,12 @@ server/
pages.router.js (2) /public/pages — the draft-preview
route precedes /:slug and is
deliberately not site-mode gated
shard.router.js (12) /public/shard/* incl. the anonymous
shard.router.js (14) /public/shard/* incl. the anonymous
SSE stream; never site-mode gated
atlas.router.js (6) /public/atlas/* — the spawn atlas.
NOT under /shard: nothing here
touches the sidecar, and unlike
/shard it IS site-mode gated
site.router.js (4) /settings /status /version /contact —
the group-root singletons; declares no
router-level middleware
@@ -358,6 +362,232 @@ analogue to a password — and there is no hash-lookup constraint (verification
unused rows and `bcrypt.compare`s each, like password verification). `used_at` is the single-use
marker. Cleared wholesale on TOTP disable / password change / password reset.
### shard_ruleset — the shard's published ruleset (Protocol 3.0)
Singleton row (`id = 1`, CHECK-constrained) holding the latest `world.ruleset` frame: `rev`,
`expansion`, `payload` JSON (the whole frame), `t`, `updated_at`. The shard re-emits the complete
ruleset on every sidecar connect, so this is an **overwrite, not an append** — and the kind is
deliberately **not** in `LOGGED_KINDS`, since logging it would put a duplicate row in `shard_events`
on every reconnect while `server.hello` already marks each of those.
The frame is stored whole rather than normalized into columns: it is a flat description of server
config that is read as one page, so splitting it up would mean a schema change every time the shard
grows a new block. `rev` (the shard's FNV-1a of the body) and `expansion` are hoisted only because
they are cheap to display — the same payload-plus-hoisted-columns shape `shard_champs` uses.
**No row means the shard has never published one** (an older plugin, or `Bridge.RulesetEnabled=false`),
served as `null` rather than `{}`: "not published yet" and "published, everything off" are different
answers and the page renders them differently.
### shard_points_boards — points / loyalty leaderboards (Protocol 3.0)
One row per point system, keyed by the shard's own `PointsType` name (`QueensLoyalty`,
`CleanUpBritannia`, …). The shard carries ~25 of these, each a standing players build over months.
Columns: `system` (PK), `name`, `name_cliloc`, `max_points`, `players`, `show_on_gump`, `payload` JSON
(the whole `points.board` frame), `t`, `updated_at`.
**The top-N list stays inside `payload`** rather than being normalized into a `shard_points_entries`
table. It is a fixed-size list (10 by default) that is only ever read whole — exactly like
`shard_governors.candidates` — so normalizing buys nothing until something needs a per-character
reverse lookup, and a character's own standings already ride inside `char.profile` instead.
Board state, not events: `points.board` is **not** in `LOGGED_KINDS`, for the same reason
`guild.update` isn't. The shard emits a frame every time anyone's score moves a top ten, so logging
would grow `shard_events` without bound for something whose only interesting value is its latest
version. There is also **no delete path** — the shard's set of systems is fixed at startup, so there is
no `points.remove` to mirror.
Two values carry non-obvious meanings, both set by the plugin and both documented in
[`link/INTEGRATION.md`](../link/INTEGRATION.md) §4:
- **`max_points = 0` means uncapped**, and on a real shard that is the *common* case (ServUO's
uncapped idiom is `double.MaxValue`, which the plugin normalises to 0). Anything rendering
`points / max_points` must special-case it.
- **`name` is usually NULL**, with `name_cliloc` set instead — most systems name themselves with a
cliloc rather than a literal. Listing therefore orders by `COALESCE(name, system)`, so boards
awaiting cliloc resolution sort by their own key rather than clumping together under NULL.
### shard_vendors / shard_vendor_items — the player-vendor marketplace (Protocol 3.0)
The shard-wide shop index, fed by `vendor.listing` / `vendor.listing.remove`. One row per player
vendor and one per priced listing. Full operator detail in [`MARKETPLACE.md`](MARKETPLACE.md); the
design is `docs/link/v3.md` §8.
| Table | Shape |
|---|---|
| `shard_vendors` | `serial` (PK), `shop_name`, `owner_serial`, `owner_name`, `map`/`x`/`y`/`z`, `region`, `house`, `item_count`, `item_total`, `truncated`, `t`, `updated_at`. Indexes on owner, map, region and `updated_at`. |
| `shard_vendor_items` | `id` (PK), `vendor_serial`, `serial`, `item_id`, `hue`, `amount`, `price`, `name`, `cliloc`, `display_name`, `child`. Indexes on `vendor_serial`, `price`, `item_id`, `display_name`, and `(display_name, price)`. |
**Ingest is per-vendor and authoritative**: the frame is the whole shop, so ingest is
delete-then-insert of that vendor's listings inside one transaction. All-or-nothing matters
specifically because the two writes are "the shop" and "what is in it" — a failure between them
leaves a shop advertising an inventory it no longer has, which is visibly wrong and indistinguishable
from a genuinely empty shop. No foreign keys, consistent with every other `shard_*` table.
**There is deliberately no `payload` column**, unlike `shard_points_boards` directly above. The
board's top-N is a fixed-size list read whole, so it lives in JSON; here the items *are* the
searchable rows, so they are normalized and nothing is left worth duplicating. The sidecar keeps the
whole blob — outage resilience is its job, search is ours.
Market state, not events: neither kind is in `LOGGED_KINDS`, and this is the strongest case of the
three v3 kinds. One frame carries up to 250 listings and the sweep re-emits a shop on any price
change, so logging would turn `shard_events` into a price history nobody reads.
Two columns carry non-obvious meanings:
- **`item_count` vs `item_total`.** `item_count` is what the frame published; `item_total` is what
the shop actually holds. They differ when `truncated` — the shard caps listings per frame
(`Bridge.MarketMaxListings`, 250 by default), and a commodity reseller with thousands of stacks
genuinely exceeds it. Any UI must show both or it presents a partial shop as complete.
- **`display_name` is denormalized at ingest**, resolved from the item's literal `name` (preferred —
a player set it, so it is more specific) else its `cliloc` against `shard_clilocs`. Resolving at
query time would put the cliloc table on the hot path and make search-by-name impossible. Because
the shard's diff sweep will not re-send an unchanged shop just because the site learned what its
items are called, **a cliloc import triggers a bulk re-resolution** of this column (after a boot
import and after an admin import; ~50 ms per thousand rows, never throws).
`updated_at` is written explicitly on every upsert rather than left to `ON UPDATE CURRENT_TIMESTAMP`,
which MariaDB does not fire when every column is written back unchanged. A shop re-published
identically is still *freshly confirmed*, and without this the staleness banner would age a perfectly
current shop forever.
### shard_feature_visibility — per-feature audience config (Protocol 3.0)
One row per shard feature: `feature` (PK), `enabled`, `audience` (a rung on the ladder in §6.5),
`stream` (whether the feature's kinds fan out over SSE at all), `field_rules` JSON (`{field: rung}`
for the sensitive fields only), `updated_by`, `updated_at`.
**An absent row means "use the compiled default", and the compiled defaults reproduce pre-3.0
behavior — so an empty table is a no-op and there is nothing to seed.** Stored rows are merged over
the defaults on read, which is also where the invariants are re-applied: a row naming an unknown
feature is ignored (a stale row must not resurrect a removed feature), an invalid rung falls back to
the default rather than failing open, and a rule touching a locked field (`acct` / `webId`) is
discarded. See §6.5.
### shard_spawn_* / shard_regions / shard_landmarks / shard_champion_spawns / shard_atlas_meta — the spawn atlas (Protocol 3.0)
Static shard **content**, not live shard state. Nothing here comes from the sidecar: the atlas is
derived from the shard's own ServUO tree, re-read on **every server boot** and hash-gated so an
unchanged tree costs one read pass and no write. Nothing is precomputed and committed — a shard's
maps change over its life, and a snapshot in the repo would silently drift from the world players
actually see. These tables stay populated whether the shard is up or not. Full operator detail in
[`SPAWN_ATLAS.md`](SPAWN_ATLAS.md); the design is `docs/link/v3.md` §6.
**No facet name appears anywhere in the code.** A shard may add facets, replace them, or rename them
when its maps are updated; the facet set is discovered from the tree, and the loose spellings in
`Data/Locations` are matched against it rather than looked up in a table.
| Table | Key columns |
|---|---|
| `shard_spawn_creatures` | `slug` PK, `name`, `total`, `points`, `facets` JSON, `art` NULL |
| `shard_spawn_points` | `id` PK, `facet`, `name`, `x`, `y`, `width`, `height`, `spawn_range`, `max_count`, `min_delay`, `max_delay`, `tod_start/end/mode`, `region`, `landmark`, `label` |
| `shard_spawn_point_types` | `(point_id, slug)` PK, `max_count` |
| `shard_regions` | `facet`, `name`, `type`, `priority`, `parent`, `rects` JSON |
| `shard_landmarks` | `facet`, `name`, `grp`, `x`, `y`, `z` |
| `shard_champion_spawns` | `slug` PK, `name`, `grp`, `type`, `random_type`, `facet`, `x`, `y`, `z`, `radius`, `label` |
| `shard_atlas_meta` | Singleton (`id = 1`), `payload` JSON (counts, a sha256 per source file, `parserVersion`), `imported_at` |
| `shard_atlas_pending` | Singleton (`id = 1`), `status` (`pending`/`rejected`), `payload` JSON, `detected_at` |
The first seven are **import-owned**: a refresh empties and reloads every one inside a single
transaction, so a failed reload leaves the previous atlas intact rather than a half-loaded world.
Nothing else writes to them, and nothing holds a foreign key to them — no FKs at all, consistent with
every other `shard_*` table.
**`shard_atlas_pending` is the security-relevant one.** A refresh that would REMOVE a facet is never
applied automatically: facet loss is indistinguishable at boot from a half-copied or mid-update tree,
so it is staged here for an admin to approve or reject, and **startup is never blocked by it**. Only
the decision is stored — source hashes plus the facet diff, a few KB — and approving re-parses the
tree, so a multi-megabyte blob never lands in the database and what gets applied matches the tree at
approval time. A rejection is remembered against those exact hashes so a declined refresh does not
re-prompt on every restart. Everything else (new facets, renamed regions, changed spawns) applies
immediately, since none of it can destroy data an operator would miss.
The boot refresh is **best-effort by contract**: no configured path, an unreadable mount, a malformed
file or a database error is caught and logged, and the site comes up serving whatever atlas it had.
The tree path comes from the `spawn_atlas_servuo_path` setting, falling back to `SERVUO_PATH`.
**A refresh re-derives when the tree changed OR the parser did.** `spawnAtlasSource.PARSER_VERSION`
is stored in `shard_atlas_meta` beside the source hashes and bumped whenever the parser produces
different data from identical files. Hashing the tree alone would strand an install whose maps never
change on whatever an older build derived — a corrected parse would ship and never reach the data.
Four column choices worth stating, because each one is a trap:
- **`spawn_range`, not `range`**, and **`grp`, not `group`** — both are reserved words.
- **`DELETE`, not `TRUNCATE`.** `TRUNCATE` is DDL in MariaDB and implicitly commits, which would
defeat the all-or-nothing reload. At ~7k rows the difference does not matter.
- **Point ids are assigned explicitly**, not left to `AUTO_INCREMENT`: the `shard_spawn_point_types`
rows need to know them, and `conn.batch()` reports no usable `insertId` for a multi-row insert.
- **Plain `INDEX` on `name`, deliberately not `FULLTEXT`.** ~800 creature rows makes a `LIKE` scan
free, and FULLTEXT's minimum token length would break searches for names like "orc".
`shard_champion_spawns` is the *configured* altar roster ("there is an Unholy Terror altar in
Deceit"). The live `champ.update` feed in `shard_champs` is the separate answer to "it is on level 3
right now". Both exist; they are not the same data.
**`shard_spawn_creatures.art` is always NULL on a fresh import.** The project ships no creature
artwork: sprites live in the operator's own client `.mul`/`.uop` files and are theirs, not ours to
redistribute. An operator supplies art via a gitignored map plus images under the (already
gitignored) `server/uploads/atlas/`. Text-only is the normal, supported state.
### shard_clilocs / shard_cliloc_meta — UO's localization table (Protocol 3.0)
Items on the wire carry a `LabelNumber`, not a name. The bridge has always sent it —
`char.profile.equipment.cliloc`, reward titles as a cliloc number in string form, and one per
marketplace listing — but with no table to resolve it against, the character sheet could only render
`id 1023721` where the game renders "quarter staff".
| Table | Shape |
|---|---|
| `shard_clilocs` | `number` INT PK, `flag`, `text` TEXT |
| `shard_cliloc_meta` | Singleton (`id = 1`), `payload` JSON (source file, sha256, count, `parserVersion`), `imported_at` |
Import-owned and all-or-nothing in one transaction, same contract as the atlas — including **`DELETE`,
not `TRUNCATE`**, for the same reason.
**Sourced from files the operator supplies**, at a path from the `cliloc_client_path` setting falling
back to `UO_CLIENT_PATH`. Nothing client-derived is committed: UO's strings are EA's, exactly as the
creature sprites are. A shard with nothing configured is fully supported — names render as ids. Full
design and operator guide: [`CLILOCS.md`](CLILOCS.md).
**It reads a SET of sources, not one file**, because shards edit items and add new ones and those
carry cliloc ids no stock client table has. A base (the converted client table) plus every overlay
under `custom/` are re-read on every boot and hash-gated **together**, exactly as the atlas re-reads
`Regions.xml` + `Locations/*.xml` + `Spawns/*.xml` + `ChampionSpawns.xml`. Later sources win, so an
overlay both adds ids and overrides stock ones, and adding one custom item never means re-exporting a
5 MB client file. Scale, measured on the live shard: its script tree references 16,434 cliloc ids and
only 37 are absent from stock — tens of entries against a 67k base, which is why this is an overlay
and not a second table.
The conversion step is not avoidable: **every current client ships its cliloc files compressed**
(first DWORD's high byte `0x8E`), and ServUO's own bundled `Ultima.StringList` cannot read that
either — so the shard cannot supply names on our behalf. The plain layout and a delimited text export
are both accepted, sniffed by header rather than extension.
Three decisions worth stating:
- **`text` is TEXT, not VARCHAR.** Long property descriptions reach 12 KB. The index that matters for
marketplace search is the denormalized `shard_vendor_items.display_name`, not this table.
- **Blank entries are dropped at import** — 123,490 parsed → **67,496** stored. Roughly half a cliloc
table is empty strings for ids the client reserves and never uses; a row that resolves to no name is
indistinguishable from no row at all, and dropping them makes the binary and text imports converge
on identical content.
- **Two refusals, one of them the atlas's.** A corrupt source fails the parse on a truncated record,
so it is caught outright and leaves the previous table serving. But a source that has **vanished**
parses perfectly and imports a table quietly missing everything it contributed — an unmounted volume
and a deliberate deletion are indistinguishable from here, which is precisely the ambiguity the
atlas stages a facet removal for. So it is escalated: `status: 'needsReview'`, nothing applied,
`missingSources` reported by both the import and `status()`, and an admin accepts it with
`{approve:true}`. That is a flag rather than the atlas's approve/reject pair because the atlas
stores a pending decision so that approving **re-parses** the tree; here nothing is stored, so
re-reading at approval time is automatic.
**Resolution is server-side and there is no public route.** The table is never served *as* a table:
67k rows would dwarf any page using them, and the Android client consumes the same already-resolved
JSON. `resolveMany()` returns only ids that resolved to something displayable — placeholders like
`~1_val~` are stripped, since the bridge sends the id and never the property packet that carries the
arguments — and it never throws, because a cliloc lookup is decoration on a character sheet.
---
## 4. API contract
@@ -372,7 +602,7 @@ are authoritative, and they answer different questions:
| Artifact | Source of truth for | Generated by |
|---|---|---|
| `server/routes.manifest.json` — mirrored as [api-route-inventory.json](./api-route-inventory.json) | **What URLs exist.** 200 public routes + 2 on the internal listener, sorted, method + path only. | `npm run routes:manifest`, by walking the live Express stack |
| `server/routes.manifest.json` — mirrored as [api-route-inventory.json](./api-route-inventory.json) | **What URLs exist.** 215 public routes + 2 on the internal listener, sorted, method + path only. | `npm run routes:manifest`, by walking the live Express stack |
| `server/swagger/swagger-output.json` — served at `/api/docs` | **What each route means.** Parameters, bodies, response codes, security. | `npm run swagger`, from `#swagger.*` annotations |
The split is deliberate: Swagger is annotation-derived, so an unannotated route is invisible in it and
@@ -556,6 +786,19 @@ from the per-route **siteMode** middleware (§5), never from an auth gate.
| GET | `/wiki` | list of pages (slug + title) |
| GET | `/wiki/:slug` | single page |
| POST | `/contact` | (rate-limited) send mail via SMTP; if unconfigured, respond `{fallback:"mailto", email}` |
| GET | `/shard/ruleset` | the shard's own published ruleset (Protocol 3.0 `world.ruleset`): expansion, which optional systems are on, skill/stat caps, account and house limits, champion scroll rules, the save/restart schedule. Served from `shard_ruleset`, so it renders while the shard is down; live via `world.ruleset` on `/shard/stream`. Behind `requireFeature('ruleset')`. **`null`** means the shard has never published one — a real answer, distinct from a published ruleset. `caps.skill` / `caps.totalSkill` are in **tenths** (1000 = 100.0). |
| GET | `/shard/points` | every points/loyalty leaderboard the shard publishes (Protocol 3.0 `points.board`) — Queen's Loyalty, Void Pool, the nine city loyalties, Clean Up Britannia, … Served from `shard_points_boards`, so it renders while the shard is down; live via `points.board` on `/shard/stream`. Behind `requireFeature('leaderboards')`, ordered by display name. **`maxPoints: 0` means uncapped** (the common case), and `nameString` is usually `null` with `nameNumber` holding a cliloc — resolve client-side or humanise the `system` key. |
| GET | `/shard/points/:system` | one board by the shard's `PointsType` name (e.g. `QueensLoyalty`); `:system` must match `/^[A-Za-z][A-Za-z0-9_]{0,47}$/` or **400** before any query runs. **404** = the shard has never published that system, which is distinct from a published board nobody has scored in yet (**200** with an empty `top`). |
| GET | `/shard/market?q=&minPrice=&maxPrice=&itemId=&map=&region=&sort=&limit=&offset=` | search the player-vendor marketplace (Protocol 3.0 `vendor.listing`). Returns **listings**, not vendors — "who sells X and for how much" is the question, and a vendor-shaped result would make every caller flatten the shops back out. Served from `shard_vendors` + `shard_vendor_items`, so it renders while the shard is down. Behind `requireFeature('market')` **and rate-limited** — the first genuinely expensive public read on the site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*` table, reachable with no session). `sort ∈ {price_asc, price_desc, recent}`. `q` matches the resolved display name **or** the item's literal name, with `%`/`_` escaped: they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let `?q=%` match every listing on the shard. Every response repeats `staleAt` (the oldest vendor row) because the shard sweeps round-robin — a banner that ages with the results it labels, not one fetched once. |
| GET | `/shard/market/meta` | index size, staleness (`staleAt`/`freshAt`) and which facets and regions actually hold vendors, so a client builds its filters without running a search it will discard. |
| GET | `/shard/market/vendors/:serial` | one shop and its listings; `:serial` must match `/^0x[0-9A-Fa-f]{1,16}$/` or **400** before any query runs. **404** = a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer, and distinguishing them would leak that a hidden vendor exists. `truncated` (with `total` exceeding `count`) means the shop holds more than the shard publishes per frame. |
| GET | `/shard/features` | the shard features **this caller** may reach plus the audience rung they resolved to (§6.5), so a client hides nav it can't follow. Reports only what the caller can see — the list itself never discloses a gated feature. Consumed by the SPA header and (pending) the Android nav. |
| GET | `/atlas/creatures?q=&facet=&limit=&offset=` | the bestiary, most numerous first, with an unpaginated `total`. Static content parsed from the shard's ServUO tree — **not** sidecar-backed, which is why the atlas sits outside `/shard`, and unlike `/shard/*` it **is** site-mode gated. Behind `requireFeature('atlas')`. `?facet=` is matched exactly and never validated against a list (no facet name exists in the code); the filter is an `EXISTS` over the points rather than a JSON path or `JSON_SEARCH` built from caller input, whose `%`/`_` wildcards would make `?facet=%` match everything. |
| GET | `/atlas/creatures/:slug` | one creature: `places` (the point-in-rect aggregate — "lizardman → Shrines, Isamu-Jima, Yew"), `spawners` (the bounded raw list, with `spawnersTruncated`), `alsoHere`. **`points` is a COUNT and `spawners` is the LIST** — named apart so one key never means a number on one route and an array on another. `minDelay`/`maxDelay` are in **seconds**, normalised at parse time from the source's per-record minutes-or-seconds. 404 = no such creature in this atlas. |
| GET | `/atlas/regions?facet=&q=` | named regions and the rectangles that placed each spawner |
| GET | `/atlas/landmarks?facet=&q=` | points of interest, labelled by `group` ("Covetous", not "Level 1") |
| GET | `/atlas/champions?facet=` | the **configured** altar roster. Not `/shard/champs`, which is the live board. |
| GET | `/atlas/meta` | facets, counts and when the atlas was parsed. Game-world facts only — the ServUO path, source hashes and any pending refresh are operator detail and live on the admin route. |
Public content GETs pass through the **siteMode** gate (§5).
@@ -570,6 +813,10 @@ the whole gate. The ops/config capabilities — `uo-link`, `email`, `discord-bot
linking carries no extra gate and the in-game staff operations carry `modAccess`. There is no residual
file: every admin route is declared in a capability router.
`GET`/`PUT /admin/shard/visibility` are the third tier on that mixed prefix: **`adminOnly`**, because
they decide what *anonymous* visitors can see (§6.5). They sit above `modAccess` deliberately — a
moderator can ban a player but cannot decide what the public internet reads.
`GET /dashboard` and `PUT /site-mode` are the one place where a **single screen spans two tiers**: the
dashboard is staff-wide, but the site-mode toggle on it is `adminOnly`. The client must therefore gate
that control on its own (`Dashboard.jsx` renders it only for `role === 'admin'`) rather than relying on
@@ -595,6 +842,13 @@ file a route sits in — that is the property the route manifest freezes.
| GET | `/users/:id/trusted-devices` | list a user's active trusted devices (never tokens) |
| DELETE | `/users/:id/trusted-devices` · `…/:deviceId` | revoke all / one of a user's trusted devices (logs `admin.trusted_device.revoke[_all]`) |
| POST | `/users/:id/mfa/reset` | recover a locked-out user: disable TOTP + revoke all trusted devices + clear recovery codes (logs `admin.user.totp.reset`) |
| GET | `/shard/atlas` | spawn-atlas status (`adminOnly`): the ServUO path, whether the tree is readable, whether it has drifted from what is loaded, counts, facets, and any refresh staged for review. The public `/atlas/meta` reports the game world only; the filesystem detail is here. |
| POST | `/shard/atlas/import` | re-import without restarting; `{force}` ignores the hash gate. **An unreadable tree answers 200 with `status:"unavailable"`, not 500** — `refresh()` reports outcomes rather than throwing (the boot path must never be blocked by a bad tree) and that contract is preserved at the API. |
| POST | `/shard/atlas/approve` · `/shard/atlas/reject` | answer a refresh staged because it would REMOVE a facet. Approving **re-parses** the tree, so what lands matches it at approval time; rejecting is remembered against those source hashes so it does not re-prompt every restart. 404 when nothing is staged. |
| PUT | `/shard/atlas/path` | point the atlas at a different tree (persisted as `spawn_atlas_servuo_path`, which wins over `SERVUO_PATH`). Blank clears it. Deliberately **does not import** — moving the mount and reloading the world are separate decisions — and returns fresh status so the panel can offer the import next. |
| GET | `/shard/clilocs` | cliloc-table status (`adminOnly`): every source found now (base first, then `custom/` overlays in merge order), what each contributed at the last import, readability, drift across the set, the entry count, and `missingSources`. `configured:false` is a supported state — item names then render as ids. No public counterpart: the table is never served *as* a table. |
| POST | `/shard/clilocs/import` | reload after a client patch or an overlay edit; `{force}` ignores the hash gate, `{approve}` accepts a **vanished** source (refused by default — see the table notes above). **A missing path — or the likely mistake of pointing at the client's own COMPRESSED `Cliloc.enu` — answers 200 with `status:"unavailable"` and a `code`, not 500.** `COMPRESSED` is called out by name: a 500 would say only "something broke", and the operator needs to be told which file to convert. |
| PUT | `/shard/clilocs/path` | point the site at a different cliloc base file or directory (persisted as `cliloc_client_path`, which wins over `UO_CLIENT_PATH`). Overlays are read from `custom/` beside it either way. Blank clears it. Deliberately **does not import**, same reasoning as the atlas path. |
Every admin write logs to `activity_log`.
@@ -628,7 +882,11 @@ who"; `activity_log` provides the history feed.
- **Cookie**: `httpOnly`, `sameSite=Lax`, `path=/`, and **`secure` decided per-request** (`COOKIE_SECURE=auto` → `secure: req.secure`).
- **Trusted-device MFA.** A second, separate httpOnly cookie (`rg_trust`, default 30d) — opaque, sha256-hashed server-side in `trusted_devices` — lets a browser/app **skip the TOTP step** (never the password) on future logins. It is a server-side, per-row-revocable record (never a JWT claim), so the stateless session JWT is unchanged and trust stays revocable. It only ever gates the **second factor**; it deliberately outlives logout, and is cleared on untrust / password change / password reset / TOTP disable. **Recovery codes** (bcrypt, single-use) are the 2FA-lockout fallback. All admin trusted-device/MFA actions and the self actions (`auth.login.trusted_device`, `account.trusted_device.*`, `account.recovery_code*`, `admin.trusted_device.*`, `admin.user.totp.reset`) are audit-logged. See `docs/website/TRUSTED_DEVICES_MFA.md`. This is the key to dual access: the cookie is `Secure` when reached through Pangolin (HTTPS, `X-Forwarded-Proto: https`) but **not** `Secure` when reached directly over the LAN IP on plain HTTP — so login works in both. `COOKIE_SECURE=true|false` can force it. Requires `trust proxy` (below). `localhost:5173` (Vite) and `localhost:3000` are same-site, so the cookie flows in dev too.
- **bcrypt** hashing (cost 10+); plaintext passwords never stored, logged, or returned.
- **Rate limiting** (`express-rate-limit`) on `/auth/login` and `/public/contact`.
- **Rate limiting** (`express-rate-limit`) on `/auth/login`, `/public/contact`, and — the only limited
*read* — `/public/shard/market` and `/public/shard/market/vendors/:serial` (60/min/IP). Every other
public read is an indexed lookup of bounded size; the marketplace search is a `LIKE` scan plus a
`COUNT` over the largest `shard_*` table, anonymous by default, so it is the one public GET that is
worth money to serve.
- **Validation** (`express-validator`) on all writes; centralized error handler.
- **helmet** with a Content-Security-Policy tuned for the built React SPA. The policies now live in
**`server/src/config/csp.js`** (`app.js` only wires them up):
@@ -672,6 +930,77 @@ who"; `activity_log` provides the history feed.
- **`app.set('trust proxy', 1)`** so secure cookies, `req.ip`, and rate-limiting work behind Pangolin.
- **CORS**: same-origin in prod (SPA served by Express). Dev only: allow `CLIENT_ORIGIN` (Vite, `http://localhost:5173`) with `credentials:true`.
### 6.5 Shard visibility — the audience boundary (Protocol 3.0)
Every shard-derived surface is gated by an **admin-configurable, per-feature and per-field** audience
setting. This **replaces** the static `PUBLIC_KINDS` allowlist that used to be the whole boundary.
Policy lives in `utils/shardVisibility.js`; rows live in `shard_feature_visibility`; the admin surface
is `GET`/`PUT /admin/shard/visibility` (`adminOnly`). Admin-facing guide:
[`SHARD_VISIBILITY.md`](SHARD_VISIBILITY.md). Design: [`../link/v3.md`](../link/v3.md) §3.
**The ladder.** `anonymous < logged_in < player < staff < admin`, each rung implying the ones below.
`viewerLevel(req)` resolves it: no session ⇒ `anonymous`; authenticated ⇒ `logged_in`; authenticated
with a linked game account ⇒ `player`; moderator ⇒ `staff`; admin ⇒ `admin`. **Staff satisfy the
`player` rung without a linked account** (consistent with `/player/*` being role-agnostic).
**`editor` gets no shard privilege** — it is a content role, and mapping it to `staff` would silently
widen what editors see.
**Two invariants that are code, not configuration.** Both are enforced server-side and both reject
rather than silently ignore:
1. **`acct` and `webId` are admin-only, always.** They are not exposed as configurable fields, and a
stored row attempting to loosen them is discarded on read as well as rejected on write. A character
name is visible in game; the account behind it and the website user it links to are not.
The lock is on the field's **meaning, not one spelling**: `isLockedField(key)` matches a key that
*is* or *ends in* `acct`/`webId`, case-insensitively, so the flattened forms the read models emit
(`shapeHouse` → `ownerAcct`, `shapeGuild` → `leaderWebId`) are covered too. An exact-key check was
the original implementation and it let `GET /public/shard/idoc` serve `ownerAcct` anonymously.
2. **A kind absent from `KIND_FEATURE` is never broadcast below `admin`.** Fail closed. This is what
keeps the kind map a security boundary rather than a convenience filter, and it means a shard that
starts emitting an unknown event degrades to staff-only, never to public.
**Fail-closed everywhere else too.** An unreadable visibility config withholds every public frame; a
DB failure falls back to the compiled defaults (pre-3.0 behavior), not to open; an unresolvable viewer
subscribes as `anonymous`. The ladder comparison uses **asymmetric** fallbacks by design — an unknown
*viewer* level floors to the bottom rung and an unknown *requirement* ceils to admin, so an
unrecognised value loses on both sides. (A single shared fallback cannot do that: whichever direction
it picks, it fails open on one side.)
**Three enforcement points, one config:**
| Where | Mechanism |
|---|---|
| Routes | `requireFeature(name)` — **404** when the feature is disabled (don't leak that it exists), **403** when the caller is below its audience. `projectFeature` then strips out-of-rung fields from the body. |
| SSE (`utils/shardBroadcast.js`) | Per-connection filtering. A subscriber's rung is resolved **once at subscribe time and frozen** for that connection, so a long-lived stream can't gain privilege; each frame is then mapped kind→feature, gated, and field-projected per viewer. Two subscribers can legitimately receive different versions of one event, or one of them nothing. |
| Nav | `GET /public/shard/features` returns only what the caller may reach, so the SPA never renders a link that would 403. Presentation only. |
Config reads are cached ~5s, so admin changes take effect within seconds **including on already-open
streams**. `PUBLIC_KINDS` still exists and is still exported (`notificationStreams.js`) but is now
**derived** from the kind map rather than hand-maintained, so the two cannot drift.
**`PUBLIC_KINDS` is a module-load constant and must not be used to answer "may this caller read this
kind?"** — it is computed from the compiled *defaults*, so it cannot see an admin's changes. Use
`visibleKinds(level, config)`, which resolves against the live config. `/feed` uses it; it originally
used `PUBLIC_KINDS` and consequently kept serving `guild.join` to anonymous callers after an admin had
moved `guilds` to `staff`. `visibleKinds` deliberately ignores the `stream` flag: that governs SSE
fan-out only, so a feature whose live firehose ships off (market) stays readable from stored history.
**Every read path that returns shard data must call `projectFeature`.** The stored-history endpoints
are not exempt — `/feed` returns the same events the stream does, and returning them unprojected
reopens on the REST side exactly what the stream closes. Relatedly, `shardEvents.db.list` treats an
**empty** `kinds` array as "serve nothing", never "no filter"; the fall-through it used to take would
have turned a fully-gated config into a dump of the entire event log.
`projectFeature` walks **arrays and plain objects only**. A `Date`, `Buffer` or other class instance
is passed through as a value — rebuilding one key-by-key yields `{}`, which is the difference between
the pure-JSON wire frames and the DB-backed read models whose rows carry real `Date` columns.
**Defaults reproduce pre-3.0 behavior exactly**, so installing the framework is a no-op until an admin
changes something — with deliberate exceptions, which are the leaks it was written to close.
`/public/shard/guilds`, `/public/shard/governors` and `/public/shard/feed` previously returned the raw
stored payload, whose actors carry `acct` and `webId`; `/public/shard/idoc` returned the flattened
`ownerAcct`. All are now stripped for every caller below admin.
---
## 7. Email

309
website/CLILOCS.md Normal file
View File

@@ -0,0 +1,309 @@
# Cliloc table (item and title names)
**Status:** Complete on `edge` — website [#115](https://gitea.whitlocktech.com/RunicGateway/website/pulls/115), docs [#70](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/70).
**Design:** [`docs/link/v3.md` §8.6](../link/v3.md) — Protocol 3.0, the dependency Part B/3 was sequenced behind.
A "cliloc" is UO's localization table: an integer id mapped to a display string.
**Items on the wire carry a `LabelNumber`, not a name.** The bridge has always
sent that number — `char.profile.equipment` has a `cliloc` field, reward titles
arrive as a cliloc number in string form, and every marketplace listing carries
one — but the site had no table to look it up in, so a character sheet could only
render `id 1023721` where the game renders **"quarter staff"**.
The number was never the missing piece. The table was.
## Why the operator has to convert the file
This is the awkward part, and it is not avoidable:
**Every current UO client ships its cliloc files compressed.** All eight
`Cliloc.*` files in a modern client (`chs`, `cht`, `deu`, `enu`, `esp`, `fra`,
`jpn`, `kor`) begin with a DWORD whose high byte is `0x8E` — the "Mythic"
compressed container. The plain layout this site parses is what those files
looked like *before* that change.
Decompressing it means an inverse-BWT coder with a frequency header — a few
hundred lines of bit-level work whose failure mode is plausible-looking garbage
rather than an error. The site has no business carrying that at runtime.
Two facts make the alternatives worse, not better:
- **ServUO cannot read it either.** Its bundled `Ultima.StringList` implements
only the plain layout, so on a modern client `VendorSearch.StringList` is null
and `VendorSearch.GetItemName` returns `item.Name` — usually nothing. The
shard cannot supply names on our behalf; the in-game Vendor Search gump has the
same gap.
- **Nothing client-derived may be committed.** UO's strings are EA's. The repo
ships no string table for the same reason it ships no artwork and no map
snapshot — see [`SPAWN_ATLAS.md`](SPAWN_ATLAS.md).
So the conversion happens **once, on the operator's machine, against their own
client**, and the site reads the result from a path it is given. A shard that
never does this is in a fully supported state: names render as ids, exactly as
they did before the table existed.
## Converting
> **Step-by-step operator instructions — where to get UOFiddler, where your
> client files are, and how to verify the import — are in
> [`UOFIDDLER.md`](UOFIDDLER.md).** This section covers the formats and the
> reasoning behind them.
Either format below is accepted; the site sniffs which one it was handed.
| Format | Fidelity | Notes |
|---|---|---|
| **Plain binary** (recommended) | Exact | 6-byte header, then `{int32 number, byte flag, uint16 length, UTF-8}` records |
| Delimited text | Loses leading/trailing whitespace | `number<TAB\|,\|;>text` per line; a header row, blank lines and `#` comments are ignored |
The whitespace caveat is real but cosmetic: ~1,300 of the 123,490 entries in a
stock `Cliloc.enu` are label prefixes like `"max = "` whose trailing space is
meaningful when the client concatenates a value onto them. Nothing on this site
concatenates, and every consumer passes through `displayText()`, which trims.
### Using the bundled tool
`server/tools/cliloc-export/` is a small .NET console app that drives
[UOFiddler](https://github.com/polserver/UOFiddler)'s `Ultima.dll` — the
decompressor that already exists and is already maintained — and writes the plain
format. It loads that DLL **reflectively** so it compiles against any SDK, and it
writes the records by hand because UOFiddler's own `SaveStringList` *re-compresses*
on save (its purpose is round-tripping a file back into the client, so its output
is byte-identical to its input — a trap worth knowing about).
```bash
cd website/server/tools/cliloc-export
dotnet build -c Release
# binary (recommended)
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.plain
# or tab-delimited
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv
```
A UOFiddler GUI export works too, but **not unmodified**: its Cliloc tab writes
`Number;Text;Flag` — three columns, the flag *last* — and the parser reads
`number<separator>text`, so the trailing field is absorbed into the name and
every item renders as `quarter staff;0`. Stripping it is one `sed`, given in
[`UOFIDDLER.md`](UOFIDDLER.md) §Route B.
The parser already tolerates `number,flag,text`, with the flag in the *middle*.
It is not extended to cover the trailing form because a final `;0` is
indistinguishable from a name that genuinely ends that way — a heuristic there
would corrupt real names to save the operator one command.
## Shard-added and shard-edited items
**Shards edit items and add new ones**, and those carry cliloc ids no stock
client table has. The table is therefore built from a **set** of sources, all
re-read on every boot and hash-gated together — the same shape as the spawn
atlas, which reads `Regions.xml` + `Locations/*.xml` + `Spawns/*.xml` +
`ChampionSpawns.xml` and merges them:
```
<cliloc path>/
clilocs.plain ← base: the converted client table
custom/
01-uomysticmoon.tsv ← overlays: shard additions and overrides
02-events.tsv
```
Overlays use the same delimited-text format, are read in **sorted order**, and
**later sources win** — so an overlay both *adds* ids the client never had and
*overrides* stock ones the shard has re-purposed. Any `.tsv`, `.csv`, `.txt`,
`.enu` or `.plain` file in `custom/` is picked up; anything else (a `README.md`,
say) is ignored.
Adding, editing or removing any overlay counts as drift, so a new custom item
needs only a file edit and a restart — or the admin panel's Import button.
**Adding one item never means re-exporting a 5 MB client file.**
The import result reports what each source contributed, which is how you confirm
an overlay took effect — `overrode: 0` on a file meant to re-label stock items
says it did not:
```json
"sources": [
{ "label": "clilocs.plain", "kind": "base", "entries": 123490, "added": 123490, "overrode": 0 },
{ "label": "custom/uomysticmoon.tsv", "kind": "custom", "entries": 2, "added": 1, "overrode": 1 }
]
```
**Why a convention rather than discovery.** Everywhere else this pipeline follows
the shard's own files, but **ServUO has no server-side notion of a custom
cliloc** — they live in the patched client a shard distributes to its players,
and nothing in the tree declares them. There is nothing to discover, so `custom/`
is the one thing here that is our convention rather than the shard's. (An
operator who *does* patch their client cliloc needs no overlay at all: convert
the patched file and their edits are simply in the base.)
Measured on the live shard for scale: its script tree references **16,434** cliloc
ids and only **37** are absent from the stock client table — tens of entries
against a 67k base, which is what makes an overlay the right shape rather than a
second full table.
## Configuring the path
Two ways to point at the sources, the setting winning over the environment:
| Source | Notes |
|---|---|
| `cliloc_client_path` setting | Admin-editable (Admin → Shard); takes effect on the next refresh without a redeploy |
| `UO_CLIENT_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |
The value may be **the base file itself or a directory to search**, because both
are natural answers to "where is it". Overlays are read from a `custom/`
directory beside the base **either way** — pointing at a file does not forfeit
them.
A directory is searched case-insensitively (the client writes `Cliloc.enu` on
Windows; the site usually runs on Linux) for, in order: `clilocs.tsv`,
`clilocs.csv`, `clilocs.plain`, `cliloc.plain`, `cliloc.plain.enu`,
`cliloc.enu.plain`, `clilocs.txt`, `cliloc.enu`.
That ordering puts explicitly-converted names first on purpose. Pointing the
setting straight at an unconverted client directory finds `cliloc.enu`, which is
compressed — and the site says so by name rather than failing obscurely:
```
status: unavailable
code: COMPRESSED
reason: This is a compressed (Mythic-format) cliloc file, which the site cannot
read. Convert it to the plain format first — see docs/website/CLILOCS.md.
```
## Refresh contract
Identical in shape to the spawn atlas, and for the same reasons:
- **It never blocks startup.** No path, an unreadable file, a wrong-format file,
a database error — all caught and logged. The site comes up either way.
- **Hash-gated.** The boot path hashes the file and skips the parse entirely when
it matches what is loaded, which is every restart that did not follow a client
patch. Measured on a stock table: **14 ms** for the no-op, **663 ms** for a full
parse and replace.
- **A `PARSER_VERSION` bump also counts as drift**, so a corrected parse reaches
an install whose client never patches.
### Two ways a refresh is refused
**A corrupt file** — the realistic failure for any single source — makes the
parser fail on a truncated record rather than yield a plausible-but-short table,
so it is caught outright. Verified: a file truncated to half its length reports
```
code: TRUNCATED
reason: Truncated record header at byte 2486759 (74909 entries read)
```
and the rows already loaded are untouched. A malformed overlay names the file it
came from (`custom/broken.tsv: No cliloc entries found…`), because "which of my
six overlay files is broken" is otherwise a guessing game.
**A source that has VANISHED** is the hazard a single file did not have. It
parses perfectly and imports a table quietly missing everything that file
contributed — and an unmounted volume looks exactly like a deliberate deletion
from here. This is the same ambiguity the atlas stages a facet removal for, so it
is escalated rather than applied:
```
status: needsReview
reason: 1 previously-loaded cliloc source(s) are missing;
the existing table is unchanged
missingSources: ["custom/uomysticmoon.tsv"]
```
`status()` reports `missingSources` too, so the panel can show it before anyone
clicks Import. An admin accepts it by re-running the import with
`{ "approve": true }`.
**Why that is a flag and not the atlas's approve/reject pair.** The atlas stores
a pending decision in its own table so that approving *re-parses the tree*, which
is what keeps a multi-megabyte blob out of the database and makes the applied
result match the tree at approval time. Here nothing is stored, so re-reading at
approval time is automatic — the decision is a single boolean on the import an
admin was already going to run.
## What gets stored
| | |
|---|---|
| Parsed from a stock `Cliloc.enu` | **123,490** entries |
| Of those, empty strings | **55,994** (ids the client reserves and never uses) |
| Stored in `shard_clilocs` | **67,496** |
Blank entries are dropped at import. A row resolving to no name is
indistinguishable from no row at all to every caller, and dropping them makes the
binary and text imports converge on **identical** content — the binary format
carries the blanks explicitly and a text export may or may not, depending on the
tool. Verified: both formats import to the same 67,496 rows with the same keys.
`text` is `TEXT`, not `VARCHAR`: the long property descriptions reach 12 KB, and
silently truncating them would be worse than storing them. The index that matters
for marketplace search is on the denormalized `shard_vendor_items.display_name`,
not here.
## How names are applied
**Resolution happens server-side.** The table is never served *as* a table and
there is no public route for it. Two reasons: 67k rows would dwarf any page that
used them, and the Android client consumes the same JSON and would otherwise need
its own copy.
`resolveMany()` takes a batch of ids and returns a `Map` holding only those that
resolved to something displayable, so "no such id" and "id with no usable name"
collapse into one branch at the call site. It never throws — a cliloc lookup is
decoration on someone's character sheet, and a database blip must not fail the
sheet. A capped in-process cache fronts it; measured cold **4.2 ms**, warm
**0.015 ms**.
### `displayText()`
Cliloc strings interpolate arguments the client pulls from an item's property
list — `~1_val~`, `~2_NAME~`. **We never have those**: the bridge sends the id,
not the packet. So a name carrying them is reduced to what is actually knowable.
| Raw | Displayed |
|---|---|
| `quarter staff` | `quarter staff` |
| `cold damage ~1_val~%` | `cold damage` |
| `[~1_stuff~]` | *(nothing — the whole string was the argument)* |
| `50%` | `50%` |
| `Runic Gateway Sigil (v2)` | `Runic Gateway Sigil (v2)` |
**Punctuation is only tidied when a placeholder was actually removed.** The
trailing `%` in row two is the unit belonging to the number we never had, and the
brackets in row three only ever wrapped the argument — but a string with no
placeholder has no such debris, and trimming it anyway corrupts real names. Rows
four and five are the ones that caught it: a shard's custom
`"Runic Gateway Sigil (v2)"` rendered as `"(v2"` while the bracket trim was
unconditional.
### Consumers
- **Character sheet equipment.** `enrichCharProfile` attaches `clilocName` to each
item. A player-given `name` always wins — "Bob's lucky axe" must not be
relabelled "hatchet" — and the client re-states that precedence.
- **Reward titles.** `titles.rewardResolved` is a parallel array with the numeric
entries turned into words (`null` where nothing resolved). The sheet used to
*skip* numeric reward titles entirely, having no way to render them.
- **Marketplace listings** (Protocol 3.0 §8) denormalize the resolved name into
`shard_vendor_items.display_name` so search can index it.
## Admin surface
All admin-only, alongside the atlas under Admin → Shard:
| Route | Purpose |
|---|---|
| `GET /api/v1/admin/shard/clilocs` | Sources found, what each contributed at the last import, readability, drift, entry count, `missingSources` |
| `POST /api/v1/admin/shard/clilocs/import` | Reload after a client patch or an overlay edit; `{ "force": true }` reimports an unchanged set, `{ "approve": true }` accepts a vanished source |
| `PUT /api/v1/admin/shard/clilocs/path` | Set the path; blank disables resolution |
A refresh **result is not an exception**: a missing file, or the likely mistake of
pointing at the client's own compressed `Cliloc.enu`, answers `200` with
`status: "unavailable"` and a reason. A `500` would say only "something broke";
the operator needs to be told which file to convert. Setting the path
deliberately does **not** import as a side effect — the response carries the
refreshed status so the panel can offer that as the next step.

167
website/MARKETPLACE.md Normal file
View File

@@ -0,0 +1,167 @@
# Marketplace — the player-vendor index
**Status:** On `edge` — servuo-plugins [#5](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/5), link [#19](https://gitea.whitlocktech.com/RunicGateway/link/pulls/19), website [#116](https://gitea.whitlocktech.com/RunicGateway/website/pulls/116).
**Design:** [`docs/link/v3.md` §8](../link/v3.md) — Protocol 3.0 Part B/3.
**Depends on:** [`CLILOCS.md`](CLILOCS.md) — without a cliloc table, listings render as item ids.
The marketplace is a searchable index of every player vendor on the shard: what
each shop is selling, for how much, and where it is standing. It is the same set
the in-game **Vendor Search** gump reads, offered from outside the game — so a
player can find the vanquishing kryss they want before logging in, and someone
who does not play at all can see that the economy exists.
Page: `/site/market`, plus `/site/market/vendors/:serial` for one shop.
## Three things the pages must say out loud
Everything below follows from how the data is gathered, and each has a visible
consequence the UI is required to surface.
**1. The prices are not live.** The shard sweeps vendors **round-robin** — at
most `Bridge.MarketSweepBatch` shops per tick — so a given shop can be a full
cycle behind. The page carries a *"prices last refreshed N minutes ago"* banner
driven by the **oldest** vendor row, not the newest: the one stale shop is the
one that wastes somebody's trip.
**2. A shop can be truncated.** `Bridge.MarketMaxListings` (250 by default) caps
how many listings one frame carries. A commodity reseller with thousands of
stacked resources is a real thing, and an uncapped frame for one is measured in
megabytes. Over the cap the shop reports `truncated`, and the vendor page says
*"showing 250 of 3,104 — this shop holds more than the shard publishes"* rather
than presenting a partial shop as complete.
**3. An item may have no name.** Items on the wire carry a cliloc id, not a name.
On a shard whose operator has not converted a cliloc table
([`CLILOCS.md`](CLILOCS.md)) the honest render is the item id — never an invented
label, which would be indistinguishable from a real one.
## Privacy: the player's own toggle wins
Only vendors whose owner left the in-game **Vendor Search** flag ON are ever sent
to the site. A player who hides their shop in game is hidden here too, and no
admin setting overrides that. When they hide one that was already indexed, the
shard emits `vendor.listing.remove` and the row is deleted — so revoking consent
takes effect, it does not merely stop refreshing.
Shop name, owner character name and location default to **Everyone**, because the
stock Vendor Search gump already shows exactly that set to any player in game.
They remain admin-configurable; see [`SHARD_VISIBILITY.md`](SHARD_VISIBILITY.md).
Account names and website user ids never cross the wire at all.
## How it is put together
```
ServUO uo-link sidecar website
────── ─────────────── ───────
BridgeMarket.cs vendors table shard_vendors
round-robin sweep ──────► (whole frame blob) ──────► shard_vendor_items
per-vendor diff GET /market (paged) + display_name
vendor.listing resolved at ingest
vendor.listing.remove
```
**The shard side** walks at most `MarketSweepBatch` vendors per tick from a
persistent cursor, diffs each against what it last published, and emits a whole
frame for any shop that moved. Per-tick cost is therefore bounded by the batch,
not by how many vendors the world holds — full coverage takes
`ceil(vendors / batch) × MarketSweepSeconds`.
**The sidecar** stores each frame whole and serves `GET /market`, its only paged
read. It normalizes nothing and defines no audiences: it is a dumb forwarder, and
search is the website's job.
**The website** splits each frame into a vendor row and its listings, replacing
that vendor's whole listing set inside one transaction (the frame is
authoritative for that vendor, never a delta). Item names are resolved against
the cliloc table **on the way in** and stored denormalized, which is what makes
search-by-name possible and keeps the cliloc table off the hot path.
## Operating it
Everything is in `Config/Bridge.cfg` on the shard. There is nothing to configure
on the website.
| Setting | Default | What it does |
|---|---|---|
| `MarketEnabled` | `true` | Master switch. Off publishes nothing; the page shows an empty index. |
| `MarketSweepSeconds` | `60` | Tick interval. |
| `MarketSweepBatch` | `25` | Vendors inventoried per tick. Clamped 1..500. |
| `MarketMaxListings` | `250` | Per-shop listing cap, after which `truncated`. Clamped 1..5000. |
**Faster coverage vs. per-tick cost.** Lowering `MarketSweepSeconds` or raising
`MarketSweepBatch` both refresh the index sooner and both cost more per tick.
The expensive part is the item walk, which recurses into every container a vendor
is selling — so a shard of big shops should raise the interval rather than the
batch.
`[bridge status` reports the sweep, including `lastMs` and `maxMs`:
```
market(enabled=True sweeps=42 scanned=108 emitted=27 removed=0 skipped=0
truncated=0 tracked=27 vendors=27 cursor=2 batch=25 lastMs=0.31 maxMs=15.40)
```
A tick over **50 ms** prints a rate-limited warning naming the knob:
```
[Bridge] market sweep took 82.4 ms (budget 50 ms) - lower Bridge.MarketSweepBatch (now 25) if this persists
```
Measured on a shard with 27 vendors × 40 listings (209k items, 43k mobiles):
**15.4 ms** for the first cold tick of 25 vendors, **0.3 ms** in steady state —
the diff is what makes an unchanged world nearly free. Note the arithmetic: 25
*full* shops at the 250-listing cap is 6,250 items ≈ 95 ms, over budget. Real
shops hold tens, which is why 25 is the default and why the warning exists.
`[bridge sweepnow` runs one tick immediately; `[bridge reload` re-reads the
settings above without a restart.
## Names arriving late
Item names come from the cliloc table, and the market sweep will **not** re-send
an unchanged shop just because the site learned what its items are called. So a
cliloc import triggers a bulk re-resolution of every stored listing — otherwise
an operator who configures clilocs after the first sweep would see item ids until
every shop happened to change on its own. It runs after a boot import and after
an admin import, takes ~50 ms per thousand listings, and never throws: a failure
leaves names exactly as they were.
## API
All under `/api/v1/public/shard`, gated by the `market` feature and
**rate-limited** — these are the first genuinely expensive public reads on the
site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*`
table, reachable with no session).
| Route | What |
|---|---|
| `GET /market` | Search. Returns **listings**, not vendors — "who sells X and for how much" is the question. `?q=&minPrice=&maxPrice=&itemId=&map=&region=&sort=&limit=&offset=`, `sort ∈ {price_asc, price_desc, recent}`. |
| `GET /market/meta` | Index size, staleness, and which facets and regions actually hold vendors — so a client builds its filters without running a search it will discard. |
| `GET /market/vendors/:serial` | One shop and its listings. **404** for a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer. |
`q` matches the resolved display name **or** the item's own literal name, because
an item with a player-set name (most of what is worth searching for on a
player-run shard) may carry a generic cliloc. `%` and `_` in a query are escaped:
they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let
a search for `%` match every listing on the shard.
Full schemas are in the OpenAPI spec (`ShardMarketPage`, `ShardMarketVendor`,
`ShardMarketMeta`, `ShardMarketListing`, `ShardMarketLocation`).
## Tables
`shard_vendors` (one row per shop) and `shard_vendor_items` (one row per priced
listing). Both are ingest-owned; nothing else writes to them. No foreign keys,
in keeping with every other `shard_*` table — the ingest transaction is what
keeps them consistent, and an FK would turn a malformed frame into a failed write
rather than a dropped row.
There is deliberately **no `payload` column** on `shard_vendors`, unlike the
points board next door. The board's top-N is a fixed-size list read whole, so it
lives in JSON; here the items *are* the searchable rows, so they are normalized
and there is nothing left worth duplicating. The sidecar keeps the whole blob,
because outage resilience is its job.
`shard_vendor_items.display_name` is denormalized and indexed (alone, and
composite with `price` for "cheapest matching X"). See "Names arriving late"
above for how it is kept current.

83
website/PROJECT_TREE.md
View File

@@ -164,6 +164,7 @@ website/
│ │ │ ├── heroLayout.js
│ │ │ ├── shardEvents.js
│ │ │ ├── useAsync.js
│ │ │ ├── useShardFeatures.js
│ │ │ └── useShardFeed.js
│ │ ├── routes/
│ │ │ ├── admin/
@@ -190,6 +191,8 @@ website/
│ │ │ │ │ ├── SettingsAdmin.jsx
│ │ │ │ │ ├── ShardAdmin.jsx
│ │ │ │ │ ├── ShardOps.jsx
│ │ │ │ │ ├── ShardVisibility.jsx
│ │ │ │ │ ├── SpawnAtlas.jsx
│ │ │ │ │ ├── UserDetail.jsx
│ │ │ │ │ ├── UserEditor.jsx
│ │ │ │ │ ├── UsersAdmin.jsx
@@ -213,17 +216,23 @@ website/
│ │ │ │ └── ResetPassword.jsx
│ │ │ ├── public/
│ │ │ │ ├── About.jsx
│ │ │ │ ├── Atlas.jsx
│ │ │ │ ├── AtlasCreature.jsx
│ │ │ │ ├── ChampSpawns.jsx
│ │ │ │ ├── CmsPage.jsx
│ │ │ │ ├── FiveOnFriday.jsx
│ │ │ │ ├── Governors.jsx
│ │ │ │ ├── Guilds.jsx
│ │ │ │ ├── Houses.jsx
│ │ │ │ ├── Leaderboards.jsx
│ │ │ │ ├── Maintenance.jsx
│ │ │ │ ├── Market.jsx
│ │ │ │ ├── MarketVendor.jsx
│ │ │ │ ├── News.jsx
│ │ │ │ ├── Newsletter.jsx
│ │ │ │ ├── NewsletterIssue.jsx
│ │ │ │ ├── Portal.jsx
│ │ │ │ ├── Rules.jsx
│ │ │ │ ├── Screenshots.jsx
│ │ │ │ ├── Shard.jsx
│ │ │ │ ├── ShardActivity.jsx
@@ -257,9 +266,12 @@ website/
│ └── sonar-test-reporter.mjs
├── server/
│ ├── db/
│ │ ├── data/
│ │ │ └── spawnAtlas.art.example.json
│ │ ├── schema.sql
│ │ └── seed.js
│ ├── scripts/
│ │ ├── importSpawnAtlas.js
│ │ └── routeManifest.js
│ ├── src/
│ │ ├── auth/
@@ -365,15 +377,27 @@ website/
│ │ │ ├── settings/
│ │ │ │ ├── settings.db.js
│ │ │ │ └── settings.model.js
│ │ │ ├── shardAtlas/
│ │ │ │ ├── shardAtlas.db.js
│ │ │ │ └── shardAtlas.model.js
│ │ │ ├── shardClilocs/
│ │ │ │ ├── shardClilocs.db.js
│ │ │ │ └── shardClilocs.model.js
│ │ │ ├── shardEvents/
│ │ │ │ ├── shardEvents.db.js
│ │ │ │ └── shardEvents.model.js
│ │ │ ├── shardLinks/
│ │ │ │ ├── shardLinks.db.js
│ │ │ │ └── shardLinks.model.js
│ │ │ ├── shardMarket/
│ │ │ │ ├── shardMarket.db.js
│ │ │ │ └── shardMarket.model.js
│ │ │ ├── shardState/
│ │ │ │ ├── shardState.db.js
│ │ │ │ └── shardState.model.js
│ │ │ ├── shardVisibility/
│ │ │ │ ├── shardVisibility.db.js
│ │ │ │ └── shardVisibility.model.js
│ │ │ ├── trustedDevices/
│ │ │ │ ├── trustedDevices.db.js
│ │ │ │ └── trustedDevices.model.js
@@ -398,12 +422,14 @@ website/
│ │ │ │ │ ├── account.router.js
│ │ │ │ │ ├── activity.router.js
│ │ │ │ │ ├── admin.controller.js
│ │ │ │ │ ├── admin.routes.js
│ │ │ │ │ ├── authProviders.controller.js
│ │ │ │ │ ├── authProviders.router.js
│ │ │ │ │ ├── botActivity.controller.js
│ │ │ │ │ ├── botActivity.router.js
│ │ │ │ │ ├── dashboard.router.js
│ │ │ │ │ ├── discordBot.controller.js
│ │ │ │ │ ├── discordBot.router.js
│ │ │ │ │ ├── email.router.js
│ │ │ │ │ ├── emailConfig.controller.js
│ │ │ │ │ ├── imageUpload.js
│ │ │ │ │ ├── index.js
@@ -414,16 +440,25 @@ website/
│ │ │ │ │ ├── pages.controller.js
│ │ │ │ │ ├── pages.router.js
│ │ │ │ │ ├── posts.router.js
│ │ │ │ │ ├── settings.router.js
│ │ │ │ │ ├── shard.router.js
│ │ │ │ │ ├── shardAtlas.controller.js
│ │ │ │ │ ├── shardClilocs.controller.js
│ │ │ │ │ ├── shardOps.controller.js
│ │ │ │ │ ├── shardVisibility.controller.js
│ │ │ │ │ ├── uoLink.controller.js
│ │ │ │ │ ├── uoLink.router.js
│ │ │ │ │ ├── uploads.router.js
│ │ │ │ │ ├── users.router.js
│ │ │ │ │ ├── usersShard.controller.js
│ │ │ │ │ └── wiki.router.js
│ │ │ │ ├── auth/
│ │ │ │ │ ├── auth.controller.js
│ │ │ │ │ ├── auth.routes.js
│ │ │ │ │ ├── index.js
│ │ │ │ │ ├── invite.controller.js
│ │ │ │ │ ├── invite.router.js
│ │ │ │ │ ├── login.router.js
│ │ │ │ │ ├── loginGuards.js
│ │ │ │ │ ├── me.routes.js
│ │ │ │ │ ├── mobile.controller.js
│ │ │ │ │ ├── mobile.routes.js
@@ -431,7 +466,10 @@ website/
│ │ │ │ │ ├── mobileSso.routes.js
│ │ │ │ │ ├── notifications.controller.js
│ │ │ │ │ ├── notifications.routes.js
│ │ │ │ │ ├── password.router.js
│ │ │ │ │ ├── passwordReset.controller.js
│ │ │ │ │ ├── register.router.js
│ │ │ │ │ ├── session.router.js
│ │ │ │ │ ├── sso.controller.js
│ │ │ │ │ ├── sso.routes.js
│ │ │ │ │ └── trustDevice.helper.js
@@ -439,13 +477,23 @@ website/
│ │ │ │ │ ├── internal.controller.js
│ │ │ │ │ └── internal.routes.js
│ │ │ │ ├── player/
│ │ │ │ │ ├── account.router.js
│ │ │ │ │ ├── appeals.controller.js
│ │ │ │ │ ├── player.routes.js
│ │ │ │ │ └── shard.controller.js
│ │ │ │ │ ├── appeals.router.js
│ │ │ │ │ ├── index.js
│ │ │ │ │ ├── shard.controller.js
│ │ │ │ │ └── shard.router.js
│ │ │ │ ├── public/
│ │ │ │ │ ├── atlas.controller.js
│ │ │ │ │ ├── atlas.router.js
│ │ │ │ │ ├── index.js
│ │ │ │ │ ├── pages.router.js
│ │ │ │ │ ├── posts.router.js
│ │ │ │ │ ├── public.controller.js
│ │ │ │ │ ├── public.routes.js
│ │ │ │ │ └── shard.controller.js
│ │ │ │ │ ├── shard.controller.js
│ │ │ │ │ ├── shard.router.js
│ │ │ │ │ ├── site.router.js
│ │ │ │ │ └── wiki.router.js
│ │ │ │ └── v1.router.js
│ │ │ ├── api.router.js
│ │ │ ├── cspReport.controller.js
@@ -455,6 +503,8 @@ website/
│ │ │ ├── auth.js
│ │ │ ├── botInternalClient.js
│ │ │ ├── botInternalKey.js
│ │ │ ├── clilocParse.js
│ │ │ ├── clilocSource.js
│ │ │ ├── db.js
│ │ │ ├── logger.js
│ │ │ ├── mailer.js
@@ -465,6 +515,9 @@ website/
│ │ │ ├── shardBroadcast.js
│ │ │ ├── shardIngest.js
│ │ │ ├── shardSales.js
│ │ │ ├── shardVisibility.js
│ │ │ ├── spawnAtlasParse.js
│ │ │ ├── spawnAtlasSource.js
│ │ │ ├── totp.js
│ │ │ ├── trustProxy.js
│ │ │ ├── uoLinkClient.js
@@ -483,11 +536,14 @@ website/
│ │ ├── appeals.pure.test.js
│ │ ├── appeals.test.js
│ │ ├── appLinks.test.js
│ │ ├── atlasController.test.js
│ │ ├── authController.test.js
│ │ ├── authMe.test.js
│ │ ├── authTrustedDevice.test.js
│ │ ├── botInternalKey.test.js
│ │ ├── botScore.test.js
│ │ ├── clilocParse.test.js
│ │ ├── clilocSource.test.js
│ │ ├── csp.test.js
│ │ ├── emailConfig.model.test.js
│ │ ├── honeypot.test.js
@@ -521,17 +577,32 @@ website/
│ │ ├── secretBox.test.js
│ │ ├── selfTrustedDevices.test.js
│ │ ├── session.test.js
│ │ ├── shardBroadcast.visibility.test.js
│ │ ├── shardControllerPublic.test.js
│ │ ├── shardIngest.champsPages.test.js
│ │ ├── shardIngest.market.test.js
│ │ ├── shardIngest.points.test.js
│ │ ├── shardIngest.protocol2.test.js
│ │ ├── shardIngest.ruleset.test.js
│ │ ├── shardMarket.model.test.js
│ │ ├── shardState.governorTerms.test.js
│ │ ├── shardState.model.test.js
│ │ ├── shardVisibility.test.js
│ │ ├── spawnAtlas.parse.test.js
│ │ ├── spawnAtlas.source.test.js
│ │ ├── ssoCallback.test.js
│ │ ├── ssoState.test.js
│ │ ├── ssoTrustedDevice.test.js
│ │ ├── totp.test.js
│ │ ├── trustedDevices.test.js
│ │ ├── trustProxy.test.js
│ │ ├── uoLinkClient.test.js
│ │ └── usernamePolicy.test.js
│ ├── tools/
│ │ └── cliloc-export/
│ │ ├── clilocexport.csproj
│ │ ├── Program.cs
│ │ └── README.md
│ ├── .env.example
│ ├── package-lock.json
│ ├── package.json

158
website/SHARD_VISIBILITY.md Normal file
View File

@@ -0,0 +1,158 @@
# Shard visibility — who sees which shard data
**Status:** Built (Protocol 3.0 Part A). Admin → Shard Visibility.
**Audience:** shard owners and admins.
**Companion to** [`../link/v3.md`](../link/v3.md) §3 (the design) and
[`BACKEND_DESIGN.md`](BACKEND_DESIGN.md) §6 (the security contract).
The website surfaces a lot of live shard data. What your players, your staff and the anonymous
internet may each see is **yours to decide**, per feature, from Admin → Shard Visibility.
Nothing changes until you change it: every setting ships at the value that reproduces how the site
behaved before this panel existed.
---
## 1. The audience ladder
Five rungs. Each one includes everyone below it.
| Rung | In the UI | Who that is |
|---|---|---|
| `anonymous` | **Everyone** | Anyone at all, signed in or not. |
| `logged_in` | **Signed in** | Any registered account, whether or not they've linked a game account. |
| `player` | **Linked players** | Accounts with a linked in-game account. **Staff always qualify**, linked or not. |
| `staff` | **Staff** | Admins and moderators. |
| `admin` | **Admins only** | Admins. |
Two notes that surprise people:
- **`editor` is a content role, not a shard role.** Editors write news and wiki pages; they get no
shard privilege from that. An editor is treated by link status like any other member. This matches
the rest of the site, where shard staff powers are admin-or-moderator.
- **Staff satisfy `player` without linking.** Otherwise an admin would be locked out of surfaces
they'd gated to players, which is how the `/player/*` routes already behave.
## 2. What you can set per feature
**Enabled.** Off means gone. The feature's pages return “not found”, not “forbidden” — a disabled
feature doesn't advertise that it exists.
**Who can see it.** The minimum rung, from the ladder above.
**Live updates.** Whether this feature pushes changes to open pages in real time. Turning it off
doesn't break the page; it just refreshes on load instead of updating in place.
**Sensitive fields.** Some features expose a field that deserves its own rung — you can publish the
board while holding back one column. See the table in §3.
## 3. The features, and their defaults
| Feature | What it exposes | Default | Sensitive fields |
|---|---|---|---|
| **Shard status** | Connection state, online count, gold-supply series | Everyone | — |
| **Activity feed** | Deaths, kills, skill gains, quests, logins | Everyone | — |
| **Champion spawns** | The live champion / mini-champ / sea-boss board | Everyone | — |
| **Guilds** | Guild rosters, alliances, leaders | Everyone | — |
| **Town governors** | City Loyalty governors, elections, term history | Everyone | — |
| **Houses / IDOC** | Houses in danger | Everyone | House owner → Staff · House price → Staff |
| **Players online** | Population aggregate, staff-online widget | Everyone | In-game location → Staff |
| **Shard rules** | Skill/stat caps, house limits, vet rewards, the ruleset | Everyone | Connect address → Everyone |
| **Spawn atlas** | Bestiary and spawn locations (static content) | Everyone | — |
| **Leaderboards** | Point and loyalty standings | Everyone | Character names → Everyone |
| **Marketplace** | The shard-wide player-vendor index | Everyone, **live updates off** | Vendor owner name → Everyone · Vendor owner character id → Everyone · In-game location → Everyone |
**Why the marketplace ships with live updates off.** A live feed of every vendor's full inventory
would be the single largest thing the site sends. No page needs it — the marketplace is a search over
stored data with a “prices last refreshed N minutes ago” stamp. Turn it on only if you want it.
**Why the marketplace's fields default to Everyone.** A vendor's shop name, its owner's character
name and where it is standing are *already* visible to every player in game: the stock Vendor Search
gump surfaces exactly that set to anyone who opens it. Publishing them on the site is not a new
disclosure. They stay configurable because a shard may still prefer to keep its economy behind a
login — and because "already public in game" is a judgement about your shard, not ours.
**Location is one setting covering four things.** Hiding it removes the facet, the coordinates, the
region *and* the house name together. That is deliberate: those are four ways of saying the same
thing, and a setting that hid the coordinates while publishing the house name would not have hidden
anything.
**Hiding the owner name also hides the owner character id.** They are separate settings so you can
be explicit, but leaving the id published while hiding the name achieves nothing — the leaderboards
and guild boards resolve that same id back to a character name. Set both.
**What hiding a vendor cannot do.** Only vendors whose owner left the in-game *Vendor Search* flag ON
are ever sent to the site, so a player who hides their shop in game is hidden here too — and no
setting on this page can override that. It works the other way as well: these settings control who
sees the index, not whether players can find each other's shops in game.
**Why house owner/price default to Staff.** The public Houses page has always been a "where are the
falling houses" board — location only. Owner and price are the staff view. That split is preserved.
## 4. What you cannot change
Two rules are enforced in code and are not settings. Attempting to set them returns an error rather
than silently ignoring you.
**1. Game account names and website user ids are admin-only, always.**
`acct` and `webId` never appear below the admin rung on any surface. A character *name* is visible in
game to anyone standing next to them; the **account** behind it is not, and neither is the website
user it's linked to. Publishing those would disclose something the shard itself doesn't, and would
tie a player's in-game identity to their forum identity without their consent.
This rule matches the *meaning* of a field, not one spelling of it. Some responses nest the player
who owns a record (`leader.acct`); others flatten it into the row (`ownerAcct`, `leaderWebId`,
`governorAcct`). Every one of those is locked, and the admin API refuses to configure any of them —
so a new response shape can't quietly reopen the hole by naming the field differently.
**2. Unknown event kinds are never broadcast below admin.**
The live stream maps each event kind to a feature. A kind with no mapping — a new event from a shard
plugin the site doesn't know yet, say — goes to admins only. It fails closed. This is what keeps the
stream safe by default when the shard starts sending something new: the worst case is that staff see
it and players don't, never the reverse.
## 5. How it's enforced
Three places, one config:
- **Page and API requests** are checked before the handler runs, and the response is then stripped of
any field above the caller's rung.
- **The live stream** resolves a viewer's rung once, when they connect, and freezes it for that
connection — a long-open page can't gain privilege because something changed underneath it. Each
event is then gated and stripped per viewer, so two people watching the same page can legitimately
receive different versions of the same event, or one of them nothing.
- **Navigation** hides links a viewer can't follow, so they don't hit a wall. This is presentation
only — the gate is server-side either way.
**Stored history answers the same way the live stream does.** The activity feed reads from the event
log rather than the live stream, but it resolves the *same* question against the *same* config: which
kinds you may read, and which fields survive. So moving a feature up a rung hides it from the history
as well as the stream — there is no back door where yesterday's copy of an event is more revealing
than today's.
One deliberate asymmetry: turning **live updates** off for a feature stops the push, not the reading.
The marketplace ships this way — its history and its pages are public, only the firehose is off.
Changes take effect within about five seconds, **including on streams that are already open**. You
don't need to restart anything.
If the database is briefly unreachable, the site falls back to the built-in defaults — the pre-v3
behavior — rather than to "everything is public".
## 6. Worked examples
**"I want a private shard — nothing public until people register."**
Set every feature to **Signed in**. Anonymous visitors still get the site itself; the shard data
disappears from the nav.
**"Publish the market, but don't tie vendors to players."**
Marketplace → Everyone, with **Vendor owner name** → Staff. Prices, items and locations stay public;
who owns each vendor doesn't.
**"Leaderboards for members only."**
Leaderboards → **Linked players**. Anyone who's linked a game account sees the standings; drive-by
visitors don't.
**"Let players see house owners."**
Houses → Everyone, **House owner** → Linked players. Note this is a real disclosure: house ownership
is visible in game, but the website makes it searchable in a way the game doesn't.

355
website/SPAWN_ATLAS.md Normal file
View File

@@ -0,0 +1,355 @@
# Spawn atlas
**Status:** Complete on `edge` — data pipeline in website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112), API + pages in website [#113](https://gitea.whitlocktech.com/RunicGateway/website/pulls/113).
**Design:** [`docs/link/v3.md` §6](../link/v3.md) — Protocol 3.0 Part C.
The spawn atlas is a browsable catalogue of what the shard *contains*: which
creatures spawn, where, how many, and which champion altars are configured. It
answers "where do I find a lizardman?" with **"Shrines, Yew, Isamu-Jima"** rather
than with a list of raw coordinates.
## Two things that shape the whole design
**The shard's ServUO tree is the single source of truth.** Nothing is
precomputed and committed to the repository. A shard's maps change over its
lifetime — facets get added, replaced, or renamed — and a snapshot in the repo
would silently drift from the world players actually see. The atlas is therefore
re-derived from the tree **on every server boot**.
**Facets are not a fixed list.** Nothing in the codebase names Felucca, Trammel,
or any other stock facet. The facet set is whatever the shard's own files
declare, discovered at parse time. A shard running entirely custom maps gets
exactly the same treatment as a stock one, with no code change.
## What it is not
The atlas is **static shard content, not live shard state.**
- It does **not** come from the sidecar. Nothing here touches the bridge, and
there is no event kind, no wire change and no `PROTOCOL_VERSION` bump for it.
Part C is website-only.
- It stays fully populated while the shard is down.
- Its champion table (`shard_champion_spawns`) is the *configured roster* —
"there is an Unholy Terror altar in Deceit". The live `champ.update` feed in
`shard_champs` is the separate, sidecar-fed answer to "it is on level 3 right
now". Both exist; do not conflate them.
Routes live at `/api/v1/public/atlas`, deliberately **not** under `/shard`,
because `/shard/*` means sidecar-dependent.
## Configuring the tree
The website needs to be able to *read* the ServUO tree — same host, a bind mount,
or a shared volume. Two ways to point at it, the setting winning over the
environment:
| Source | Notes |
|---|---|
| `spawn_atlas_servuo_path` setting | Admin-editable; changes take effect on the next refresh without a redeploy |
| `SERVUO_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |
With neither set the atlas is simply skipped — the site runs normally without
one.
## The boot path
On every start the server hashes the source files and compares them against what
is loaded. Unchanged (the normal case on a restart) costs one read pass, ~120 ms,
and no database write. A real change costs a ~400 ms parse and a reload.
Two contracts govern it:
**1. It never blocks startup.** No configured path, an unreadable mount, a
malformed file, a database error — every one is caught and logged, and the site
comes up serving whatever atlas it already had.
**2. A facet disappearing is never applied automatically.** Losing a facet looks
exactly like a half-copied or mid-update tree, and boot cannot tell that apart
from a real map change. That refresh is *staged* for a human instead. Everything
else — new facets, renamed regions, changed spawns — applies immediately, since
none of it can destroy something an operator would miss.
```
boot
└─ path configured? no ──▶ skip
└─ tree readable? no ──▶ warn, carry on
└─ hashes changed? no ──▶ done (nothing parsed)
└─ parse
└─ a facet would be removed?
no ──▶ import
yes ──▶ stage for admin review; atlas unchanged
```
### Approving or rejecting a staged refresh
Only the *decision* is stored, never the parsed world — a few KB of source hashes
plus the facet diff. Approving **re-parses** the tree, so what lands matches the
tree at approval time rather than at boot, and a multi-megabyte blob never sits
in the database.
A rejection is remembered against those exact source hashes, so a declined
refresh does not re-prompt on every restart. Change the tree and the hashes
differ, which asks again.
From **Admin → Spawn Atlas**, or from the CLI:
```bash
cd website/server
npm run atlas:import -- --status # what is loaded, and what is pending
npm run atlas:import -- --approve # apply the staged refresh
npm run atlas:import -- --reject # keep the current atlas, dismiss it
```
## The CLI
The server refreshes itself on boot, so this is for applying a map change
*without* a restart, and for the approve/reject flow above.
```bash
npm run atlas:import # import if the tree differs
npm run atlas:import -- --servuo <path> # override the path for this run
npm run atlas:import -- --force # reimport even if unchanged
```
`--servuo` is a per-run override and deliberately does **not** persist — changing
where the atlas permanently reads from is an admin action, not a side effect of a
one-off import.
## Sources
| File | Count (stock ServUO 57.4) | Used for |
|---|---|---|
| `Spawns/*.xml` | 13 files, ~10.5 MB | Every spawner: location, size, delays, time-of-day, creature types |
| `Data/Regions.xml` | 129 KB, nested | Named regions and their rectangles |
| `Data/Locations/*.xml` | 6 files | Landmarks (dungeon levels, town markers) |
| `Config/ChampionSpawns.xml` | 4.8 KB | Configured champion altars |
**A stock tree has 13 spawn files but only 6 facets.** `Eodon.xml`,
`GravewaterLake.xml`, `TreasuresOfKotl.xml` and the other named-area files hold
TerMur/Trammel points. The facet always comes from each record's own `<Map>`,
never from the file name.
## How a coordinate becomes a place name
This is the transform the atlas exists for, in `resolveRegion()`:
1. The highest-`priority` named region whose rectangle contains the point. Ties
break toward the **smallest** rect, so a specific room wins over the
dungeon-wide rect enclosing it.
2. Otherwise the nearest landmark within the landmark radius (200 tiles by
default), labelled by its **group** ("Covetous"), not its individual marker
("Level 1").
3. Otherwise `"Wilderness"`.
The radius cap in step 2 is what keeps step 3 reachable. Without it the nearest
landmark is always *some* landmark however far away, and open countryside gets
labelled with a dungeon on the far side of the map.
Against stock ServUO this resolves **83.2%** of points (5,369 of 6,455): 3,681 by
region, 1,688 by landmark, 1,086 Wilderness.
## Three quirks in the source data
Each of these is silent if unhandled — the atlas still builds, it is just wrong.
**Facet names disagree between sources.** `Data/Locations/*.xml` spells them
`Ter Mur` and `Tokuno Islands`, while `<Map>` and `<Facet name>` say `TerMur` and
`Tokuno`. Unreconciled, the landmark bucket is keyed differently from the points
looking it up, so the fallback never fires and every unregioned spawn on those
facets reads "Wilderness".
This is reconciled **by matching, not by a lookup table** — there is no list of
facet names anywhere. `facetKey()` collapses spelling differences (lowercase,
alphanumerics only), and `resolveFacetName()` matches a loose spelling against
the canonical set discovered from the shard's own spawn and region data, by exact
key then by prefix in either direction. A name matching nothing keeps its own
name: forcing a wrong match would file a real custom facet's landmarks under the
wrong facet, which is worse than leaving it alone.
**Spawn type tokens carry XmlSpawner directives.** The `<Objects2>` type is not
always a bare class name:
```
Fairy,{RND,4,8} alchemist/z/-50 Agralem/Name/Agralem
GargishRouser,1 greatape,true GargishRefugee/hue/34532
```
Taken literally these invent creatures that do not exist *and* split real ones in
two, because `Fairy` and `Fairy,{RND,4,8}` slug apart into separate entries. 71 of
845 were affected. Everything from the first `/` or `,` is stripped, leaving 800
real creatures.
**Case is inconsistent across files.** The same creature is `Lizardman` in one
file and `lizardman` in another. Slugging collapses them correctly, but the
display name is chosen deterministically — most common spelling wins, ties break
to the more capitalised form, then alphabetically — because otherwise it would
depend on file read order and change on an unrelated restart.
## Tables
All are **import-owned**: a refresh empties and reloads them in one transaction,
so a failed reload leaves the previous atlas intact rather than a half-loaded
world. Nothing else writes to them and nothing holds a foreign key to them — no
FKs at all, consistent with every other `shard_*` table. Full column listings in
[`BACKEND_DESIGN.md`](BACKEND_DESIGN.md).
| Table | Rows (stock) | Notes |
|---|---|---|
| `shard_spawn_creatures` | 800 | `slug` PK; `total` = sum of each type's own max; nullable `art` |
| `shard_spawn_points` | 6,455 | `spawn_range`, since `range` is reserved in MariaDB |
| `shard_spawn_point_types` | 23,927 | The many-to-many; one spawner commonly carries six types |
| `shard_regions` | 387 | Flattened out of the nesting; `rects` JSON |
| `shard_landmarks` | 558 | `grp`, since `group` is reserved in SQL |
| `shard_champion_spawns` | 25 | Configured altars, not the live feed |
| `shard_atlas_meta` | 1 | Singleton; source hashes, for the change check |
| `shard_atlas_pending` | 0–1 | Singleton; a staged refresh awaiting admin review |
`shard_spawn_creatures.name` carries a plain `INDEX`, deliberately **not
`FULLTEXT`**: ~800 rows makes a `LIKE` scan free, and FULLTEXT's minimum token
length would break searches for names like "orc".
The reload uses `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and would
implicitly commit, defeating the all-or-nothing guarantee. Point ids are assigned
explicitly rather than left to `AUTO_INCREMENT`, because the join rows need them
and `conn.batch()` reports no usable `insertId`.
## Artwork — operator-supplied, never shipped
**This project ships no creature art and no extraction tooling, and never will.**
UO sprites live in the operator's own client `.mul`/`.uop` files. They are the
operator's, not ours to redistribute.
The atlas is fully functional as text. `shard_spawn_creatures.art` is nullable
and is NULL on every fresh import; pages render without images, which is the
normal and supported state, not a degraded one.
An operator who wants art — step-by-step, with the UOFiddler side spelled out, in
[`UOFIDDLER.md`](UOFIDDLER.md) §Part 2:
1. Extracts it from **their own** client files (UOFiddler, ClassicUO tooling, or
any art extractor).
2. Drops the images under `server/uploads/atlas/`.
3. Copies `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json`
and maps creature slugs to file names.
4. Restarts, or runs `npm run atlas:import -- --force`.
Both `spawnAtlas.art.json` and `server/uploads/` are gitignored, so neither the
map nor the images can be committed by accident.
## Code layout
| File | Role |
|---|---|
| `src/utils/spawnAtlasParse.js` | **Pure and fs-free** parsers, so CI covers them with no ServUO tree. Zero dependencies. |
| `src/utils/spawnAtlasSource.js` | The only thing that reads a ServUO tree; shared by the boot path and the CLI |
| `src/model/shardAtlas/shardAtlas.db.js` | The one-transaction replace |
| `src/model/shardAtlas/shardAtlas.model.js` | The refresh decision, staging, approve/reject |
| `scripts/importSpawnAtlas.js` | Thin CLI over the model |
Parsing notes:
- `Regions.xml`, `Locations/*.xml` and `ChampionSpawns.xml` genuinely nest, and
get a small hand-rolled **subset** tokenizer — elements, attributes,
self-closing tags, comments, the XML declaration, CDATA, and the five
predefined entities plus numeric refs. It is not a general-purpose XML parser
and must not be reused as one.
- The ~10.5 MB of `Spawns/*.xml` never touches that tokenizer. Those records are
flat, so they get a streaming regex sweep instead; a DOM would allocate a node
per element across ~40 fields on every record to keep 14 of them. **Do not put
the Points files through a DOM parser.**
- `<Objects2>` is `Type:MX=n:SB=…` segments joined by `:OBJ=`. Split on `:OBJ=`
*first* — a naive `split(':')` shreds it. A single Trammel point carries six
types.
- **Respawn delays are stored in two different units, per record.** XmlSpawner
writes `MinDelay`/`MaxDelay` in minutes, and switches to seconds only when a
spawner's delay does not divide into whole minutes — flagging that with
`DelayInSec` on the same record. A `5` therefore means five *minutes* on one
spawner and five *seconds* on the next, and both are plausible respawn times,
so a reader assuming either unit is silently wrong about the other. Stock
ServUO 57.4 has ~170 second-flagged spawners out of 6,455. The parser
normalises everything to **seconds**; the API and UI carry seconds throughout.
### The parser version
`spawnAtlasSource.js` exports `PARSER_VERSION`, stored in `shard_atlas_meta`
alongside the source hashes and bumped whenever the parser derives **different
data from identical files** — a fixed misreading, a new field, a changed unit.
A refresh re-derives when the tree changed **or** the parser did. Hashing the
tree alone would be a trap: an install whose maps never change would keep serving
whatever an older build derived, indefinitely, and a deploy that corrects the
parse would never reach the data. A version mismatch counts as drift, so the
correction lands on the next boot without an operator having to know it happened.
## The API
Everything is served from MariaDB. Nothing on this path touches the sidecar, so
the pages stay complete while the shard is down — which is why the routes sit at
`/api/v1/public/atlas` and **not** under `/public/shard`, where a prefix means
"sidecar-dependent". Unlike `/shard/*`, they *are* `siteMode`-gated, like
`/posts` and `/wiki`: a bestiary is site content and follows site content's rules.
Every route carries `requireFeature('atlas')` — **404** when an admin has
disabled the feature (its pages must not reveal that it exists) and **403** when
the caller sits below its configured audience. The default is `anonymous`, so the
gates are inert until an admin changes something. Responses are field-projected
like every other shard read; `atlas` declares no sensitive fields today, and the
projection call is there so the first one that does is covered by construction
rather than by a retrofit ([`v3.md` §3.6.1](../link/v3.md)).
| Route | Answers |
|---|---|
| `GET /atlas/creatures?q=&facet=&limit=&offset=` | The bestiary, most numerous first, paginated with an unpaginated `total` |
| `GET /atlas/creatures/:slug?facet=&points=` | One creature: `places`, `spawners`, `alsoHere` |
| `GET /atlas/regions?facet=&q=` | Named regions and their rectangles |
| `GET /atlas/landmarks?facet=&q=` | Points of interest, labelled by `group` |
| `GET /atlas/champions?facet=` | The configured altar roster |
| `GET /atlas/meta` | Facets, counts and when the atlas was parsed |
Two shapes worth knowing:
- **`places` is the aggregate the atlas exists for.** "Lizardman → Shrines,
Isamu-Jima, Yew", grouped in SQL rather than by summing 6,455 point rows in
Node. `spawners` is the raw list underneath it, bounded, with
`spawnersTruncated` saying when it was cut.
- **`points` is a COUNT, `spawners` is the LIST.** The two are named apart
deliberately: the same key meaning a number on the search route and an array on
the detail route is the kind of thing a client only discovers in production.
`GET /atlas/meta` reports the **game world only**. The ServUO path, the per-file
hashes and any pending refresh describe the operator's filesystem, and live on
the admin route instead.
A facet is never validated against a list — nothing in the codebase names one.
`?facet=` is length-bounded and matched exactly, so an unknown name returns an
empty result rather than an error. The filter is an `EXISTS` over the points and
deliberately not a JSON path or `JSON_SEARCH` built from caller input: that
function treats `%` and `_` as wildcards, which would make `?facet=%` match
everything.
## The admin panel
**Admin → Spawn Atlas** (`/admin/shard-atlas`, admin-only — it reads a path on
the server's filesystem and replaces every atlas table, which is closer to a
deploy action than to moderation).
| Route | Does |
|---|---|
| `GET /admin/shard/atlas` | Status: path, readable, drift, counts, facets, pending |
| `POST /admin/shard/atlas/import` | Import now; `{ force: true }` ignores the hash gate |
| `POST /admin/shard/atlas/approve` | Apply a staged refresh, facet loss and all |
| `POST /admin/shard/atlas/reject` | Keep the current atlas; remember the decision |
| `PUT /admin/shard/atlas/path` | Point the atlas at a different tree |
Three behaviours that are deliberate:
- **An unreadable tree is a 200, not a 500.** `refresh()` reports outcomes rather
than throwing, because the boot path must never be stopped by a bad tree, and
that contract is preserved at the API. The panel says *"The tree could not be
read: …"*; a 500 would say only that something broke.
- **Setting the path does not import.** Moving the mount and reloading the world
are separate decisions, and an operator fixing a typo should not have a
multi-thousand-row replace happen under them. The response carries fresh status
so the panel can offer the import as the next step.
- **Every action is written to the admin activity log** (`shard.atlas.import` /
`.approve` / `.reject` / `.path`).

282
website/UOFIDDLER.md Normal file
View File

@@ -0,0 +1,282 @@
# Extracting from your own UO client (UOFiddler)
**Audience:** the shard operator, once, at setup time.
**Related:** [`CLILOCS.md`](CLILOCS.md) (why the cliloc conversion is unavoidable),
[`SPAWN_ATLAS.md`](SPAWN_ATLAS.md) (where creature art fits).
Two features read data that **only exists inside a UO client**, and a UO client's
files are EA's, not ours to redistribute. So neither this repo nor any image we
publish can ship them — the operator extracts from **their own** client, once,
and points the site at the result.
| Feature | What it needs | Required? | Without it |
|---|---|---|---|
| **Item / title names** ([`CLILOCS.md`](CLILOCS.md)) | `Cliloc.enu`, converted | No | Names render as raw ids — `id 1023721` instead of *quarter staff* |
| **Creature art** ([`SPAWN_ATLAS.md`](SPAWN_ATLAS.md)) | Sprites from `.mul`/`.uop` | No | Atlas pages render as text, which is the normal state |
**Both are optional and neither is load-bearing.** A shard that never does any of
this is fully supported. Do part one and skip part two if art is not worth your
time — they share only the tool.
Everything you extract stays **outside the repository**: the converted cliloc
file lives at a path you choose, and `spawnAtlas.art.json` plus `server/uploads/`
are gitignored, so none of it can be committed by accident.
---
## Part 0 — Get UOFiddler
[UOFiddler](https://github.com/polserver/UOFiddler) is the community client-file
editor. We use it because its `Ultima.dll` already contains the cliloc
decompressor, maintained by people who do this for a living.
1. Download the latest release zip from
<https://github.com/polserver/UOFiddler/releases/latest> — one asset, named
`UOFiddler-<version>.zip` (4.22.2 is ~2 MB).
2. Extract it. The zip contains a single top-level folder, and the two files that
matter are at **its root**:
```
UOFiddler-4.22.2/
Ultima.dll ← the decompressor (Part 1 needs this path)
UoFiddler.exe ← the GUI (Part 2 needs this)
plugins/
…
```
3. **Runtime:** UOFiddler 4.22.2 is built for **.NET 10**. Running `UoFiddler.exe`
needs the .NET 10 **Desktop** Runtime (Windows only); loading `Ultima.dll` from
the converter in Part 1 needs the .NET 10 runtime. Install from
<https://dotnet.microsoft.com/download/dotnet/10.0>.
### Finding your client files
The cliloc file is in your **UO client installation directory**, not in your
ServUO tree — the shard server has no copy of it. Look for `Cliloc.enu` (English;
the other seven are `chs`, `cht`, `deu`, `esp`, `fra`, `jpn`, `kor`) beside
`art.mul` / `artLegacyMUL.uop`. The EA Classic Client's default location is:
```
C:\Program Files (x86)\Electronic Arts\Ultima Online Classic\
```
**If your shard distributes its own patched client to players, use that copy.**
Any cliloc edits you shipped to players are then already in the base table and
you need no overlay for them (see [`CLILOCS.md`](CLILOCS.md) §Shard-added and
shard-edited items).
---
## Part 1 — Convert the cliloc table
**Goal:** turn the client's compressed `Cliloc.enu` into a file the site can
read, and point the site at it.
The site cannot read `Cliloc.enu` directly. Every modern client compresses it
(the "Mythic" container), and so does ServUO's own bundled `Ultima.StringList` —
which is why the shard cannot supply names on our behalf either. The full
reasoning is in [`CLILOCS.md`](CLILOCS.md) §Why the operator has to convert the
file; this section is just the procedure.
Two routes. **The bundled tool is the recommended one** — the GUI export needs a
fixup step, described below.
### Route A — the bundled converter (recommended)
Needs a .NET SDK (any version 8 or newer — the project targets `net8.0` and rolls
forward, so whatever you have works) **plus** the .NET 10 runtime from Part 0,
which is what actually loads `Ultima.dll`.
```bash
cd website/server/tools/cliloc-export
dotnet build -c Release
# plain binary — recommended, exact
dotnet run -c Release -- \
"/path/to/UOFiddler-4.22.2/Ultima.dll" \
"/path/to/UO client/Cliloc.enu" \
/srv/uo-data/clilocs.plain
# or tab-delimited text, if you want to eyeball or hand-edit it
dotnet run -c Release -- \
"/path/to/UOFiddler-4.22.2/Ultima.dll" \
"/path/to/UO client/Cliloc.enu" \
/srv/uo-data/clilocs.tsv --tsv
```
Expected output for a stock English client:
```
wrote 123490 entries to /srv/uo-data/clilocs.plain (maxTextBytes=12150, skippedOversize=0)
```
**Sanity-check that number.** A stock `Cliloc.enu` is ~123,000 entries. A few
hundred means it read something else and you should not ship the result. The
tool exits non-zero and says `no entries were written — is that a cliloc file?`
when it gets nothing at all.
The conversion runs on whatever machine has the client (usually Windows), and the
site reads the output wherever it runs — so **copy the output file to the server**
if those are different machines. It is a single self-contained file (~5 MB); the
`--tsv` form is larger but diff-able.
<details>
<summary>Errors you may hit</summary>
| Message | Cause |
|---|---|
| `Ultima.StringList not found — is that really UOFiddler's Ultima.dll?` | First argument points at some other `Ultima.dll` (ServUO ships one too — it is **not** the same assembly and cannot do this) |
| `You must install .NET to run this application` | Missing the .NET 10 runtime from Part 0 step 3 |
| `Unexpected Ultima.StringList API` | UOFiddler older than 4.21 |
| `usage: clilocexport …` | Fewer than three arguments |
</details>
### Route B — the UOFiddler GUI
Use this if you would rather not install a .NET SDK. **It needs one extra step**,
so do not skip the fixup.
1. Launch `UoFiddler.exe` and point it at your client directory when it asks
(or **Options → Path Settings**).
2. Open the **Cliloc** tab and use its **export to CSV** action.
3. It writes `CliLoc.csv` to UOFiddler's configured output path, in **three**
columns with a header row:
```
Number;Text;Flag
1023721;quarter staff;0
```
4. **Strip the trailing flag column.** The site's text parser reads
`number<TAB|,|;>text`, so that third field is otherwise absorbed into the name
and every item on the site renders as `quarter staff;0`.
```bash
sed -E 's/;[0-9]+$//' CliLoc.csv > clilocs.csv
```
```powershell
Get-Content CliLoc.csv |
ForEach-Object { $_ -replace ';\d+$','' } |
Set-Content -Encoding utf8 clilocs.csv
```
The header row needs no removal — a line whose first field is not an integer
is skipped. Blank entries (`1005008;`) survive the fixup correctly and are
dropped at import, as intended.
5. Copy `clilocs.csv` to the server.
**Why the fixup is not just done for us:** the parser already handles
`number,flag,text` — the flag in the *middle*, which is what several exports
emit. UOFiddler puts it at the *end*, where it is indistinguishable from a name
that genuinely ends in `;0`. One `sed` on the operator's side beats a parser
heuristic that would corrupt real names.
### Point the site at it
Two ways, the setting winning over the environment:
| Where | How |
|---|---|
| **Admin → Shard → cliloc path** | Takes effect on the next refresh, no redeploy |
| `UO_CLIENT_PATH` env var | The deploy-time default |
The value may be **the file itself or a directory to search** — both are natural
answers to "where is it", and overlays are picked up either way.
Setting the path deliberately does **not** import as a side effect. Click
**Import** (or `POST /api/v1/admin/shard/clilocs/import`) to load it.
### Verify
`GET /api/v1/admin/shard/clilocs`, or the Admin → Shard panel, reports what each
source contributed:
```json
"sources": [
{ "label": "clilocs.plain", "kind": "base", "entries": 123490, "added": 123490, "overrode": 0 }
]
```
Roughly **67,500 rows stored** from a stock table is correct — about half a
cliloc table is empty strings for ids the client reserves and never uses.
Then load any character sheet with equipment: items should show names rather than
`id 1023721`.
<details>
<summary>What a refusal means</summary>
A bad file answers `200` with a `status` and a named reason, not a `500` — you
need to be told *which file* to fix.
| `code` | Meaning |
|---|---|
| `COMPRESSED` | You pointed at the raw client `Cliloc.enu`. Convert it — this whole page. |
| `TRUNCATED` | Half-copied file. Re-copy; the loaded table is untouched. |
| `EMPTY` | A text source with no parseable rows — the file is named in the reason. |
| `status: needsReview` + `missingSources` | A previously-loaded source has vanished (unmounted volume? deliberate deletion?). Nothing changes until you re-import with `{ "approve": true }`. |
</details>
### Custom items — do *not* re-export for these
Shard-added items carry ids no client table has. Drop a small delimited file in a
`custom/` directory beside the base file and re-import:
```
/srv/uo-data/
clilocs.plain ← base, from this guide
custom/
01-uomysticmoon.tsv ← your additions and overrides
```
Files are read in sorted order and **later sources win**, so an overlay both adds
new ids and overrides stock ones you have re-purposed. **Adding one item never
means re-exporting a 5 MB client file.** Details in [`CLILOCS.md`](CLILOCS.md).
---
## Part 2 — Creature art for the spawn atlas (optional)
**Goal:** put sprites on atlas pages. Purely cosmetic — the atlas is fully
functional as text, and `art` is NULL on every fresh import.
**This project ships no art and no art-extraction tooling, and never will.**
1. In `UoFiddler.exe` (paths configured as in Route B step 1), open the
**Animations** tab for creature sprites — or **Items** for object art — find
the creature, and export as PNG. Right-click an entry for its export options,
or use the tab's *Export All* action for a batch. (4.22.2 added an export
option to the Animation tab's thumbnail list, which is the convenient one
here.)
2. Put the images under `server/uploads/atlas/`.
3. Copy `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json` in
the same directory and map creature slugs to file names:
```json
{
"lizardman": "lizardman.png",
"orc": "orc.png"
}
```
**Keys are the slugs the atlas API reports**, derived from the type names in
your own shard's `Spawns/*.xml` — read them off the atlas rather than guessing.
A creature with no entry renders without art, which is the default.
4. Restart, or `npm run atlas:import -- --force`.
The art map is re-read on every atlas refresh, so adding one image is an edit plus
a refresh. Both `spawnAtlas.art.json` and `server/uploads/` are gitignored.
---
## Licensing, briefly
UO's strings and sprites are EA's. Extracting from **your own** client for
**your own** shard is the arrangement here; redistributing the extracted files is
not something this project does or can advise on. That is the whole reason this
page exists instead of a download link.

60
website/api-route-inventory.json
View File

@@ -257,6 +257,26 @@
"method": "GET",
"path": "/api/v1/admin/shard/accounts"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/atlas"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/atlas/approve"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/atlas/import"
},
{
"method": "PUT",
"path": "/api/v1/admin/shard/atlas/path"
},
{
"method": "POST",
"path": "/api/v1/admin/shard/atlas/reject"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/audit"
@@ -313,6 +333,14 @@
"method": "GET",
"path": "/api/v1/admin/shard/vendors/:account"
},
{
"method": "GET",
"path": "/api/v1/admin/shard/visibility"
},
{
"method": "PUT",
"path": "/api/v1/admin/shard/visibility"
},
{
"method": "PUT",
"path": "/api/v1/admin/site-mode"
@@ -705,6 +733,30 @@
"method": "GET",
"path": "/api/v1/player/shard/vendors/:account"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/champions"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/creatures"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/creatures/:slug"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/landmarks"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/meta"
},
{
"method": "GET",
"path": "/api/v1/public/atlas/regions"
},
{
"method": "POST",
"path": "/api/v1/public/contact"
@@ -737,6 +789,10 @@
"method": "GET",
"path": "/api/v1/public/shard/economy"
},
{
"method": "GET",
"path": "/api/v1/public/shard/features"
},
{
"method": "GET",
"path": "/api/v1/public/shard/feed"
@@ -769,6 +825,10 @@
"method": "GET",
"path": "/api/v1/public/shard/presence"
},
{
"method": "GET",
"path": "/api/v1/public/shard/ruleset"
},
{
"method": "GET",
"path": "/api/v1/public/shard/status"