Compare commits
26 Commits
70d49b7792
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 8322e8318c | |||
| 25a5734107 | |||
| c583ddb77f | |||
| 29056ba996 | |||
| 186f057bc0 | |||
|
|
33a013ca98 | ||
| 5e2bc22a94 | |||
| 04cd64b838 | |||
| 916c11ee92 | |||
|
|
ea3755760d | ||
|
|
54b3002701 | ||
| 9d98109628 | |||
| d0808b766b | |||
| eab0a83f26 | |||
| 1444c77413 | |||
| 2e24427032 | |||
| afcdb373ec | |||
| 45fb4a3f15 | |||
| 5a091157d6 | |||
| 3a1bbdd165 | |||
| 32def88c4e | |||
| 71207cef16 | |||
| cdea1aa7cd | |||
| 6ce60a82c3 | |||
| 06b4a06baa | |||
|
|
f2fa6abff7 |
@@ -20,6 +20,10 @@ ci/ cross-cutting CI/quality notes
|
||||
| [HERO_EDITOR.md](website/HERO_EDITOR.md) | Hero canvas editor feature spec |
|
||||
| [WIKI_UPGRADE.md](website/WIKI_UPGRADE.md) | Wiki subsystem upgrade notes |
|
||||
| [SHARD_VISIBILITY.md](website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework |
|
||||
| [SPAWN_ATLAS.md](website/SPAWN_ATLAS.md) | The bestiary / spawn atlas: what the shard contains, parsed from its own ServUO tree |
|
||||
| [CLILOCS.md](website/CLILOCS.md) | UO's id → name table: converting one from your client so items have names |
|
||||
| [UOFIDDLER.md](website/UOFIDDLER.md) | **Operator runbook** — step-by-step extraction from your own UO client (cliloc table, creature art) |
|
||||
| [MARKETPLACE.md](website/MARKETPLACE.md) | The player-vendor index: how it is gathered, what it costs, how to tune it |
|
||||
| [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) |
|
||||
| [PROJECT_TREE.md](website/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout |
|
||||
|
||||
|
||||
119
android/PLAN.md
119
android/PLAN.md
@@ -1,6 +1,6 @@
|
||||
# Android App — Plan
|
||||
|
||||
Status: **M0–M7 landed; M7 (push notifications) both parts done — Part 1 backend (website#78) and Part 2 app (Android-app#15) plus a small `push.ntfyUrl` settings addition (website#79). Remaining: set the shard's `NTFY_*` deploy config so push lights up, and cut the v1 tag. M9 (native SSO login) is now underway backend-first — the Mobile SSO Authorization Bridge is being built in `website/` + `docs/` ahead of the app-side client (§4.2, §9 M9); custom-scheme callback only for now, App Links deferred (see [`APP_LINKS.md`](./APP_LINKS.md)).** This document is the
|
||||
Status: **M0–M7 landed; M7 (push notifications) both parts done — Part 1 backend (website#78) and Part 2 app (Android-app#15) plus a small `push.ntfyUrl` settings addition (website#79). Remaining: set the shard's `NTFY_*` deploy config so push lights up, and cut the v1 tag. M9 (native SSO login) is now underway backend-first — the Mobile SSO Authorization Bridge is being built in `website/` + `docs/` ahead of the app-side client (§4.2, §9 M9); custom-scheme callback only for now, App Links deferred (see [`APP_LINKS.md`](./APP_LINKS.md)). **M11 (Protocol 3.0 shard parity)** is scoped and next: the app sees none of the four shard features v3 added (`ruleset`, `leaderboards`, `market`, `atlas`) and does not consult `GET /public/shard/features`, so it gates shard nav on session role alone while an admin can switch any of those surfaces off or raise its audience — the v3 `edge` → `main` cutover is held until it lands (§9 M11).** This document is the
|
||||
design contract for the `RunicGateway/Android-app` repo. It was written before implementation so the
|
||||
API changes it depends on could be landed in `website/` and `docs/` first. The authoritative API
|
||||
reference is the committed OpenAPI spec at `website/server/swagger/swagger-output.json` (regenerated
|
||||
@@ -631,6 +631,8 @@ not rank).
|
||||
| News & content | everyone | `/public/posts/:category`, `/public/pages/:slug` |
|
||||
| Wiki | everyone | `/public/wiki`, `/public/wiki/categories`, `/public/wiki/tags`, `/public/wiki/:slug` |
|
||||
| Shard (live) | everyone | `/public/shard/*` + `/public/shard/stream` (SSE) |
|
||||
| **Rules / Leaderboards / Market** | everyone, *if the shard publishes them* | `/public/shard/{ruleset,points,market}` (M11) |
|
||||
| **Atlas** (bestiary) | everyone, *if the shard publishes it* | `/public/atlas/*` (M11) |
|
||||
| Contact | everyone | `/public/contact` |
|
||||
| **My Account** | signed-in | `/player/account/*` (or `/admin/account/*` for staff — see §6.4) |
|
||||
| **My Characters / Vendors / Houses** | `player` (linked) | `/player/shard/*` |
|
||||
@@ -639,6 +641,12 @@ not rank).
|
||||
Guidelines:
|
||||
- The menu is **declarative + data-driven**, not a pile of `if role ==` checks — one list of entries
|
||||
with a `minAccess`/`requiredCapability` field, filtered by the session.
|
||||
- **Session role is not the only gate on shard surfaces (M11).** Every shard-derived feature is
|
||||
*admin-configurable* — it can be switched off or raised to a higher audience rung — so a shard entry
|
||||
is filtered by the session role **and** by `GET /public/shard/features`, which reports the features
|
||||
the caller may actually reach. While that answer is unknown (in flight, or the lookup failed) the app
|
||||
shows everything: the server gates regardless, and a nav that flickers in on every load is worse than
|
||||
a link that briefly `403`s.
|
||||
- Never hide the fact that more exists behind auth in a way that misleads; anonymous users see public
|
||||
groups and a "Sign in" affordance.
|
||||
- The server is the source of truth: a hidden/greyed item is a UX convenience; every gated call still
|
||||
@@ -663,9 +671,15 @@ Guidelines:
|
||||
### 6.2 Public shard (live)
|
||||
- Status/online/feed/economy/champs/guilds/governors(+history)/presence/houses/idoc — the
|
||||
`/public/shard/*` GETs.
|
||||
- **Live updates** — subscribe to `GET /public/shard/stream` (SSE, safe kinds only) and patch the
|
||||
in-memory boards in place (champ/guild/city/house/presence update+remove frames). Reconnect with
|
||||
backoff; fall back to poll if SSE drops.
|
||||
- **Live updates** — subscribe to `GET /public/shard/stream` (SSE) and patch the in-memory boards in
|
||||
place (champ/guild/city/house/presence update+remove frames). Reconnect with backoff; fall back to
|
||||
poll if SSE drops. What arrives on the stream is **resolved from the caller's audience rung at
|
||||
subscribe time**, not from a fixed allowlist (Protocol 3.0 §3.6) — the stream request carries the
|
||||
bearer like every other call, so a signed-in app session sees exactly what the same account sees on
|
||||
the web.
|
||||
- **Visibility + the Protocol 3.0 surfaces (M11)** — `GET /public/shard/features` drives which of these
|
||||
the menu offers; `GET /public/shard/{ruleset,points,points/:system,market,market/meta,market/vendors/:serial}`
|
||||
and `GET /public/atlas/*` are the new reads. Full contract and traps in §9 M11.
|
||||
|
||||
### 6.3 Player self-service & game data (bearer)
|
||||
- **Account** — `GET /player/account`; `PATCH /player/account/username`;
|
||||
@@ -675,6 +689,11 @@ Guidelines:
|
||||
- **My game data** — `GET /player/shard/roster/:account`, `/char/:serial`, `/vendors/:account`,
|
||||
`/sales`, `/houses`. All ownership-checked server-side; a `503` means shard/sidecar down → show an
|
||||
"offline, retry" state (see §7).
|
||||
- **The character sheet carries two things the app does not yet read (M11):** the `points` block
|
||||
(per-character loyalty/points standings, Protocol 3.0 §7.3) and the server-resolved cliloc names on
|
||||
`equipment[].clilocName` / `titles.rewardResolved` (§8.6). Both are served **ungated** on this route —
|
||||
a character's own standings are self-service data and do not depend on the public `leaderboards`
|
||||
feature being visible, which is the behavior the app must mirror rather than re-gate.
|
||||
- **Presentation is text-only for v1.** Character sheets and vendor listings render as data/text — no
|
||||
item icons or paperdoll art. A richer "pretty paperdoll" view is a **future** enhancement (pending the
|
||||
art/asset work on the platform side) and is explicitly out of the first release.
|
||||
@@ -883,10 +902,98 @@ push, and Play (M6–M8) follow the designed app.
|
||||
shard-write actions degrade gracefully when the sidecar is offline. Excluded: hero/CMS block
|
||||
editor, Discord-bot config, uo-link config, OAuth-provider setup.
|
||||
|
||||
12. **M11 — Protocol 3.0 shard parity** (post-v1; scoped 2026-07-30). The website's Protocol 3.0 work
|
||||
added four shard features and, with them, an **admin-configurable visibility framework** the app
|
||||
knows nothing about. `link/v3.md` §10 deferred the app side as a follow-up; it is now scoped
|
||||
deliberately, and **the v3 `edge` → `main` cutover is held until both parts land** so web and app
|
||||
surface the same shard on the same day (decided 2026-07-30).
|
||||
|
||||
Neither part is coupled to the cutover *merge order*, which is what makes holding it a schedule
|
||||
decision rather than a technical one: against a pre-v3 website every new route and
|
||||
`/public/shard/features` simply `404`s, and each consumer below falls back to exactly today's
|
||||
behavior. The app declares no protocol version and never talks to the sidecar.
|
||||
|
||||
- **Part 1 — the visibility rules + the read-model adds.** The security-shaped half, reviewed on
|
||||
its own:
|
||||
- `GET /public/shard/features` → `{ level, features[] }`: the features **this caller** may reach.
|
||||
A new singleton cache mirrors the web client's (`lib/useShardFeatures.js`): per-viewer but
|
||||
stable for a session, invalidated on sign-in/out and on a server switch.
|
||||
- `MenuEntry` gains `feature: String?` beside its existing `access`, so the one declarative menu
|
||||
(§5) filters on the session role **and** the shard's live feature config. While the lookup is
|
||||
in flight or has failed, **show everything** — the same deliberate fail-open the web client
|
||||
takes, because the server gates regardless and a nav that flickers in on every load is worse
|
||||
than a link that briefly `403`s. The gate is server-side; hiding is presentation.
|
||||
- **`404` and `403` mean different things here** and neither is a generic error:
|
||||
`requireFeature` `404`s a *disabled* feature (deliberately not disclosing that it exists) and
|
||||
`403`s a viewer *below its audience*. Both render "not available on this shard", alongside the
|
||||
existing `503` = shard offline (§7).
|
||||
- **The `level` from `/features` is authoritative — do not re-derive the rung from the role.**
|
||||
The server's ladder is `anonymous → logged_in → player → staff → admin`, where `player` means
|
||||
*a linked game account* and staff always satisfy `player` (the same superset rule `Menu.kt`
|
||||
already encodes as `isPlayer || isStaff`).
|
||||
- **`char.profile.points`** → the "Loyalty & Points" block the web character sheet gained:
|
||||
`CharProfileDto.points[{system, nameString, points, maxPoints, rank?}]`. Three traps, all of
|
||||
them things a real shard does and a fake one does not (`v3.md` §7.5): `maxPoints == 0` means
|
||||
**uncapped** and is the *common* case, so nothing may divide by it; `nameString` is usually
|
||||
`null` because most systems name themselves with a cliloc, making the humanise-the-`system`-key
|
||||
path the **primary** one rather than a fallback; and `rank` is absent unless the shard runs
|
||||
`PointsProfileRank=true` — absent and "unranked" are different answers.
|
||||
- **Cliloc-resolved names** (`v3.md` §8.6, already live on the website): `EquipmentDto` gains
|
||||
`name` + `clilocName` and `TitlesDto` gains `rewardResolved`, so equipment stops rendering as a
|
||||
layer or a bare id. Precedence is `name → clilocName → layer`: a player-given name outranks the
|
||||
resolved type name, and the server applies the same order. A shard with no cliloc table
|
||||
configured sends neither field and the sheet renders exactly as it does today.
|
||||
- `ActorDto` keeps its `acct` / `webId` fields (nullable, so nothing breaks) but its KDoc stops
|
||||
describing them as available: they are **locked to the admin rung**, always, and stripped from
|
||||
every response below it.
|
||||
- **Part 2 — the four new screens**, each hidden by its feature name in the menu:
|
||||
- **Rules** — `GET /public/shard/ruleset` (`ruleset`). A `null` body means "the shard has not
|
||||
published its ruleset yet", which is a different state from the feature being disabled. Every
|
||||
block is optional and omitted when its system is off. **`caps.skill` / `caps.totalSkill` are in
|
||||
tenths** (1000 = 100.0) and must be converted — the raw number is actively misleading, not
|
||||
merely unhelpful. Live via the `world.ruleset` frame, which is on the public stream by default.
|
||||
- **Leaderboards** — `GET /public/shard/points`, `/points/:system` (`leaderboards`). The same
|
||||
`maxPoints`/`nameString` traps as the profile block. Live via `points.board`.
|
||||
- **Market** — `GET /public/shard/market` (`q`, `minPrice`, `maxPrice`, `itemId`, `map`, `region`,
|
||||
`sort`, `limit`, `offset`), `/market/meta` for the filter options + staleness, and
|
||||
`/market/vendors/:serial` (`market`). Four things this screen must get right: it is the site's
|
||||
first **rate-limited** public endpoint, so handle `429` the way the contact form does; the
|
||||
*"prices last refreshed N minutes ago"* banner is **required, not decoration** — the shard
|
||||
sweeps vendors round-robin, so a listing can legitimately be a full cycle stale and a page
|
||||
implying live prices sends people to an item that sold twenty minutes ago; a `truncated` shop
|
||||
must say so; and `location` is a **nested object** that an admin may gate away entirely, which
|
||||
the vendor screen renders as "hidden by the shard" (a real answer) rather than as blank
|
||||
coordinates — same for `ownerName` / `ownerSerial`. **The `market` SSE fan-out is off by
|
||||
default** (a live firehose of vendor inventories would be the site's biggest bandwidth
|
||||
consumer), so the screen is a plain paginated read and must never depend on live frames.
|
||||
- **Atlas** — `GET /public/atlas/{creatures,creatures/:slug,regions,landmarks,champions,meta}`
|
||||
(`atlas`). Note the path: `/public/atlas`, **not** `/public/shard` — the atlas is static shard
|
||||
*content*, not live shard *state*, and unlike `/shard/*` it **is** `siteMode`-gated like
|
||||
`/posts` and `/wiki`, so a site in maintenance mode withholds it independently of the sidecar.
|
||||
Three traps. Two are units/naming, from `v3.md` §6.3: respawn delays are **seconds**
|
||||
throughout, and `points` is a *count* on the search route while `spawners` is the *list* on
|
||||
the detail route. The third is a **shape**: `places` is a list of
|
||||
`{facet, label, spawners, maxAlive}` **objects**, not of place-name strings — it is the
|
||||
aggregate the screen exists to show ("Shrines, Isamu-Jima, Yew"), it arrives only on the
|
||||
detail route, and typing it `List<String>` makes that whole route fail to decode while the
|
||||
request itself returns `200`.
|
||||
- **Verification** — the five-rung walk (`anonymous`, `logged_in`, `player`, `staff`, `admin`)
|
||||
against a local website on the cutover branch, per
|
||||
[`../link/v3.md`](../link/v3.md) §11 and the shard-visibility smoke harness; plus one pass with
|
||||
**every feature disabled** in Admin → Shard Visibility, confirming the app *hides* each surface
|
||||
instead of erroring on it. Unit tests cover the menu filter (role × feature set), the
|
||||
`404`/`403`/`503` mapping, and DTO decode for each new shape. **Decode tests must feed real
|
||||
captured JSON**, not DTOs built in Kotlin: the fakes under `data/api/fake/` construct objects
|
||||
directly, so they can never catch a wire/type mismatch — which is how the `places` shape above
|
||||
shipped past a green suite.
|
||||
- **Excluded**, in the same class as M10's exclusions: the admin *configuration* panels — Shard
|
||||
Visibility, Spawn Atlas and Cliloc import — alongside the hero/CMS block editor, Discord-bot
|
||||
config, uo-link config and OAuth-provider setup.
|
||||
|
||||
### Deferred (not a milestone)
|
||||
|
||||
- **`/api/mobile` facade migration + app-version floor** — briefly planned as M11 (2026-07-22), now
|
||||
**deferred with no app work scheduled**. The website's router refactor is being done in place with
|
||||
- **`/api/mobile` facade migration + app-version floor** — briefly planned as its own milestone
|
||||
(2026-07-22), now **deferred with no app work scheduled**. The website's router refactor is being done in place with
|
||||
every URL byte-identical and `/api/v1` is not being retired, so the app's ~70 hardcoded `api/v1/…`
|
||||
endpoints, its SSE path, and its SSO URLs keep working untouched. If the mobile contract ever needs
|
||||
to diverge from web, the migration comes back — starting from a one-line alias mount on the server,
|
||||
|
||||
@@ -85,6 +85,7 @@ android-app/
|
||||
│ │ │ │ │ │ │ ├── PlayerShardDto.kt
|
||||
│ │ │ │ │ │ │ ├── PostDto.kt
|
||||
│ │ │ │ │ │ │ ├── PublicDto.kt
|
||||
│ │ │ │ │ │ │ ├── ShardContentDto.kt
|
||||
│ │ │ │ │ │ │ ├── ShardDto.kt
|
||||
│ │ │ │ │ │ │ ├── SsoDto.kt
|
||||
│ │ │ │ │ │ │ └── WikiDto.kt
|
||||
@@ -106,6 +107,7 @@ android-app/
|
||||
│ │ │ │ │ ├── NotificationsRepository.kt
|
||||
│ │ │ │ │ ├── PlayerShardRepository.kt
|
||||
│ │ │ │ │ ├── SettingsRepository.kt
|
||||
│ │ │ │ │ ├── ShardFeaturesRepository.kt
|
||||
│ │ │ │ │ ├── ShardRepository.kt
|
||||
│ │ │ │ │ └── WikiRepository.kt
|
||||
│ │ │ │ ├── di/
|
||||
@@ -171,6 +173,8 @@ android-app/
|
||||
│ │ │ │ │ ├── session/
|
||||
│ │ │ │ │ │ └── SessionViewModel.kt
|
||||
│ │ │ │ │ ├── shard/
|
||||
│ │ │ │ │ │ ├── AtlasScreen.kt
|
||||
│ │ │ │ │ │ ├── AtlasViewModel.kt
|
||||
│ │ │ │ │ │ ├── ChampsScreen.kt
|
||||
│ │ │ │ │ │ ├── ChampsViewModel.kt
|
||||
│ │ │ │ │ │ ├── FrameFields.kt
|
||||
@@ -180,7 +184,13 @@ android-app/
|
||||
│ │ │ │ │ │ ├── GuildsViewModel.kt
|
||||
│ │ │ │ │ │ ├── HousesScreen.kt
|
||||
│ │ │ │ │ │ ├── HousesViewModel.kt
|
||||
│ │ │ │ │ │ ├── LeaderboardsScreen.kt
|
||||
│ │ │ │ │ │ ├── LeaderboardsViewModel.kt
|
||||
│ │ │ │ │ │ ├── LiveBoard.kt
|
||||
│ │ │ │ │ │ ├── MarketScreen.kt
|
||||
│ │ │ │ │ │ ├── MarketViewModel.kt
|
||||
│ │ │ │ │ │ ├── RulesScreen.kt
|
||||
│ │ │ │ │ │ ├── RulesViewModel.kt
|
||||
│ │ │ │ │ │ ├── ShardComponents.kt
|
||||
│ │ │ │ │ │ ├── ShardEventText.kt
|
||||
│ │ │ │ │ │ ├── ShardScreen.kt
|
||||
@@ -284,6 +294,7 @@ android-app/
|
||||
│ │ │ │ │ ├── PlayerShardDtoTest.kt
|
||||
│ │ │ │ │ ├── PublicDtoTest.kt
|
||||
│ │ │ │ │ ├── ShardBoardDtoTest.kt
|
||||
│ │ │ │ │ ├── ShardContentDtoTest.kt
|
||||
│ │ │ │ │ ├── ShardDtoTest.kt
|
||||
│ │ │ │ │ ├── SsoDtoTest.kt
|
||||
│ │ │ │ │ └── WikiDtoTest.kt
|
||||
@@ -294,7 +305,8 @@ android-app/
|
||||
│ │ │ │ └── FakeShardStream.kt
|
||||
│ │ │ └── repository/
|
||||
│ │ │ ├── AccountTrustedDevicesTest.kt
|
||||
│ │ │ └── ConnectionVersionGuardTest.kt
|
||||
│ │ │ ├── ConnectionVersionGuardTest.kt
|
||||
│ │ │ └── ShardFeaturesRepositoryTest.kt
|
||||
│ │ ├── ui/
|
||||
│ │ │ ├── admin/
|
||||
│ │ │ │ ├── AdminContentViewModelTest.kt
|
||||
@@ -304,7 +316,8 @@ android-app/
|
||||
│ │ │ ├── contact/
|
||||
│ │ │ │ └── ContactViewModelTest.kt
|
||||
│ │ │ ├── navigation/
|
||||
│ │ │ │ └── MenuAccessTest.kt
|
||||
│ │ │ │ ├── MenuAccessTest.kt
|
||||
│ │ │ │ └── MenuFeatureGatingTest.kt
|
||||
│ │ │ ├── notifications/
|
||||
│ │ │ │ └── NotificationRoutingTest.kt
|
||||
│ │ │ ├── player/
|
||||
@@ -315,6 +328,8 @@ android-app/
|
||||
│ │ │ │ ├── FrameFieldsTest.kt
|
||||
│ │ │ │ ├── LiveBoardTest.kt
|
||||
│ │ │ │ ├── ShardBoardViewModelTest.kt
|
||||
│ │ │ │ ├── ShardContentHelpersTest.kt
|
||||
│ │ │ │ ├── ShardContentViewModelTest.kt
|
||||
│ │ │ │ └── ShardEventTextTest.kt
|
||||
│ │ │ ├── theme/
|
||||
│ │ │ │ └── BrandColorTest.kt
|
||||
|
||||
446
installer/PLAN.md
Normal file
446
installer/PLAN.md
Normal file
@@ -0,0 +1,446 @@
|
||||
# Runic Gateway Installer — plan
|
||||
|
||||
Status: **planning**. No installer code exists yet. This document is the design of record; it
|
||||
supersedes the informal overview it grew out of, which described a ServUO integration that does not
|
||||
match how `servuo-plugins` actually ships (see [Corrections](#corrections-to-the-original-overview)).
|
||||
|
||||
---
|
||||
|
||||
## 1. Purpose
|
||||
|
||||
Take a stock ServUO installation and configure it for Runic Gateway with minimal manual steps, while
|
||||
keeping the components separated and independently maintainable.
|
||||
|
||||
The installer handles environment detection, ServUO overlay deployment, the optional stock-file
|
||||
patch tier, uo-link installation and service registration, version tracking, diagnostics, and
|
||||
updates from Gitea releases.
|
||||
|
||||
**It is a deployment tool, not a hosted bootstrapper.** There is no `curl | bash`, no installer
|
||||
service, and no hosted bootstrap script. Artifacts are downloaded from a Gitea release page and run.
|
||||
|
||||
**It does not replace ServUO startup behavior.** ServUO keeps running through its existing
|
||||
release/start scripts. The installer never writes a launcher.
|
||||
|
||||
### Decisions locked
|
||||
|
||||
| Question | Decision |
|
||||
|---|---|
|
||||
| Audience | **Public** — any ServUO operator, not just shards we run |
|
||||
| Code signing | **Unsigned.** `SHA256SUMS` is the trust anchor; SmartScreen/Gatekeeper warnings are expected and documented, as with most self-hosted tooling |
|
||||
| Language | **Rust** — single static binary per OS, reuses the cross-compile pattern already proven in `link/.gitea/workflows/release.yml` |
|
||||
| Plugin source | **Release tarball artifact** — no git and no Gitea credentials on the shard host |
|
||||
| Composition | **Published bundle manifest** (§7.1). CI names an exact, protocol-checked combination of component versions; the installer fetches it at run time and `--bundle <tag>` pins one. Component releases regenerate JSON, not the installer binary |
|
||||
| Token handoff | **Print token + prefilled admin URL** at the end of the run |
|
||||
| Repo | **New repo**, `RunicGateway/installer`. It deploys *both* other components, so living inside `link/` would invert the dependency |
|
||||
| ServUO version | **Warn and skip.** Patches are verified against stock 57.4 only; on anything else the base install proceeds and the patch tier is skipped with a warning. Forks are the norm in a public audience — refusing outright would block most operators |
|
||||
| Uninstall | **Never touches the ServUO tree.** Removes uo-link and its service entry, then *prints* the overlay files to delete and the patch hunks to revert. Reverting is the operator's call |
|
||||
|
||||
---
|
||||
|
||||
## 2. Corrections to the original overview
|
||||
|
||||
These are not wording nits — each one changes what the installer has to do.
|
||||
|
||||
### 2.1 There is no `RunicGateway.dll` and no `Plugins/` directory
|
||||
|
||||
The plugin ships as **C# source** and ServUO compiles it at boot. The real deployable is
|
||||
`servuo-plugins/overlay/`, which mirrors the server root:
|
||||
|
||||
```
|
||||
overlay/
|
||||
├── Config/Bridge.cfg
|
||||
└── Scripts/
|
||||
├── Scripts.csproj # Phase 0 — whole-file overwrite of a stock file
|
||||
└── Custom/Bridge/*.cs # 22 files
|
||||
```
|
||||
|
||||
So the plugin step is a hash-compare file sync, not a DLL drop — mechanically easier than the
|
||||
overview assumed. The sting is that **a successful copy does not mean a working bridge.** Per
|
||||
`link/SHARD_PREREQS.md`, `ScriptCompiler.Compile()` shells out to `dotnet build`, prints the output,
|
||||
**ignores the exit code**, and reloads the existing `Scripts.dll`. A broken script build is
|
||||
invisible: the shard boots clean on stale code. Diagnostics must therefore verify *post-boot* state,
|
||||
never treat "files copied" as success.
|
||||
|
||||
### 2.2 Stock ServUO files *are* modified — by an optional tier
|
||||
|
||||
`servuo-plugins/patches/` holds unified diffs against stock ServUO 57.4, plus two `.cs` files that
|
||||
can only be copied *after* their patch lands (they reference symbols the patch introduces):
|
||||
|
||||
| Patch | Target | Companion file | Rebuild required |
|
||||
|---|---|---|---|
|
||||
| `playervendor-sale-eventsink.patch` | `Server/EventSink.cs` | `BridgeVendorSale.cs` | **Core** — `dotnet build ServUO.sln`; the dynamic script build is not enough |
|
||||
| `playervendor-sale-gump.patch` | `Scripts/Gumps/PlayerVendorGumps.cs` | (same unit as above) | script build |
|
||||
| `commandlogging-event.patch` | `Scripts/Commands/Logging.cs` | `BridgeModerationAudit.cs` | script build |
|
||||
|
||||
Plus `overlay/Scripts/Scripts.csproj`, which overwrites a stock file (Phase 0 — it fixes the silent
|
||||
ServUO build bug above).
|
||||
|
||||
This is the hardest part of the installer. `git apply` against a hand-modified shard will fail, and
|
||||
most real shards are hand-modified. Therefore:
|
||||
|
||||
- The patch tier is **opt-in and skippable**. The base install must complete without it.
|
||||
- Always dry-run (`git apply --check`) before applying, and report per-patch.
|
||||
- When skipped or failed, say plainly what is lost: **no `vendor.sale` events, no in-game moderation
|
||||
audit forwarding**.
|
||||
- The `EventSink.cs` patch must warn loudly that a **core solution rebuild** is required, not just a
|
||||
shard restart.
|
||||
- On any ServUO version other than stock **57.4**, skip the whole tier with a warning and continue
|
||||
with the base install. Do not attempt to apply unverified diffs to an unknown tree.
|
||||
- Record applied patches in `install.json`, **and cache the applied `.patch` files** next to it
|
||||
(`/etc/runicgateway/patches/`, `%ProgramData%\RunicGateway\patches\`). Re-runs stay idempotent,
|
||||
and uninstall can print the exact hunks offline long after the release tarball is gone (§5,
|
||||
Phase 4).
|
||||
|
||||
### 2.3 Config paths collide with what the sidecar actually reads
|
||||
|
||||
The sidecar reads `$UOLINK_CONFIG`, else `sidecar.toml` in the **working directory**
|
||||
(`link/sidecar/src/config.rs`), with keys `[shard].bind`, `[web].bind`, `[web].auth_token`,
|
||||
`[store].path`. The overview proposed a `config.toml` with `[updates]`, `[link]`, `[servuo]` — keys
|
||||
the sidecar cannot read.
|
||||
|
||||
Two files, two owners:
|
||||
|
||||
| File | Owner | Contents |
|
||||
|---|---|---|
|
||||
| `/etc/runicgateway/sidecar.toml` | uo-link | The sidecar's own schema, unchanged. Service sets `UOLINK_CONFIG` to this path |
|
||||
| `/etc/runicgateway/install.json` | installer | Deployed versions, file hashes, applied patches, ServUO path, timestamps |
|
||||
|
||||
**Working-directory trap:** the sidecar writes both `sidecar.toml` and `uo-link.db` relative to CWD.
|
||||
Under `C:\Program Files\` that fails or silently lands in VirtualStore. The service definitions must
|
||||
pin `UOLINK_CONFIG` and `UOLINK_DB_PATH` explicitly:
|
||||
|
||||
- Linux: config `/etc/runicgateway/sidecar.toml`, db `/var/lib/runicgateway/uo-link.db`, dedicated
|
||||
service user
|
||||
- Windows: binary under `%ProgramFiles%\RunicGateway\`, **data under `%ProgramData%\RunicGateway\`**
|
||||
|
||||
### 2.4 The token handoff was missing entirely
|
||||
|
||||
The whole point is the website reaching the sidecar, and today that is manual and undocumented in
|
||||
the install flow: the sidecar generates a token on first run and logs it, then a human pastes base
|
||||
URL, WS URL, token, and protocol version into Admin → Shard, where it is AES-GCM encrypted and
|
||||
becomes write-only. This is the largest "I installed it and nothing happened" failure mode.
|
||||
|
||||
The installer closes it by printing a copy-paste block at the end of a successful run — see §6.
|
||||
|
||||
### 2.5 `deploy.ps1` cannot be the cross-platform deployer
|
||||
|
||||
It is PowerShell-only; a Linux ServUO host running .NET typically has no `pwsh`. It also hard-throws
|
||||
when the ServUO process is running — correct behavior, and the installer must inherit it (detect and
|
||||
refuse, rather than corrupt a live `Scripts.dll`). The installer reimplements the sync natively; it
|
||||
is a short hash-compare-and-copy that never deletes.
|
||||
|
||||
`deploy.ps1` **stays** in `servuo-plugins` as the developer-facing tool. The installer is for
|
||||
operators.
|
||||
|
||||
### 2.6 Prerequisites the overview assumed away
|
||||
|
||||
- **`servuo-plugins` has no release workflow.** Only `link` does. "Pull latest repository" is
|
||||
replaced by a release tarball, which has to be built first (Phase 0).
|
||||
- **arm64 is not buildable today.** `link/release.yml` cross-compiles only
|
||||
`x86_64-unknown-linux-gnu` and `x86_64-pc-windows-gnu`. An arm64 `.deb` needs another cross
|
||||
toolchain.
|
||||
- **The compat matrix has no home.** `PROTOCOL_VERSION` currently lives only in
|
||||
`link/sidecar/src/main.rs`. The sidecar publishes it via `X-UOLink-Version` and `/health`, and the
|
||||
website stores an expected value — but the *plugin's* protocol version is not queryable before
|
||||
boot. See §7.
|
||||
|
||||
---
|
||||
|
||||
## 3. Distribution model
|
||||
|
||||
Components are published as Gitea release artifacts. Operators download from the release page
|
||||
(browser, `curl`/`wget`, or `scp` to the server) and run the binary.
|
||||
|
||||
```
|
||||
Runic Gateway Installer v1.0.0
|
||||
├── runicgateway-installer-windows-x86_64.exe
|
||||
├── runicgateway-installer-linux-x86_64
|
||||
└── SHA256SUMS
|
||||
|
||||
uo-link v3.x.y (existing release, extended)
|
||||
├── uo-link-sidecar-windows-x86_64.exe
|
||||
├── uo-link-sidecar-linux-x86_64
|
||||
├── runicgateway-link_<ver>_amd64.deb (Phase 5)
|
||||
└── SHA256SUMS
|
||||
|
||||
servuo-plugins v<ver> (new release, Phase 0)
|
||||
├── runicgateway-overlay-<ver>.tar.gz # overlay/ + patches/ + manifest.json
|
||||
└── SHA256SUMS
|
||||
```
|
||||
|
||||
Binding those together is the **bundle manifest** (§7.1) — published by the installer repo's CI, not
|
||||
by any component, and the thing the installer actually resolves against.
|
||||
|
||||
```bash
|
||||
scp runicgateway-installer-linux-x86_64 user@server:/tmp/
|
||||
chmod +x runicgateway-installer-linux-x86_64
|
||||
sudo ./runicgateway-installer-linux-x86_64
|
||||
```
|
||||
|
||||
### Unsigned-binary posture
|
||||
|
||||
Because releases are unsigned, trust is anchored on checksums and the operator's own verification.
|
||||
The docs must state this up front rather than let users discover it as a scary dialog:
|
||||
|
||||
- Every release publishes `SHA256SUMS`; the install docs lead with the verification command for both
|
||||
OSes.
|
||||
- Windows will show a SmartScreen "unrecognized app" prompt. Documented, with the exact click path.
|
||||
- The installer verifies the SHA256 of everything **it** downloads (overlay tarball, sidecar binary)
|
||||
against the release's `SHA256SUMS` and refuses on mismatch. Self-verification is not optional just
|
||||
because the installer itself is unsigned.
|
||||
- Revisit signing if it ever becomes affordable; the release layout should not have to change.
|
||||
|
||||
---
|
||||
|
||||
## 4. Component architecture
|
||||
|
||||
```
|
||||
Runic Gateway Installer (Rust, one binary per OS)
|
||||
│
|
||||
┌───────────────┴────────────────┐
|
||||
▼ ▼
|
||||
ServUO integration uo-link
|
||||
│ │
|
||||
┌────────┴────────┐ ┌────────┴────────┐
|
||||
▼ ▼ ▼ ▼
|
||||
overlay sync patch tier (opt-in) binary install service registration
|
||||
(never deletes) (git apply + guard) + config + data (systemd / Windows SCM)
|
||||
```
|
||||
|
||||
Each component keeps its own lifecycle. ServUO's existing startup process is untouched.
|
||||
|
||||
---
|
||||
|
||||
## 5. Phases
|
||||
|
||||
### Phase 0 — prerequisites (no installer code)
|
||||
|
||||
Repo work that must land before an installer can exist.
|
||||
|
||||
1. **`servuo-plugins`: add `.gitea/workflows/release.yml`.** Retarget the release *engine* half of
|
||||
`link/release.yml` (its header comment explicitly anticipates this — the plan/release steps
|
||||
consume only `{version, changelog, artifacts}`). The adapter half produces
|
||||
`runicgateway-overlay-<ver>.tar.gz` containing `overlay/`, `patches/`, and a `manifest.json`
|
||||
(version, commit, per-file SHA256, declared protocol version, minimum ServUO version).
|
||||
2. **`link`: make the sidecar installable.** Confirm/settle default data paths, and add a way to
|
||||
read back config non-interactively (e.g. `--print-config` emitting JSON: bind addresses, token,
|
||||
protocol version, db path) so the installer does not have to scrape logs for the token.
|
||||
3. **Bundle CI in the installer repo** (§7). Compose job (read both repos' latest releases → run the
|
||||
two gates → publish `bundle.json`), the nightly cron, and the dispatch step appended to each
|
||||
component's release workflow. This must exist before Phase 1 is useful, since the installer
|
||||
resolves what to install *from* the bundle.
|
||||
4. **`docs`: this file, plus `docs/installer/INSTALL.md`** (the operator-facing guide) once the
|
||||
shape is settled.
|
||||
|
||||
### Phase 1 — installer core
|
||||
|
||||
- ServUO root detection and validation (`ServUO.exe`, `Scripts/`, `Config/`), with version detection
|
||||
and an explicit refusal when the ServUO process is running.
|
||||
- Overlay sync: fetch tarball → verify SHA256 → hash-compare against the server tree → add/change,
|
||||
**never delete**. Port of `deploy.ps1` semantics including its `-Verify` dry run (`--verify`).
|
||||
- Write `install.json`: component, version, source commit, per-file hashes, applied patches,
|
||||
timestamp.
|
||||
- Idempotent re-runs; a second run with no upstream change reports "unchanged" and writes nothing.
|
||||
|
||||
### Phase 2 — uo-link install and service
|
||||
|
||||
- Linux: binary → `/usr/bin/runicgateway-link`, config → `/etc/runicgateway/sidecar.toml`, db →
|
||||
`/var/lib/runicgateway/`, systemd unit with a dedicated user, `enable` + `start`.
|
||||
- Windows: `%ProgramFiles%\RunicGateway\`, data in `%ProgramData%\RunicGateway\`, service
|
||||
registration with automatic start and restart-on-failure.
|
||||
- Both: `UOLINK_CONFIG` and `UOLINK_DB_PATH` pinned in the service definition (§2.3).
|
||||
- Token surfacing (§6).
|
||||
|
||||
### Phase 3 — patch tier (opt-in)
|
||||
|
||||
Everything in §2.2. Detect applicability, dry-run, apply, record, warn about the core rebuild, and
|
||||
degrade loudly rather than silently.
|
||||
|
||||
### Phase 4 — diagnostics and updates
|
||||
|
||||
`runicgateway doctor` — the command that makes the whole thing supportable:
|
||||
|
||||
```
|
||||
✓ ServUO found /opt/ServUO (57.4)
|
||||
✓ Overlay in sync 23 files, all hashes match install.json
|
||||
⚠ Patch tier 1 of 3 applied — vendor.sale unavailable
|
||||
✓ uo-link installed 3.0.1
|
||||
✓ Service running, enabled
|
||||
✓ Sidecar reachable 127.0.0.1:8080 /health ok
|
||||
✓ Protocol sidecar 3 = overlay manifest 3
|
||||
✗ Shard connected no shard has dialed in since boot
|
||||
```
|
||||
|
||||
The last check matters most: it is the only thing that distinguishes "files copied" from "the bridge
|
||||
actually works" (§2.1).
|
||||
|
||||
`runicgateway update` — resolves the current bundle (§7.1), then acts asymmetrically by component,
|
||||
deliberately:
|
||||
|
||||
- **uo-link**: compare the bundle's version against what is installed → download → verify checksum →
|
||||
replace binary → restart service.
|
||||
- **plugin overlay**: download the bundle's overlay tarball → verify → re-sync → record commit →
|
||||
tell the operator ServUO must restart (the installer does not restart the shard).
|
||||
|
||||
Because both come from one bundle, an update always moves to a combination whose protocol versions
|
||||
were checked together, rather than to two independently-latest artifacts that may disagree.
|
||||
|
||||
`runicgateway uninstall` — **removes only what it exclusively owns, and never edits the ServUO
|
||||
tree.** The installer cannot know what the operator has changed in those files since deployment, so
|
||||
a clever automatic revert risks silently eating their work. It removes and it reports:
|
||||
|
||||
| Action | Scope |
|
||||
|---|---|
|
||||
| Removed | uo-link binary, its service entry (systemd unit / Windows service), `install.json` and the cached patch set |
|
||||
| Kept | `sidecar.toml` and `uo-link.db` (config and history survive; `--purge` to drop them) |
|
||||
| **Printed, not done** | Every overlay file deployed into the ServUO tree, listed by path, for the operator to delete |
|
||||
| **Printed, not done** | The exact hunks each applied patch added to `EventSink.cs`, `PlayerVendorGumps.cs`, `Logging.cs`, rendered from the cached `.patch` files, for the operator to revert by hand |
|
||||
|
||||
The printed report is also written to a file, so it survives the terminal scrollback of a long
|
||||
uninstall.
|
||||
|
||||
### Phase 5 — packaging polish
|
||||
|
||||
`.deb` packaging, Windows MSI, arm64 cross build, and optional automated backup before upgrade.
|
||||
Deliberately last: v1 can register services directly (`sc create` / a written systemd unit) and ship
|
||||
plain binaries. Nothing in Phases 1–4 should have to change to add these.
|
||||
|
||||
---
|
||||
|
||||
## 6. Token handoff (the end of a successful run)
|
||||
|
||||
```
|
||||
Runic Gateway is installed.
|
||||
|
||||
One manual step remains — connect the website to this sidecar:
|
||||
|
||||
Base URL http://<this-host>:8080
|
||||
WebSocket URL ws://<this-host>:8080/ws
|
||||
Protocol version 3
|
||||
Auth token 4f9c... (also in /etc/runicgateway/sidecar.toml)
|
||||
|
||||
Paste these into Admin → Shard on your Runic Gateway site:
|
||||
https://<your-site>/admin/shard
|
||||
|
||||
The token is write-only once saved — the site will never show it back to you.
|
||||
```
|
||||
|
||||
The installer prompts for the site URL only to build that link; it never contacts the website. A
|
||||
future "installer registers itself with the website" flow (claim code + authenticated endpoint) is
|
||||
explicitly **out of scope** — it is real backend work in a security-sensitive area and can be added
|
||||
later without changing anything here.
|
||||
|
||||
---
|
||||
|
||||
## 7. Version tracking, the bundle, and release orchestration
|
||||
|
||||
Three components version independently, bound by a protocol contract:
|
||||
|
||||
- **sidecar** — `PROTOCOL_VERSION` in `link/sidecar/src/main.rs`, exposed on `/health` and as
|
||||
`X-UOLink-Version` on every response; a mismatch is rejected `409`.
|
||||
- **website** — stores an expected protocol version in `uoLinkConfig` (admin-managed).
|
||||
- **plugin overlay** — has no queryable version before ServUO boots. The overlay release
|
||||
`manifest.json` declares it, and `install.json` records what was deployed.
|
||||
|
||||
### 7.1 The bundle manifest
|
||||
|
||||
**The bundle is the compat matrix.** Rather than the installer hardcoding versions or blindly
|
||||
resolving "latest", CI publishes a small manifest naming an exact, checked combination:
|
||||
|
||||
```json
|
||||
{
|
||||
"bundle": "2026.08.01",
|
||||
"protocol": 3,
|
||||
"link": { "version": "3.0.1", "sha256": "a91f..." },
|
||||
"overlay": { "version": "2.4.0", "commit": "a81f42c", "sha256": "7c3e..." }
|
||||
}
|
||||
```
|
||||
|
||||
The installer fetches the current bundle at run time; `--bundle <tag>` pins an older one for a
|
||||
reproducible install. Because the bundle is data, **a new `link` release regenerates ~20 lines of
|
||||
JSON and leaves the installer binary untouched** — operators do not re-download the installer to
|
||||
pick up a sidecar patch, and the installer does not accumulate releases whose code is byte-identical.
|
||||
|
||||
Two gates run at compose time, both cheap and both worth it:
|
||||
|
||||
1. The sidecar's `PROTOCOL_VERSION` must equal the overlay manifest's declared protocol version.
|
||||
This is the check that catches an `edge`/`main` protocol mismatch before it reaches an operator.
|
||||
2. Every referenced asset must exist and its SHA256 must match the publishing repo's `SHA256SUMS`.
|
||||
|
||||
### 7.2 What triggers a bundle
|
||||
|
||||
| Trigger | Why |
|
||||
|---|---|
|
||||
| `link` publishes a release | Its release job `POST`s to the installer repo's workflow-dispatch endpoint as its final step. `link/.gitea/workflows/release.yml` already declares `workflow_dispatch: {}` and already holds a `write:repository` token |
|
||||
| `servuo-plugins` publishes a release | Same, once Phase 0 gives it a release workflow |
|
||||
| Nightly cron on the installer repo | Recomputes from whatever the latest releases actually are, so a missed or failed dispatch self-heals instead of silently pinning operators to a stale sidecar |
|
||||
|
||||
`repository_dispatch` is deliberately avoided — support for it is uncertain on this Gitea version,
|
||||
whereas dispatching an existing `workflow_dispatch` workflow via the API works today.
|
||||
|
||||
### 7.3 Stale-overlay handling: dispatch, don't wait
|
||||
|
||||
Each component **self-releases on merge to its own `main`**, using the same conventional-commit
|
||||
engine. Note that "updated since the last release" must mean *releasable* commits — the engine sets
|
||||
`RELEASE=false` when nothing but `docs:`/`chore:` has landed, so a docs typo correctly does **not**
|
||||
cut an overlay release, and the bundle keeps using the existing one.
|
||||
|
||||
So by the time the installer's CI looks, the release normally already exists. If it finds
|
||||
`servuo-plugins` main ahead of its latest release *with* releasable commits, it:
|
||||
|
||||
1. fires that repo's release workflow via workflow-dispatch and **does not wait for it**,
|
||||
2. composes this bundle from the assets that exist right now,
|
||||
3. writes a loud warning into the job summary.
|
||||
|
||||
The new overlay release lands minutes later on its own and the nightly cron folds it into the next
|
||||
bundle. This gets the automation without the flaky part: dispatching another repo's workflow is
|
||||
fine — that workflow still runs its own gates — but *polling* it is not, because Gitea's dispatch
|
||||
endpoint returns no run handle, so the job would have to guess which run is its own and hold a
|
||||
runner idle meanwhile. The warning exists so a genuinely broken release workflow surfaces once
|
||||
rather than being silently retriggered every night forever.
|
||||
|
||||
### 7.4 Open risk
|
||||
|
||||
The v3 cutover is mid-flight — protocol work landed on `edge` branches with the `edge → main`
|
||||
cutover still open across four repos. Until that lands, `main` and `edge` disagree about
|
||||
`PROTOCOL_VERSION`, so the installer must not hardcode a version anywhere; it reads what the
|
||||
artifacts declare, and §7.1's gate 1 is what stops a mismatched pair from being published as a
|
||||
bundle. See `docs/link/v3.md`.
|
||||
|
||||
---
|
||||
|
||||
## 8. Open questions
|
||||
|
||||
1. **Windows service mechanism** — `sc create` against the plain console binary (simplest, works
|
||||
today), a bundled WinSW/NSSM shim, or a native `--service` mode in the sidecar using the
|
||||
`windows-service` crate (cleanest, but changes `link`). Recommendation: `sc create` for v1,
|
||||
revisit if restart semantics prove inadequate.
|
||||
2. **Does the installer manage ServUO stop/start?** Currently it refuses while ServUO runs and tells
|
||||
the operator to restart afterward. Offering to stop/start would be friendlier but means owning
|
||||
another shard's process lifecycle, and the shard's own start scripts vary.
|
||||
3. **Co-location assumption** — the shard dials out to the sidecar on loopback `127.0.0.1:7788`, so
|
||||
sidecar and ServUO must share a host. Should the installer support installing only uo-link on a
|
||||
different host, or hard-assume co-location?
|
||||
4. **Branch targeting for the new repo** — `link`, `website`, `servuo-plugins` and `docs` are
|
||||
mid-cutover between `edge` and `main`. The installer repo starts clean on `main`; the Phase 0
|
||||
`servuo-plugins` release workflow needs a target branch decision.
|
||||
|
||||
Resolved and moved into §1 / §2.2 / §5: uninstall scope, and minimum ServUO version.
|
||||
|
||||
---
|
||||
|
||||
## 9. Administrator experience
|
||||
|
||||
Before:
|
||||
|
||||
```
|
||||
find plugins → copy files → edit ServUO → download bridge → start bridge
|
||||
→ configure startup → find the token → troubleshoot paths
|
||||
```
|
||||
|
||||
After:
|
||||
|
||||
```
|
||||
download artifact → verify checksum → run installer → select ServUO directory
|
||||
→ install components → paste 4 values into Admin → Shard → start ServUO normally
|
||||
```
|
||||
@@ -31,31 +31,31 @@ Missing or wrong token → **401** `{"error":"missing or invalid auth token"}`.
|
||||
|
||||
The wire protocol is versioned so a mismatch is caught immediately instead of failing weirdly.
|
||||
|
||||
- Every response carries an **`X-UOLink-Version: 2`** header.
|
||||
- `GET /health` and the WebSocket `ws.hello` frame include `"protocol": 2`.
|
||||
- **Optionally**, send `X-UOLink-Version: 2` on your requests. If it disagrees with the sidecar, the request is rejected **409 Conflict**:
|
||||
- Every response carries an **`X-UOLink-Version: 3`** header.
|
||||
- `GET /health` and the WebSocket `ws.hello` frame include `"protocol": 3`.
|
||||
- **Optionally**, send `X-UOLink-Version: 3` on your requests. If it disagrees with the sidecar, the request is rejected **409 Conflict**:
|
||||
|
||||
```json
|
||||
{ "error": "protocol version mismatch", "sidecar_protocol": 2, "client_protocol": "1" }
|
||||
{ "error": "protocol version mismatch", "sidecar_protocol": 3, "client_protocol": "2" }
|
||||
```
|
||||
|
||||
Pin the version you built against and compare it to the header (or `/health.protocol`) at startup.
|
||||
|
||||
**v2 (Protocol 2.0)** added the account-provisioning surface (§6.x: `POST /accounts/create`, `DELETE /link/{account}`) and the `account.*` events. Outbound event kinds are **additive** — a v1 client that ignores unknown kinds keeps working against the live feed — but the new *endpoints* require a v2 sidecar. If you send `X-UOLink-Version: 1`, calls to the new endpoints are refused with the 409 above.
|
||||
|
||||
**v3 (Protocol 3.0) is being built and the version has not been bumped yet.** It is defined as *adds
|
||||
`world.ruleset`, `points.board`, `vendor.listing` / `vendor.listing.remove`*, and the bump to
|
||||
`X-UOLink-Version: 3` happens **exactly once**, at the end, when [`v3.md`](v3.md) §4's `edge` → `main`
|
||||
cutover lands — because a bump is an operator-visible hard break (409 on every protected route, and
|
||||
the website's WS closes on the `ws.hello` mismatch), so doing it per phase would break the site
|
||||
repeatedly.
|
||||
**v3 (Protocol 3.0)** adds `world.ruleset`, `points.board` and `vendor.listing` /
|
||||
`vendor.listing.remove`, with the `GET /ruleset`, `/points` and `/market` reads that serve them from
|
||||
the sidecar's store. Same shape as the v2 bump: the event kinds are additive, so a v2 client that
|
||||
ignores unknown kinds keeps working against the live feed, but the three new endpoints require a v3
|
||||
sidecar. There is deliberately **no feature-negotiation array** — v3 implies all three kinds, so the
|
||||
version number alone tells you what is available.
|
||||
|
||||
Until then, sidecars on `edge` still report `2` while already carrying some v3 kinds and endpoints.
|
||||
That is safe in the direction that matters: event kinds are additive, and a client that ignores
|
||||
unknown kinds and tolerates a `404` on a not-yet-present endpoint keeps working. What you must **not**
|
||||
do is infer feature availability from the version number during this window — probe the endpoint, or
|
||||
treat a missing `world.ruleset` as "this shard hasn't published one". There is deliberately **no
|
||||
feature-negotiation array**: v3 implies all three kinds.
|
||||
**Upgrading a v2 integration.** The bump is an operator-visible hard break in one direction only: a
|
||||
client still declaring `2` gets a 409 on every protected route and, on the WebSocket, a closed
|
||||
connection on the `ws.hello` mismatch. So update the pinned version at the same time you deploy the
|
||||
v3 sidecar. Nothing that existed in v2 changed shape, so that is the whole migration — the website
|
||||
does it with a one-shot boot migration of its `uo_link_config.protocol` row ([`v3.md`](v3.md) §4.1);
|
||||
a third-party client changes the constant it sends.
|
||||
|
||||
---
|
||||
|
||||
@@ -68,7 +68,7 @@ GET /health (no auth)
|
||||
```json
|
||||
{
|
||||
"status": "ok", // "ok" when plugin connected AND db reachable, else "degraded"
|
||||
"protocol": 1,
|
||||
"protocol": 3,
|
||||
"plugin_connected": true, // is the shard link up right now?
|
||||
"database": "ok", // "ok" | "error"
|
||||
"uptime": "3d 12h",
|
||||
@@ -91,7 +91,7 @@ A push-only stream of game events as they happen. You do **not** send commands o
|
||||
**On connect**, the first frame is:
|
||||
|
||||
```json
|
||||
{ "kind": "ws.hello", "protocol": 1 }
|
||||
{ "kind": "ws.hello", "protocol": 3 }
|
||||
```
|
||||
|
||||
**Then** a continuous stream of event frames, each with at least `t` (epoch ms) and `kind`. Route on `kind`.
|
||||
@@ -449,6 +449,78 @@ would carry ~25 zeroes. `maxPoints` follows the same `0 == uncapped` rule as the
|
||||
a points lookup stops at the character's own row, but a rank must count every row that beats them, in
|
||||
every system, on every profile build. Derive rank from `points.board` instead for anyone in the top N.
|
||||
|
||||
#### Player-vendor marketplace (Protocol 3.0)
|
||||
|
||||
The shard-wide shop index: every player vendor's shop name, owner, location and priced inventory —
|
||||
the same set the in-game **Vendor Search** gump reads, published so a site can offer the same search
|
||||
from outside the game.
|
||||
|
||||
An **amortized round-robin diff sweep**, not a snapshot RPC, and the distinction is load-bearing:
|
||||
`rpc.rs::try_route` correlates a reply on the FIRST frame carrying a matching `reqId`, so a chunked
|
||||
reply sharing one `reqId` would deliver chunk 1 to the HTTP caller and leak chunks 2..N onto the
|
||||
broadcast feed. A whole-world snapshot could not fit in one frame inside the 10 s reply timeout
|
||||
either. The per-account `vendor.snapshot` RPC (§5) is unaffected and still serves the player portal.
|
||||
|
||||
Each tick inventories at most `Bridge.MarketSweepBatch` vendors (default 25) starting from a
|
||||
persistent cursor, so **per-tick cost is bounded independently of world size**; full coverage takes
|
||||
`ceil(vendors / batch) × MarketSweepSeconds`. A vendor is emitted only when its contents, prices,
|
||||
shop name or location actually change.
|
||||
|
||||
| kind | fields | notes |
|
||||
|------|--------|-------|
|
||||
| `vendor.listing` | `serial`, `shopName`, `ownerSerial`, `ownerName`, `location{}`, `count`, `total`, `truncated`, `items[]` | One vendor's complete shop — **never a delta**. The latest frame for a `serial` replaces the previous one outright. |
|
||||
| `vendor.listing.remove` | `serial` | The shop is gone from the index: dismissed, expired, or its owner switched off the in-game Vendor Search flag. |
|
||||
|
||||
```json
|
||||
{"kind":"vendor.listing","serial":"0x40001234",
|
||||
"shopName":"Darrow's Bargains","ownerSerial":"0x1A2B","ownerName":"Darrow",
|
||||
"location":{"map":"Trammel","x":1421,"y":1699,"z":0,
|
||||
"region":"Britain","house":"Darrow's Villa"},
|
||||
"count":2,"total":2,"truncated":false,
|
||||
"items":[{"serial":"0x40012ABC","itemId":3922,"hue":0,"amount":1,
|
||||
"price":25000,"name":null,"cliloc":1023721},
|
||||
{"serial":"0x40012ABD","itemId":7026,"hue":1157,"amount":3,
|
||||
"price":500,"name":"a shard sigil","cliloc":1041243}],
|
||||
"t":1752489280000}
|
||||
```
|
||||
|
||||
**Six things consumers get wrong.**
|
||||
|
||||
1. **`name` is `null` for nearly every item; `cliloc` is the real label.** Items carry a
|
||||
`LabelNumber`, not a name. The plugin deliberately never calls `VendorSearch.GetItemName`, which
|
||||
builds an `ObjectPropertyList`, serialises it and byte-parses the packet **per item** — a
|
||||
multi-hundred-millisecond stall across a full pass. (It would not work anyway: every current
|
||||
client ships its cliloc files compressed and ServUO's bundled `Ultima.StringList` cannot read
|
||||
them, so the in-game gump has the same gap.) Resolve clilocs consumer-side; a non-null `name` is a
|
||||
player-set literal and is strictly more specific, so **prefer it over the cliloc**.
|
||||
2. **`location` is one nested object, and it may be absent entirely.** It is nested so that a
|
||||
consumer gating vendor whereabouts gates one field rather than five that can drift apart — the
|
||||
website's `market.location` rule removes the whole object. Treat a missing `location` as "not
|
||||
published", not as an error.
|
||||
3. **`truncated` means the shop holds more than the frame carries.** `count` is what was published,
|
||||
`total` is what the shop actually holds, capped by `Bridge.MarketMaxListings` (default 250). A
|
||||
commodity reseller with thousands of stacked resources is real and an uncapped frame for one is
|
||||
measured in megabytes. Say "showing 250 of 3,104" rather than presenting a partial shop as
|
||||
complete.
|
||||
4. **`child: true` means the price buys the ENCLOSING CONTAINER.** ServUO prices a container as a
|
||||
unit and everything inside inherits that price with no `VendorItem` of its own; `DoSearch`
|
||||
surfaces the same flag. A UI that prints the container's price against each item inside it is
|
||||
lying about the shard.
|
||||
5. **Opted-out vendors are absent, and that is a privacy control.** `pv.VendorSearch` is the player's
|
||||
own in-game toggle and the sweep honours it — hide your vendor in game and it is hidden here too.
|
||||
The same goes for `Map.Internal` and a null backpack, matching `DoSearch`. Process
|
||||
`vendor.listing.remove` promptly: it is how a player *revoking* that consent reaches you.
|
||||
6. **Prices are inherently stale, by design.** The round-robin sweep means a shop can be a full cycle
|
||||
behind. Any UI over this must say how old the data may be — the website derives it from the oldest
|
||||
vendor row.
|
||||
|
||||
Entries carry `ownerSerial`/`ownerName` and **never `acct` or `webId`**, the same rule `points.board`
|
||||
follows. Absent entirely if the shard runs `Bridge.MarketEnabled=false` or an older plugin. Render
|
||||
from `GET /market` (§6) on connect, then keep live with these events — though note that a live
|
||||
firehose of whole vendor inventories is the largest stream the bridge produces, and a consumer that
|
||||
only needs a browsable index (as the website does) is better served by the REST read plus the
|
||||
periodic re-sweep.
|
||||
|
||||
---
|
||||
|
||||
## 5. REST — read queries
|
||||
@@ -829,6 +901,35 @@ standings built over months and blanking them during a restart reads as data los
|
||||
one excluded by `Bridge.PointsSystems`). That is distinct from a published board nobody has scored in
|
||||
yet, which is **200** with an empty `top[]` — and the two are worth rendering differently.
|
||||
|
||||
### Player-vendor marketplace (Protocol 3.0)
|
||||
|
||||
```
|
||||
GET /market?limit=200&offset=0
|
||||
→ { "vendors": [ {"kind":"vendor.listing","serial":"0x40001234",
|
||||
"shopName":"Darrow's Bargains","ownerSerial":"0x1A2B","ownerName":"Darrow",
|
||||
"location":{"map":"Trammel","x":1421,"y":1699,"z":0,
|
||||
"region":"Britain","house":"Darrow's Villa"},
|
||||
"count":2,"total":2,"truncated":false,"items":[ ... ],"t":...}, ... ],
|
||||
"total": 137, "limit": 200, "offset": 0 }
|
||||
```
|
||||
|
||||
Every vendor's latest shop, exactly as `vendor.listing` published it (§4 for the frame and its six
|
||||
gotchas). Served from the sidecar's projection, so it answers while the shard is down.
|
||||
|
||||
**This is the only PAGED read the sidecar serves**, because it is the only board that can be a whole
|
||||
world's inventory. `limit` is clamped to 1..1000 (default 200); `total` is returned so a caller knows
|
||||
when to stop rather than paging until it sees a short page, which would race a concurrent sweep.
|
||||
Ordering is by **serial**, not by shop name — a serial is stable while a shop name is renameable, so
|
||||
a rename mid-walk cannot make a vendor skip or repeat a page.
|
||||
|
||||
The route is `/market` and deliberately **not** `/vendors`: `/vendors/{account}` next door is the
|
||||
per-account RPC (§5), and two routes a prefix apart meaning "this player's shops" and "every shop on
|
||||
the shard" is a trap nobody wins.
|
||||
|
||||
Frames are served **verbatim**, owner names and coordinates included. That is not an oversight: the
|
||||
sidecar defines no audiences. Deciding who may see what is the consuming site's job — see
|
||||
[`v3.md`](v3.md) §3 for how the website does it.
|
||||
|
||||
---
|
||||
|
||||
## 7. Status codes
|
||||
@@ -854,7 +955,7 @@ yet, which is **200** with an empty `top[]` — and the two are worth rendering
|
||||
A typical character page:
|
||||
|
||||
```js
|
||||
const H = { "Authorization": `Bearer ${TOKEN}`, "X-UOLink-Version": "2" };
|
||||
const H = { "Authorization": `Bearer ${TOKEN}`, "X-UOLink-Version": "3" };
|
||||
|
||||
// 1. render the roster
|
||||
const roster = await fetch(`${BASE}/roster/${account}`, { headers: H }).then(r => r.json());
|
||||
|
||||
29
link/PLAN.md
29
link/PLAN.md
@@ -284,6 +284,15 @@ Counts in `hello` are a live snapshot taken on the Core thread, not a cached val
|
||||
|
||||
`Item.Name` is frequently `null`; the display name is `LabelNumber`, a cliloc id. **There is no `Data/Cliloc.enu` in this repo** — `BRIDGE_FINDINGS.md` §IV.4 is wrong about this. Cliloc data lives in the client install, which `DataPath` resolves to `D:\Games\Electronic Arts\Ultima Online Classic\`. Ship **both** `name` (when non-null) and `cliloc`, and resolve the number **on the website** against a cliloc map. That avoids a server-side dependency on the client directory.
|
||||
|
||||
**Update (3.0).** That recommendation held, and the reason it had to hold turned out to be stronger
|
||||
than "avoids a dependency": **ServUO cannot resolve clilocs either.** Every current client ships its
|
||||
`Cliloc.*` files compressed, and the bundled `Ultima.StringList` reads only the older plain layout —
|
||||
so `VendorSearch.StringList` is null and `VendorSearch.GetItemName` returns `item.Name` on any modern
|
||||
shard. The in-game Vendor Search gump has the same gap, which is why `vendor.listing` never calls it.
|
||||
Pushing name resolution to the plugin was never an option. See [`v3.md`](v3.md) §8.6 and
|
||||
`docs/website/CLILOCS.md` for how the site gets a table instead (the operator converts one from their
|
||||
own client, once).
|
||||
|
||||
---
|
||||
|
||||
## 8. Corrections to `BRIDGE_FINDINGS.md`
|
||||
@@ -327,6 +336,23 @@ leaderboards. `BridgePoints` is the widest read the bridge performs: ten of Serv
|
||||
keep a row for every character ever created, so it selects the top N in a single bounded pass rather
|
||||
than sorting, and runs on a deliberately slow 300 s interval.
|
||||
|
||||
Also shipped: **`vendor.listing`** ([`v3.md`](v3.md) §8), `BridgeMarket.cs`, the shard-wide
|
||||
player-vendor index. It introduces the one sweep pattern the bridge did not previously have — an
|
||||
**amortized round-robin**. Every other sweep walks its whole collection per tick, which is fine for
|
||||
tens of houses or a fixed set of point systems and is not fine for a world of shops whose inventories
|
||||
recurse into containers. `BridgeMarket` inventories at most `MarketSweepBatch` vendors per tick from
|
||||
a persistent cursor, so the per-tick cost is bounded by the batch rather than by world size, and full
|
||||
coverage takes `ceil(vendors / batch) x MarketSweepSeconds`. Measured at **15.4 ms** for a cold tick
|
||||
of 25 vendors x 40 listings and **0.3 ms** in steady state (the per-vendor diff), on a shard of 209k
|
||||
items / 43k mobiles. It is also the first stream to honour a per-player privacy toggle: ServUO's own
|
||||
`PlayerVendor.VendorSearch` flag, so a shop hidden in game is hidden on the site.
|
||||
|
||||
That completes 3.0's feature work, so the last step is the version itself: `PROTOCOL_VERSION` **2 →
|
||||
3** and the coordinated `edge` → `main` merge across all four repos ([`v3.md`](v3.md) §4 and §4.1).
|
||||
The bump is deliberately the *only* thing that happens at that moment — v3 adds kinds and endpoints
|
||||
but changes nothing that already existed in v2 — so the operator-visible break is limited to
|
||||
re-pinning the version, which the website does for itself in a one-shot boot migration.
|
||||
|
||||
### Config keys (`Config/Bridge.cfg`)
|
||||
|
||||
```ini
|
||||
@@ -342,7 +368,8 @@ Read in `Configure()` via `Config.Get<T>("Bridge.<Key>", default)`. Key scope is
|
||||
|
||||
The set above is the 1.0 sample, not the current one — every later phase added keys (sweep intervals
|
||||
for each board, the town-crier/news caps, the admin write plane, account provisioning, and 3.0's
|
||||
`RulesetEnabled` / `PublicConnectAddress` / `RulesetIncludeSchedule`, and the `Points*` block).
|
||||
`RulesetEnabled` / `PublicConnectAddress` / `RulesetIncludeSchedule`, and the `Points*` and `Market*`
|
||||
blocks).
|
||||
**`servuo-plugins/overlay/Config/Bridge.cfg`
|
||||
is the authoritative, commented list**; `BridgeConfig.cs` holds the defaults.
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ link/
|
||||
│ ├── scripts/
|
||||
│ │ └── gen_tree.py
|
||||
│ ├── workflows/
|
||||
│ │ ├── pr-checks.yml
|
||||
│ │ ├── release.yml
|
||||
│ │ ├── sonarqube.yml
|
||||
│ │ └── sync-project-tree.yml
|
||||
|
||||
170
link/v3.md
170
link/v3.md
@@ -1,6 +1,6 @@
|
||||
# Protocol 3.0 — Shard content, standings & the visibility framework
|
||||
|
||||
**Status:** In progress. All work lands on an `edge` branch in each repo; `edge` → `main` is the v3 cutover.
|
||||
**Status:** Feature-complete on `edge`; the cutover (order 6) is in review. All work lands on an `edge` branch in each repo; `edge` → `main` is the v3 cutover.
|
||||
**Date:** 2026-07-28
|
||||
**Codebase:** ServUO 57.4, `<servuo>`, net48 / x64, Expansion **EJ**.
|
||||
**Companion to** [`PLAN.md`](PLAN.md) (1.0 read/event plane), [`PROTOCOL_2.md`](PROTOCOL_2.md) (2.0 provisioning + world-state streams), [`ADMIN_CONTROLS.md`](ADMIN_CONTROLS.md) (staff write plane), [`INTEGRATION.md`](INTEGRATION.md) (website API).
|
||||
@@ -15,14 +15,19 @@ Each part is marked off here as it lands on `edge`. §9 carries the same state p
|
||||
| 2 | **B/1** — `world.ruleset` (§5) | ✅ **Done** | servuo-plugins [#3](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/3), link [#17](https://gitea.whitlocktech.com/RunicGateway/link/pulls/17), website [#111](https://gitea.whitlocktech.com/RunicGateway/website/pulls/111), docs [#66](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/66) |
|
||||
| 3 | **C** — spawn atlas (§6) | ✅ **Done** | website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112) (parsers + CLI + tables) + [#113](https://gitea.whitlocktech.com/RunicGateway/website/pulls/113) (API + pages + admin panel), docs [#67](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/67) + [#68](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/68) |
|
||||
| 4 | **B/2** — `points.board` (§7) | ✅ **Done** | servuo-plugins [#4](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/4), link [#18](https://gitea.whitlocktech.com/RunicGateway/link/pulls/18), website [#114](https://gitea.whitlocktech.com/RunicGateway/website/pulls/114), docs [#69](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/69) |
|
||||
| 5a | **B/3 dependency** — cliloc table (§8.6) | 🟨 In review | website [#115](https://gitea.whitlocktech.com/RunicGateway/website/pulls/115), docs [#70](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/70) |
|
||||
| 5b | **B/3** — `vendor.listing` (§8) | ⬜ Not started | — |
|
||||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3 (§4) | ⬜ Not started | — |
|
||||
| 5a | **B/3 dependency** — cliloc table (§8.6) | ✅ **Done** | website [#115](https://gitea.whitlocktech.com/RunicGateway/website/pulls/115), docs [#70](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/70) |
|
||||
| 5b | **B/3** — `vendor.listing` (§8) | ✅ **Done** | servuo-plugins [#5](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/5), link [#19](https://gitea.whitlocktech.com/RunicGateway/link/pulls/19), website [#116](https://gitea.whitlocktech.com/RunicGateway/website/pulls/116), docs [#71](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/71) |
|
||||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3 (§4) | 🟨 In review | the bump: link [#20](https://gitea.whitlocktech.com/RunicGateway/link/pulls/20), website [#117](https://gitea.whitlocktech.com/RunicGateway/website/pulls/117), docs [#72](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/72) — then `edge` → `main`: servuo-plugins [#6](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/6), link [#21](https://gitea.whitlocktech.com/RunicGateway/link/pulls/21), website [#118](https://gitea.whitlocktech.com/RunicGateway/website/pulls/118), docs [#73](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/73) |
|
||||
|
||||
Order 5 split in two once §8.6's cliloc dependency turned out to be a client-format problem rather
|
||||
than a parser (see §8.6). 5a is website-only and lands first so the marketplace ships with real item
|
||||
names; 5b is the four-repo wire change.
|
||||
|
||||
**The `edge` → `main` half of order 6 is held for Android parity** (decided 2026-07-30, see §10): the
|
||||
app sees none of the four new features and gates shard nav on session role alone, so merging the
|
||||
cutover first would ship a shard whose app client silently disagrees with the web client about what is
|
||||
public. The **bump** PRs into `edge` are unaffected and merge normally.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why 3.0
|
||||
@@ -242,6 +247,39 @@ admin-set `uo_link_config.protocol` column — so it happens **exactly once**, a
|
||||
from 2 to 3, so the cutover doesn't require a manual admin edit. `UOLINK_PROTOCOL` still overrides.
|
||||
- No feature-negotiation array anywhere — v3 implies all three kinds.
|
||||
|
||||
### 4.1 What the bump actually touches
|
||||
|
||||
The version lives in five places, and all five move together:
|
||||
|
||||
| Where | Change |
|
||||
|---|---|
|
||||
| `link/sidecar/src/main.rs` | `PROTOCOL_VERSION` 2 → 3 (with the v3 note beside the v2 one), plus the sidecar README's worked example |
|
||||
| `website/server/db/schema.sql` | `uo_link_config.protocol` column default 1 → 3, plus the boot migration below |
|
||||
| `website/server/src/model/uoLinkConfig/uoLinkConfig.model.js` | `DEFAULT_PROTOCOL` — what a site with nothing saved yet declares |
|
||||
| `website/server/src/utils/uoLinkClient.js` + `uoLinkSocket.js` | the `config.protocol || …` fallbacks, so an unset value can never quietly send `1` and 409 with a confusing message |
|
||||
| `website/client/.../ShardAdmin.jsx`, `website/.env.example` | the admin form's initial value and the documented env default |
|
||||
|
||||
**The migration has to be one-shot, and that is the only subtle part.** `schema.sql` is re-run on
|
||||
*every* boot (`utils/db.js::ensureSchema`), and every other statement in its migration block is an
|
||||
idempotent `ADD COLUMN IF NOT EXISTS` / `MODIFY`. A bare `UPDATE uo_link_config SET protocol = 3`
|
||||
would not be idempotent in the sense that matters: `protocol` is **admin-editable**, so an operator
|
||||
who deliberately pins an older sidecar in Admin → Shard would silently be un-pinned on the next
|
||||
restart. It is therefore gated on a marker row in `settings`:
|
||||
|
||||
```sql
|
||||
ALTER TABLE uo_link_config MODIFY COLUMN protocol INT NOT NULL DEFAULT 3;
|
||||
UPDATE uo_link_config SET protocol = 3
|
||||
WHERE id = 1 AND protocol < 3
|
||||
AND NOT EXISTS (SELECT 1 FROM settings WHERE `key` = 'uo_link_protocol_3_migrated');
|
||||
INSERT IGNORE INTO settings (`key`, value) VALUES ('uo_link_protocol_3_migrated', '1');
|
||||
```
|
||||
|
||||
The marker is written *after* the `UPDATE`, so the first boot on the new build migrates and every
|
||||
later boot is a no-op. A fresh install has no `uo_link_config` row to update and simply gets the
|
||||
marker plus the new column default. `protocol < 3` rather than `= 2` so an install that never left
|
||||
the old default of `1` is carried across too — it could not have been talking to a v2 sidecar
|
||||
anyway.
|
||||
|
||||
---
|
||||
|
||||
## 5. Part B/1 — `world.ruleset` ✅ Done
|
||||
@@ -312,8 +350,11 @@ Sidecar — `store.rs`: singleton `ruleset(id CHECK(id=1), rev, json, updated_t)
|
||||
`main.rs`: new arm in the board-projection match; `web.rs`: `GET /ruleset` served from the store, so
|
||||
it answers during a shard outage (`PROTOCOL_2.md` §12.2).
|
||||
|
||||
Website — `uoLinkClient.getRuleset()`; `uoLinkSocket.backfill()` (object-shaped, so follow the
|
||||
`getPresence()` block's explicit form, not the array-only `snapshot()` helper); `shardIngest.js` →
|
||||
Website — `uoLinkClient.getRuleset()`; `uoLinkSocket.backfill()` (object-shaped, so it cannot use the
|
||||
array-only `snapshot()` helper — but it **must still go through `shardIngest.ingest()`**, as
|
||||
`ingestEach` does, rather than calling `shardState.setRuleset` directly: the two arrival orders have
|
||||
to produce the same stored frame, and a direct call quietly made backfill a second writer that
|
||||
skipped the normalization below); `shardIngest.js` →
|
||||
`shardState.setRuleset`, **not** in `LOGGED_KINDS` (it re-arrives every reconnect and `server.hello`
|
||||
already marks those); `KIND_FEATURE['world.ruleset'] = 'ruleset'`; `shard_ruleset` singleton table
|
||||
(`rev`, `expansion`, `payload JSON`, `t`); `GET /public/shard/ruleset` behind
|
||||
@@ -322,6 +363,17 @@ already marks those); `KIND_FEATURE['world.ruleset'] = 'ruleset'`; `shard_rulese
|
||||
Client — NEW `routes/public/Rules.jsx` at `/site/rules`, alongside
|
||||
`/site/champs|guilds|governors|houses`; live via `useShardFeed({ filter: new Set(['world.ruleset']) })`.
|
||||
|
||||
**The `shard` field falls back to the instance's own name.** ServUO ships `Server.cfg` with
|
||||
`Name=My Shard`, so an operator who never edited it publishes that verbatim — which is the shard
|
||||
saying *unnamed*, not naming anything, and the rules page then reads "My Shard" under a header
|
||||
carrying the real one. `shardIngest` substitutes `settings.getInstanceName()` (the admin-editable
|
||||
site title, else `BRAND_NAME` — the same resolution `getPublic().brand.name` uses, so one install
|
||||
never shows two names) when `shard` is absent, blank, or exactly the stock default, matched
|
||||
case-insensitively and trim-tolerantly but only as a **whole** value: a shard genuinely called
|
||||
*"My Shard Reborn"* has named itself and keeps it. Applied at **ingest**, not on read, because the
|
||||
ruleset is also broadcast live — the same object goes to the SSE fan-out, so a read-time
|
||||
substitution would be undone by the next reconnect's frame.
|
||||
|
||||
### 5.4 Risk
|
||||
|
||||
Perf is nil (~3 KB per connect). The only real risk is publishing a secret, mitigated by the explicit
|
||||
@@ -600,6 +652,15 @@ Client — NEW `routes/public/Leaderboards.jsx` at `/site/leaderboards`; a "Loya
|
||||
added to `components/CharacterSheet.jsx`, one edit serving both `PlayerCharacter.jsx` and
|
||||
`AdminCharacter.jsx`.
|
||||
|
||||
**An unscored board still renders a row.** Most systems on a young shard have `top: []`, and a page
|
||||
of blank cards reads as broken rather than as new — so a board with no entries shows a single
|
||||
placeholder bearing the **instance's own name** with an em dash where a score goes, above the
|
||||
existing "nobody has earned points here yet" line. It is deliberately **not** shaped like an entry —
|
||||
no rank, no medal, no bar, muted — because a placeholder that looked like a real standing would be a
|
||||
fabricated one; the first real entry replaces it outright. Purely presentational: the API keeps
|
||||
sending an empty `top`, so no consumer ever receives an invented row. Web and app render it the same
|
||||
way (`Leaderboards.jsx`, `LeaderboardsScreen.kt`).
|
||||
|
||||
### 7.5 What the run against a real shard changed
|
||||
|
||||
The plan above was written from reading `PointsSystem.cs`. Booting the actual shard (ServUO 57.4, a
|
||||
@@ -639,7 +700,7 @@ if it is renamed back.
|
||||
|
||||
---
|
||||
|
||||
## 8. Part B/3 — `vendor.listing`
|
||||
## 8. Part B/3 — `vendor.listing` 🟨 In review
|
||||
|
||||
### 8.1 It cannot be an RPC, and this is load-bearing
|
||||
|
||||
@@ -807,6 +868,75 @@ and text paths converge on identical content.
|
||||
driven by `staleAt` (the oldest `shard_vendors.updated_at`). The round-robin sweep means data is
|
||||
inherently up to one full cycle old, and the UI must say so.
|
||||
|
||||
Shipped with a second page, `routes/public/MarketVendor.jsx` at `/site/market/vendors/:serial` —
|
||||
where a search result points. It is the only surface that can render the two states the result list
|
||||
cannot: a `truncated` shop (*"showing 250 of 3,104 — this shop holds more than the shard
|
||||
publishes"*) and a `location` an admin has gated away, which is a real answer rather than an empty
|
||||
coordinate.
|
||||
|
||||
### 8.8 What the build changed
|
||||
|
||||
Four things the implementation settled differently from §8 as written, all of them found by building
|
||||
against the live shard.
|
||||
|
||||
**1. `location` is a nested object, not flat `map`/`x`/`y`/`region`.** §8.1's payload sketch had them
|
||||
flat, and it would have made `market.location` — a rule Part A pre-wired — **inert**, exactly like
|
||||
the `characterName` miss §7.5 records: `projectValue` matches literal JSON keys, so there is no
|
||||
`location` key for the rule to match. Flat keys would have needed five rules that could drift apart.
|
||||
Nesting makes one rule hide the facet, the coordinates, the region and the house together, on the
|
||||
live frame and the stored read model alike, because both now spell it the same way.
|
||||
|
||||
The other pre-wired rule, `market.ownerName`, checked out — it is a real key on the frame. Owner is
|
||||
written as flat `ownerSerial`/`ownerName` rather than through `BridgeJson.Actor`, which would add
|
||||
`acct` and `webId`; same argument `points.board` makes. `ownerSerial` was **added** to the
|
||||
configurable fields alongside `ownerName`, because an admin who hides the owner's name and leaves a
|
||||
serial every other board resolves back to that name has not hidden anything.
|
||||
|
||||
**2. The per-vendor diff signature is the full listing set, not §8.3's `count | Σ(serial ^ price)`.**
|
||||
That hash collides on the single most common change a shop makes: two items swapping prices, which
|
||||
is what re-pricing looks like. The signature is built over the same buffer the frame is written
|
||||
from, in the same order, so a match really does mean an identical frame.
|
||||
|
||||
**3. There is no `payload` column on `shard_vendors`.** §8.5 implied the board pattern (whole frame
|
||||
in JSON, columns hoisted for display). It does not apply here: the items ARE the searchable rows, so
|
||||
they are normalized into `shard_vendor_items` and there is nothing left worth duplicating. The
|
||||
sidecar keeps the whole blob, because outage resilience is its job and search is not.
|
||||
|
||||
**4. Sweep cost is reported, and a slow tick warns.** The batch cap is a *claim* about per-tick cost,
|
||||
and an operator tuning `MarketSweepBatch` was otherwise tuning blind. `[bridge status` now carries
|
||||
`lastMs`/`maxMs`, and a tick over 50 ms prints a rate-limited warning naming the knob.
|
||||
|
||||
Measured on the live shard (27 vendors × 40 listings, 209k items / 43k mobiles):
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| First tick — 25 vendors emitted cold | **15.4 ms** |
|
||||
| Second tick — the remaining 2 | **3.4 ms** |
|
||||
| Steady state — nothing changed | **0.3 ms** |
|
||||
| Website `/market` search over 1,040 listings | 1,040 total, names resolved |
|
||||
| Cliloc re-resolution pass over 1,040 rows | **50 ms** |
|
||||
|
||||
The diff is what makes the steady state ~free; the batch cap is what bounds the cold case. Note the
|
||||
arithmetic the warning exists for: at the default cap of 250 listings, a batch of 25 **full** shops
|
||||
is 6,250 items ≈ 95 ms — over budget. Real shops hold tens, which is why 25 is the default, but a
|
||||
shard of commodity resellers should lower the batch, and now it will be told to.
|
||||
|
||||
Two smaller things worth not rediscovering:
|
||||
|
||||
- **`BridgeJson.Escape` takes a NON-NULL string** — it dereferences `value.Length` immediately — and
|
||||
`BridgeJson.Str` writes its own `,"key":` prefix, so neither serves a value inside a hand-built
|
||||
object. Nearly everything this frame writes is legitimately null (an item's plain `Name` is null
|
||||
for almost every item; a vendor in the street has no house), so that is the common path, not an
|
||||
edge case. `BridgeMarket.Text()` is the two-line writer that was missing.
|
||||
- **The ServUO console writes in the OS code page**, so an em dash in a `Console.WriteLine` renders
|
||||
as `???` in the log an operator would paste into an issue. Bridge console output is ASCII.
|
||||
|
||||
Search-side, one thing the site had to fix rather than inherit: `%` and `_` in a user's query are
|
||||
**LIKE** metacharacters, not SQL ones, so parameterization does not neutralize them — a search for
|
||||
`%` would otherwise match every listing on the shard. `shardMarket.db.js` escapes them. (The atlas's
|
||||
`LIKE` searches predate this and have the same shape over a much smaller table; worth a follow-up,
|
||||
not a blocker here.)
|
||||
|
||||
---
|
||||
|
||||
## 9. Sequencing
|
||||
@@ -817,9 +947,9 @@ inherently up to one full cycle old, and the UI must say so.
|
||||
| 2 | **B/1** — `world.ruleset` (§5) | all four | new kind | ✅ Done |
|
||||
| 3 | **C** — spawn atlas (§6) | website, docs | none | ✅ Done |
|
||||
| 4 | **B/2** — `points.board` (§7) | all four | new kind + `char.profile` field | ✅ Done |
|
||||
| 5a | **B/3 dependency** — cliloc table (§8.6) | website, docs | none | 🟨 In review |
|
||||
| 5b | **B/3** — `vendor.listing` (§8) | all four | new kinds | ⬜ |
|
||||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3, `edge` → `main` | all four | the bump | ⬜ |
|
||||
| 5a | **B/3 dependency** — cliloc table (§8.6) | website, docs | none | ✅ Done |
|
||||
| 5b | **B/3** — `vendor.listing` (§8) | all four | new kinds | ✅ Done |
|
||||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3, `edge` → `main` | all four | the bump | 🟨 In review — `edge` → `main` held for Android parity (§10) |
|
||||
|
||||
---
|
||||
|
||||
@@ -841,8 +971,24 @@ inherently up to one full cycle old, and the UI must say so.
|
||||
- `npm run swagger` **and** `npm run routes:manifest` on every route-touching PR — both are committed
|
||||
artifacts, and `test/routeManifest.test.js` fails on drift.
|
||||
|
||||
**Follow-up, not scoped for 3.0:** the Android app consumes the same public/player shard API and will
|
||||
need `/public/shard/features` to hide its own nav. Track separately against `android-app/`.
|
||||
**Android parity — now scoped, and it gates the cutover (decided 2026-07-30).** This was written as a
|
||||
"track separately" follow-up. It was re-examined before the cutover and the gap is wider than nav
|
||||
hiding: the app consumes the same public/player shard API but has **no consumer for any of the four new
|
||||
features** (`ruleset`, `leaderboards`, `market`, `atlas`), no `points` block on its character sheet, no
|
||||
cliloc-resolved item names (§8.6), and — the part that matters for §3 — **it gates shard navigation on
|
||||
session role alone**, so an admin who disables a feature or raises its audience leaves the app
|
||||
rendering entries that `404`/`403` into a generic error where the web client hides them.
|
||||
|
||||
Two things were verified as already correct and are recorded so they are not re-derived: the app's SSE
|
||||
request rides the same authenticated OkHttp client as every other call, so an app session resolves to
|
||||
the same audience rung as the same account on the web; and every shard DTO in the app is
|
||||
nullable-with-defaults, so field projection strips fields without a deserialization failure.
|
||||
|
||||
Scoped as **M11 in [`../android/PLAN.md`](../android/PLAN.md) §9**, two PRs (the visibility rules +
|
||||
read-model adds, then the four screens). `edge` → `main` is held until both land, so web and app
|
||||
surface the same shard on the same day. Neither PR is coupled to the merge order — on a pre-v3 website
|
||||
every new route and `/public/shard/features` `404`s and the app falls back to today's behavior — so
|
||||
holding the cutover is a schedule decision, not a technical dependency.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -407,6 +407,50 @@ Two values carry non-obvious meanings, both set by the plugin and both documente
|
||||
cliloc rather than a literal. Listing therefore orders by `COALESCE(name, system)`, so boards
|
||||
awaiting cliloc resolution sort by their own key rather than clumping together under NULL.
|
||||
|
||||
### shard_vendors / shard_vendor_items — the player-vendor marketplace (Protocol 3.0)
|
||||
|
||||
The shard-wide shop index, fed by `vendor.listing` / `vendor.listing.remove`. One row per player
|
||||
vendor and one per priced listing. Full operator detail in [`MARKETPLACE.md`](MARKETPLACE.md); the
|
||||
design is `docs/link/v3.md` §8.
|
||||
|
||||
| Table | Shape |
|
||||
|---|---|
|
||||
| `shard_vendors` | `serial` (PK), `shop_name`, `owner_serial`, `owner_name`, `map`/`x`/`y`/`z`, `region`, `house`, `item_count`, `item_total`, `truncated`, `t`, `updated_at`. Indexes on owner, map, region and `updated_at`. |
|
||||
| `shard_vendor_items` | `id` (PK), `vendor_serial`, `serial`, `item_id`, `hue`, `amount`, `price`, `name`, `cliloc`, `display_name`, `child`. Indexes on `vendor_serial`, `price`, `item_id`, `display_name`, and `(display_name, price)`. |
|
||||
|
||||
**Ingest is per-vendor and authoritative**: the frame is the whole shop, so ingest is
|
||||
delete-then-insert of that vendor's listings inside one transaction. All-or-nothing matters
|
||||
specifically because the two writes are "the shop" and "what is in it" — a failure between them
|
||||
leaves a shop advertising an inventory it no longer has, which is visibly wrong and indistinguishable
|
||||
from a genuinely empty shop. No foreign keys, consistent with every other `shard_*` table.
|
||||
|
||||
**There is deliberately no `payload` column**, unlike `shard_points_boards` directly above. The
|
||||
board's top-N is a fixed-size list read whole, so it lives in JSON; here the items *are* the
|
||||
searchable rows, so they are normalized and nothing is left worth duplicating. The sidecar keeps the
|
||||
whole blob — outage resilience is its job, search is ours.
|
||||
|
||||
Market state, not events: neither kind is in `LOGGED_KINDS`, and this is the strongest case of the
|
||||
three v3 kinds. One frame carries up to 250 listings and the sweep re-emits a shop on any price
|
||||
change, so logging would turn `shard_events` into a price history nobody reads.
|
||||
|
||||
Two columns carry non-obvious meanings:
|
||||
|
||||
- **`item_count` vs `item_total`.** `item_count` is what the frame published; `item_total` is what
|
||||
the shop actually holds. They differ when `truncated` — the shard caps listings per frame
|
||||
(`Bridge.MarketMaxListings`, 250 by default), and a commodity reseller with thousands of stacks
|
||||
genuinely exceeds it. Any UI must show both or it presents a partial shop as complete.
|
||||
- **`display_name` is denormalized at ingest**, resolved from the item's literal `name` (preferred —
|
||||
a player set it, so it is more specific) else its `cliloc` against `shard_clilocs`. Resolving at
|
||||
query time would put the cliloc table on the hot path and make search-by-name impossible. Because
|
||||
the shard's diff sweep will not re-send an unchanged shop just because the site learned what its
|
||||
items are called, **a cliloc import triggers a bulk re-resolution** of this column (after a boot
|
||||
import and after an admin import; ~50 ms per thousand rows, never throws).
|
||||
|
||||
`updated_at` is written explicitly on every upsert rather than left to `ON UPDATE CURRENT_TIMESTAMP`,
|
||||
which MariaDB does not fire when every column is written back unchanged. A shop re-published
|
||||
identically is still *freshly confirmed*, and without this the staleness banner would age a perfectly
|
||||
current shop forever.
|
||||
|
||||
### shard_feature_visibility — per-feature audience config (Protocol 3.0)
|
||||
|
||||
One row per shard feature: `feature` (PK), `enabled`, `audience` (a rung on the ladder in §6.5),
|
||||
@@ -745,6 +789,9 @@ from the per-route **siteMode** middleware (§5), never from an auth gate.
|
||||
| GET | `/shard/ruleset` | the shard's own published ruleset (Protocol 3.0 `world.ruleset`): expansion, which optional systems are on, skill/stat caps, account and house limits, champion scroll rules, the save/restart schedule. Served from `shard_ruleset`, so it renders while the shard is down; live via `world.ruleset` on `/shard/stream`. Behind `requireFeature('ruleset')`. **`null`** means the shard has never published one — a real answer, distinct from a published ruleset. `caps.skill` / `caps.totalSkill` are in **tenths** (1000 = 100.0). |
|
||||
| GET | `/shard/points` | every points/loyalty leaderboard the shard publishes (Protocol 3.0 `points.board`) — Queen's Loyalty, Void Pool, the nine city loyalties, Clean Up Britannia, … Served from `shard_points_boards`, so it renders while the shard is down; live via `points.board` on `/shard/stream`. Behind `requireFeature('leaderboards')`, ordered by display name. **`maxPoints: 0` means uncapped** (the common case), and `nameString` is usually `null` with `nameNumber` holding a cliloc — resolve client-side or humanise the `system` key. |
|
||||
| GET | `/shard/points/:system` | one board by the shard's `PointsType` name (e.g. `QueensLoyalty`); `:system` must match `/^[A-Za-z][A-Za-z0-9_]{0,47}$/` or **400** before any query runs. **404** = the shard has never published that system, which is distinct from a published board nobody has scored in yet (**200** with an empty `top`). |
|
||||
| GET | `/shard/market?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=` | search the player-vendor marketplace (Protocol 3.0 `vendor.listing`). Returns **listings**, not vendors — "who sells X and for how much" is the question, and a vendor-shaped result would make every caller flatten the shops back out. Served from `shard_vendors` + `shard_vendor_items`, so it renders while the shard is down. Behind `requireFeature('market')` **and rate-limited** — the first genuinely expensive public read on the site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*` table, reachable with no session). `sort ∈ {price_asc, price_desc, recent}`. `q` matches the resolved display name **or** the item's literal name, with `%`/`_` escaped: they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let `?q=%` match every listing on the shard. Every response repeats `staleAt` (the oldest vendor row) because the shard sweeps round-robin — a banner that ages with the results it labels, not one fetched once. |
|
||||
| GET | `/shard/market/meta` | index size, staleness (`staleAt`/`freshAt`) and which facets and regions actually hold vendors, so a client builds its filters without running a search it will discard. |
|
||||
| GET | `/shard/market/vendors/:serial` | one shop and its listings; `:serial` must match `/^0x[0-9A-Fa-f]{1,16}$/` or **400** before any query runs. **404** = a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer, and distinguishing them would leak that a hidden vendor exists. `truncated` (with `total` exceeding `count`) means the shop holds more than the shard publishes per frame. |
|
||||
| GET | `/shard/features` | the shard features **this caller** may reach plus the audience rung they resolved to (§6.5), so a client hides nav it can't follow. Reports only what the caller can see — the list itself never discloses a gated feature. Consumed by the SPA header and (pending) the Android nav. |
|
||||
| GET | `/atlas/creatures?q=&facet=&limit=&offset=` | the bestiary, most numerous first, with an unpaginated `total`. Static content parsed from the shard's ServUO tree — **not** sidecar-backed, which is why the atlas sits outside `/shard`, and unlike `/shard/*` it **is** site-mode gated. Behind `requireFeature('atlas')`. `?facet=` is matched exactly and never validated against a list (no facet name exists in the code); the filter is an `EXISTS` over the points rather than a JSON path or `JSON_SEARCH` built from caller input, whose `%`/`_` wildcards would make `?facet=%` match everything. |
|
||||
| GET | `/atlas/creatures/:slug` | one creature: `places` (the point-in-rect aggregate — "lizardman → Shrines, Isamu-Jima, Yew"), `spawners` (the bounded raw list, with `spawnersTruncated`), `alsoHere`. **`points` is a COUNT and `spawners` is the LIST** — named apart so one key never means a number on one route and an array on another. `minDelay`/`maxDelay` are in **seconds**, normalised at parse time from the source's per-record minutes-or-seconds. 404 = no such creature in this atlas. |
|
||||
@@ -835,7 +882,11 @@ who"; `activity_log` provides the history feed.
|
||||
- **Cookie**: `httpOnly`, `sameSite=Lax`, `path=/`, and **`secure` decided per-request** (`COOKIE_SECURE=auto` → `secure: req.secure`).
|
||||
- **Trusted-device MFA.** A second, separate httpOnly cookie (`rg_trust`, default 30d) — opaque, sha256-hashed server-side in `trusted_devices` — lets a browser/app **skip the TOTP step** (never the password) on future logins. It is a server-side, per-row-revocable record (never a JWT claim), so the stateless session JWT is unchanged and trust stays revocable. It only ever gates the **second factor**; it deliberately outlives logout, and is cleared on untrust / password change / password reset / TOTP disable. **Recovery codes** (bcrypt, single-use) are the 2FA-lockout fallback. All admin trusted-device/MFA actions and the self actions (`auth.login.trusted_device`, `account.trusted_device.*`, `account.recovery_code*`, `admin.trusted_device.*`, `admin.user.totp.reset`) are audit-logged. See `docs/website/TRUSTED_DEVICES_MFA.md`. This is the key to dual access: the cookie is `Secure` when reached through Pangolin (HTTPS, `X-Forwarded-Proto: https`) but **not** `Secure` when reached directly over the LAN IP on plain HTTP — so login works in both. `COOKIE_SECURE=true|false` can force it. Requires `trust proxy` (below). `localhost:5173` (Vite) and `localhost:3000` are same-site, so the cookie flows in dev too.
|
||||
- **bcrypt** hashing (cost 10+); plaintext passwords never stored, logged, or returned.
|
||||
- **Rate limiting** (`express-rate-limit`) on `/auth/login` and `/public/contact`.
|
||||
- **Rate limiting** (`express-rate-limit`) on `/auth/login`, `/public/contact`, and — the only limited
|
||||
*read* — `/public/shard/market` and `/public/shard/market/vendors/:serial` (60/min/IP). Every other
|
||||
public read is an indexed lookup of bounded size; the marketplace search is a `LIKE` scan plus a
|
||||
`COUNT` over the largest `shard_*` table, anonymous by default, so it is the one public GET that is
|
||||
worth money to serve.
|
||||
- **Validation** (`express-validator`) on all writes; centralized error handler.
|
||||
- **helmet** with a Content-Security-Policy tuned for the built React SPA. The policies now live in
|
||||
**`server/src/config/csp.js`** (`app.js` only wires them up):
|
||||
|
||||
@@ -44,6 +44,11 @@ they did before the table existed.
|
||||
|
||||
## Converting
|
||||
|
||||
> **Step-by-step operator instructions — where to get UOFiddler, where your
|
||||
> client files are, and how to verify the import — are in
|
||||
> [`UOFIDDLER.md`](UOFIDDLER.md).** This section covers the formats and the
|
||||
> reasoning behind them.
|
||||
|
||||
Either format below is accepted; the site sniffs which one it was handed.
|
||||
|
||||
| Format | Fidelity | Notes |
|
||||
@@ -77,8 +82,16 @@ dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/cli
|
||||
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv
|
||||
```
|
||||
|
||||
A UOFiddler GUI export works equally well — anything producing one of the two
|
||||
shapes above is fine.
|
||||
A UOFiddler GUI export works too, but **not unmodified**: its Cliloc tab writes
|
||||
`Number;Text;Flag` — three columns, the flag *last* — and the parser reads
|
||||
`number<separator>text`, so the trailing field is absorbed into the name and
|
||||
every item renders as `quarter staff;0`. Stripping it is one `sed`, given in
|
||||
[`UOFIDDLER.md`](UOFIDDLER.md) §Route B.
|
||||
|
||||
The parser already tolerates `number,flag,text`, with the flag in the *middle*.
|
||||
It is not extended to cover the trailing form because a final `;0` is
|
||||
indistinguishable from a name that genuinely ends that way — a heuristic there
|
||||
would corrupt real names to save the operator one command.
|
||||
|
||||
## Shard-added and shard-edited items
|
||||
|
||||
|
||||
167
website/MARKETPLACE.md
Normal file
167
website/MARKETPLACE.md
Normal file
@@ -0,0 +1,167 @@
|
||||
# Marketplace — the player-vendor index
|
||||
|
||||
**Status:** On `edge` — servuo-plugins [#5](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/5), link [#19](https://gitea.whitlocktech.com/RunicGateway/link/pulls/19), website [#116](https://gitea.whitlocktech.com/RunicGateway/website/pulls/116).
|
||||
**Design:** [`docs/link/v3.md` §8](../link/v3.md) — Protocol 3.0 Part B/3.
|
||||
**Depends on:** [`CLILOCS.md`](CLILOCS.md) — without a cliloc table, listings render as item ids.
|
||||
|
||||
The marketplace is a searchable index of every player vendor on the shard: what
|
||||
each shop is selling, for how much, and where it is standing. It is the same set
|
||||
the in-game **Vendor Search** gump reads, offered from outside the game — so a
|
||||
player can find the vanquishing kryss they want before logging in, and someone
|
||||
who does not play at all can see that the economy exists.
|
||||
|
||||
Page: `/site/market`, plus `/site/market/vendors/:serial` for one shop.
|
||||
|
||||
## Three things the pages must say out loud
|
||||
|
||||
Everything below follows from how the data is gathered, and each has a visible
|
||||
consequence the UI is required to surface.
|
||||
|
||||
**1. The prices are not live.** The shard sweeps vendors **round-robin** — at
|
||||
most `Bridge.MarketSweepBatch` shops per tick — so a given shop can be a full
|
||||
cycle behind. The page carries a *"prices last refreshed N minutes ago"* banner
|
||||
driven by the **oldest** vendor row, not the newest: the one stale shop is the
|
||||
one that wastes somebody's trip.
|
||||
|
||||
**2. A shop can be truncated.** `Bridge.MarketMaxListings` (250 by default) caps
|
||||
how many listings one frame carries. A commodity reseller with thousands of
|
||||
stacked resources is a real thing, and an uncapped frame for one is measured in
|
||||
megabytes. Over the cap the shop reports `truncated`, and the vendor page says
|
||||
*"showing 250 of 3,104 — this shop holds more than the shard publishes"* rather
|
||||
than presenting a partial shop as complete.
|
||||
|
||||
**3. An item may have no name.** Items on the wire carry a cliloc id, not a name.
|
||||
On a shard whose operator has not converted a cliloc table
|
||||
([`CLILOCS.md`](CLILOCS.md)) the honest render is the item id — never an invented
|
||||
label, which would be indistinguishable from a real one.
|
||||
|
||||
## Privacy: the player's own toggle wins
|
||||
|
||||
Only vendors whose owner left the in-game **Vendor Search** flag ON are ever sent
|
||||
to the site. A player who hides their shop in game is hidden here too, and no
|
||||
admin setting overrides that. When they hide one that was already indexed, the
|
||||
shard emits `vendor.listing.remove` and the row is deleted — so revoking consent
|
||||
takes effect, it does not merely stop refreshing.
|
||||
|
||||
Shop name, owner character name and location default to **Everyone**, because the
|
||||
stock Vendor Search gump already shows exactly that set to any player in game.
|
||||
They remain admin-configurable; see [`SHARD_VISIBILITY.md`](SHARD_VISIBILITY.md).
|
||||
Account names and website user ids never cross the wire at all.
|
||||
|
||||
## How it is put together
|
||||
|
||||
```
|
||||
ServUO uo-link sidecar website
|
||||
────── ─────────────── ───────
|
||||
BridgeMarket.cs vendors table shard_vendors
|
||||
round-robin sweep ──────► (whole frame blob) ──────► shard_vendor_items
|
||||
per-vendor diff GET /market (paged) + display_name
|
||||
vendor.listing resolved at ingest
|
||||
vendor.listing.remove
|
||||
```
|
||||
|
||||
**The shard side** walks at most `MarketSweepBatch` vendors per tick from a
|
||||
persistent cursor, diffs each against what it last published, and emits a whole
|
||||
frame for any shop that moved. Per-tick cost is therefore bounded by the batch,
|
||||
not by how many vendors the world holds — full coverage takes
|
||||
`ceil(vendors / batch) × MarketSweepSeconds`.
|
||||
|
||||
**The sidecar** stores each frame whole and serves `GET /market`, its only paged
|
||||
read. It normalizes nothing and defines no audiences: it is a dumb forwarder, and
|
||||
search is the website's job.
|
||||
|
||||
**The website** splits each frame into a vendor row and its listings, replacing
|
||||
that vendor's whole listing set inside one transaction (the frame is
|
||||
authoritative for that vendor, never a delta). Item names are resolved against
|
||||
the cliloc table **on the way in** and stored denormalized, which is what makes
|
||||
search-by-name possible and keeps the cliloc table off the hot path.
|
||||
|
||||
## Operating it
|
||||
|
||||
Everything is in `Config/Bridge.cfg` on the shard. There is nothing to configure
|
||||
on the website.
|
||||
|
||||
| Setting | Default | What it does |
|
||||
|---|---|---|
|
||||
| `MarketEnabled` | `true` | Master switch. Off publishes nothing; the page shows an empty index. |
|
||||
| `MarketSweepSeconds` | `60` | Tick interval. |
|
||||
| `MarketSweepBatch` | `25` | Vendors inventoried per tick. Clamped 1..500. |
|
||||
| `MarketMaxListings` | `250` | Per-shop listing cap, after which `truncated`. Clamped 1..5000. |
|
||||
|
||||
**Faster coverage vs. per-tick cost.** Lowering `MarketSweepSeconds` or raising
|
||||
`MarketSweepBatch` both refresh the index sooner and both cost more per tick.
|
||||
The expensive part is the item walk, which recurses into every container a vendor
|
||||
is selling — so a shard of big shops should raise the interval rather than the
|
||||
batch.
|
||||
|
||||
`[bridge status` reports the sweep, including `lastMs` and `maxMs`:
|
||||
|
||||
```
|
||||
market(enabled=True sweeps=42 scanned=108 emitted=27 removed=0 skipped=0
|
||||
truncated=0 tracked=27 vendors=27 cursor=2 batch=25 lastMs=0.31 maxMs=15.40)
|
||||
```
|
||||
|
||||
A tick over **50 ms** prints a rate-limited warning naming the knob:
|
||||
|
||||
```
|
||||
[Bridge] market sweep took 82.4 ms (budget 50 ms) - lower Bridge.MarketSweepBatch (now 25) if this persists
|
||||
```
|
||||
|
||||
Measured on a shard with 27 vendors × 40 listings (209k items, 43k mobiles):
|
||||
**15.4 ms** for the first cold tick of 25 vendors, **0.3 ms** in steady state —
|
||||
the diff is what makes an unchanged world nearly free. Note the arithmetic: 25
|
||||
*full* shops at the 250-listing cap is 6,250 items ≈ 95 ms, over budget. Real
|
||||
shops hold tens, which is why 25 is the default and why the warning exists.
|
||||
|
||||
`[bridge sweepnow` runs one tick immediately; `[bridge reload` re-reads the
|
||||
settings above without a restart.
|
||||
|
||||
## Names arriving late
|
||||
|
||||
Item names come from the cliloc table, and the market sweep will **not** re-send
|
||||
an unchanged shop just because the site learned what its items are called. So a
|
||||
cliloc import triggers a bulk re-resolution of every stored listing — otherwise
|
||||
an operator who configures clilocs after the first sweep would see item ids until
|
||||
every shop happened to change on its own. It runs after a boot import and after
|
||||
an admin import, takes ~50 ms per thousand listings, and never throws: a failure
|
||||
leaves names exactly as they were.
|
||||
|
||||
## API
|
||||
|
||||
All under `/api/v1/public/shard`, gated by the `market` feature and
|
||||
**rate-limited** — these are the first genuinely expensive public reads on the
|
||||
site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*`
|
||||
table, reachable with no session).
|
||||
|
||||
| Route | What |
|
||||
|---|---|
|
||||
| `GET /market` | Search. Returns **listings**, not vendors — "who sells X and for how much" is the question. `?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=`, `sort ∈ {price_asc, price_desc, recent}`. |
|
||||
| `GET /market/meta` | Index size, staleness, and which facets and regions actually hold vendors — so a client builds its filters without running a search it will discard. |
|
||||
| `GET /market/vendors/:serial` | One shop and its listings. **404** for a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer. |
|
||||
|
||||
`q` matches the resolved display name **or** the item's own literal name, because
|
||||
an item with a player-set name (most of what is worth searching for on a
|
||||
player-run shard) may carry a generic cliloc. `%` and `_` in a query are escaped:
|
||||
they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let
|
||||
a search for `%` match every listing on the shard.
|
||||
|
||||
Full schemas are in the OpenAPI spec (`ShardMarketPage`, `ShardMarketVendor`,
|
||||
`ShardMarketMeta`, `ShardMarketListing`, `ShardMarketLocation`).
|
||||
|
||||
## Tables
|
||||
|
||||
`shard_vendors` (one row per shop) and `shard_vendor_items` (one row per priced
|
||||
listing). Both are ingest-owned; nothing else writes to them. No foreign keys,
|
||||
in keeping with every other `shard_*` table — the ingest transaction is what
|
||||
keeps them consistent, and an FK would turn a malformed frame into a failed write
|
||||
rather than a dropped row.
|
||||
|
||||
There is deliberately **no `payload` column** on `shard_vendors`, unlike the
|
||||
points board next door. The board's top-N is a fixed-size list read whole, so it
|
||||
lives in JSON; here the items *are* the searchable rows, so they are normalized
|
||||
and there is nothing left worth duplicating. The sidecar keeps the whole blob,
|
||||
because outage resilience is its job.
|
||||
|
||||
`shard_vendor_items.display_name` is denormalized and indexed (alone, and
|
||||
composite with `price` for "cheapest matching X"). See "Names arriving late"
|
||||
above for how it is kept current.
|
||||
@@ -164,6 +164,7 @@ website/
|
||||
│ │ │ ├── heroLayout.js
|
||||
│ │ │ ├── shardEvents.js
|
||||
│ │ │ ├── useAsync.js
|
||||
│ │ │ ├── useShardFeatures.js
|
||||
│ │ │ └── useShardFeed.js
|
||||
│ │ ├── routes/
|
||||
│ │ │ ├── admin/
|
||||
@@ -190,6 +191,8 @@ website/
|
||||
│ │ │ │ │ ├── SettingsAdmin.jsx
|
||||
│ │ │ │ │ ├── ShardAdmin.jsx
|
||||
│ │ │ │ │ ├── ShardOps.jsx
|
||||
│ │ │ │ │ ├── ShardVisibility.jsx
|
||||
│ │ │ │ │ ├── SpawnAtlas.jsx
|
||||
│ │ │ │ │ ├── UserDetail.jsx
|
||||
│ │ │ │ │ ├── UserEditor.jsx
|
||||
│ │ │ │ │ ├── UsersAdmin.jsx
|
||||
@@ -213,17 +216,23 @@ website/
|
||||
│ │ │ │ └── ResetPassword.jsx
|
||||
│ │ │ ├── public/
|
||||
│ │ │ │ ├── About.jsx
|
||||
│ │ │ │ ├── Atlas.jsx
|
||||
│ │ │ │ ├── AtlasCreature.jsx
|
||||
│ │ │ │ ├── ChampSpawns.jsx
|
||||
│ │ │ │ ├── CmsPage.jsx
|
||||
│ │ │ │ ├── FiveOnFriday.jsx
|
||||
│ │ │ │ ├── Governors.jsx
|
||||
│ │ │ │ ├── Guilds.jsx
|
||||
│ │ │ │ ├── Houses.jsx
|
||||
│ │ │ │ ├── Leaderboards.jsx
|
||||
│ │ │ │ ├── Maintenance.jsx
|
||||
│ │ │ │ ├── Market.jsx
|
||||
│ │ │ │ ├── MarketVendor.jsx
|
||||
│ │ │ │ ├── News.jsx
|
||||
│ │ │ │ ├── Newsletter.jsx
|
||||
│ │ │ │ ├── NewsletterIssue.jsx
|
||||
│ │ │ │ ├── Portal.jsx
|
||||
│ │ │ │ ├── Rules.jsx
|
||||
│ │ │ │ ├── Screenshots.jsx
|
||||
│ │ │ │ ├── Shard.jsx
|
||||
│ │ │ │ ├── ShardActivity.jsx
|
||||
@@ -257,9 +266,12 @@ website/
|
||||
│ └── sonar-test-reporter.mjs
|
||||
├── server/
|
||||
│ ├── db/
|
||||
│ │ ├── data/
|
||||
│ │ │ └── spawnAtlas.art.example.json
|
||||
│ │ ├── schema.sql
|
||||
│ │ └── seed.js
|
||||
│ ├── scripts/
|
||||
│ │ ├── importSpawnAtlas.js
|
||||
│ │ └── routeManifest.js
|
||||
│ ├── src/
|
||||
│ │ ├── auth/
|
||||
@@ -365,15 +377,27 @@ website/
|
||||
│ │ │ ├── settings/
|
||||
│ │ │ │ ├── settings.db.js
|
||||
│ │ │ │ └── settings.model.js
|
||||
│ │ │ ├── shardAtlas/
|
||||
│ │ │ │ ├── shardAtlas.db.js
|
||||
│ │ │ │ └── shardAtlas.model.js
|
||||
│ │ │ ├── shardClilocs/
|
||||
│ │ │ │ ├── shardClilocs.db.js
|
||||
│ │ │ │ └── shardClilocs.model.js
|
||||
│ │ │ ├── shardEvents/
|
||||
│ │ │ │ ├── shardEvents.db.js
|
||||
│ │ │ │ └── shardEvents.model.js
|
||||
│ │ │ ├── shardLinks/
|
||||
│ │ │ │ ├── shardLinks.db.js
|
||||
│ │ │ │ └── shardLinks.model.js
|
||||
│ │ │ ├── shardMarket/
|
||||
│ │ │ │ ├── shardMarket.db.js
|
||||
│ │ │ │ └── shardMarket.model.js
|
||||
│ │ │ ├── shardState/
|
||||
│ │ │ │ ├── shardState.db.js
|
||||
│ │ │ │ └── shardState.model.js
|
||||
│ │ │ ├── shardVisibility/
|
||||
│ │ │ │ ├── shardVisibility.db.js
|
||||
│ │ │ │ └── shardVisibility.model.js
|
||||
│ │ │ ├── trustedDevices/
|
||||
│ │ │ │ ├── trustedDevices.db.js
|
||||
│ │ │ │ └── trustedDevices.model.js
|
||||
@@ -398,12 +422,14 @@ website/
|
||||
│ │ │ │ │ ├── account.router.js
|
||||
│ │ │ │ │ ├── activity.router.js
|
||||
│ │ │ │ │ ├── admin.controller.js
|
||||
│ │ │ │ │ ├── admin.routes.js
|
||||
│ │ │ │ │ ├── authProviders.controller.js
|
||||
│ │ │ │ │ ├── authProviders.router.js
|
||||
│ │ │ │ │ ├── botActivity.controller.js
|
||||
│ │ │ │ │ ├── botActivity.router.js
|
||||
│ │ │ │ │ ├── dashboard.router.js
|
||||
│ │ │ │ │ ├── discordBot.controller.js
|
||||
│ │ │ │ │ ├── discordBot.router.js
|
||||
│ │ │ │ │ ├── email.router.js
|
||||
│ │ │ │ │ ├── emailConfig.controller.js
|
||||
│ │ │ │ │ ├── imageUpload.js
|
||||
│ │ │ │ │ ├── index.js
|
||||
@@ -414,16 +440,25 @@ website/
|
||||
│ │ │ │ │ ├── pages.controller.js
|
||||
│ │ │ │ │ ├── pages.router.js
|
||||
│ │ │ │ │ ├── posts.router.js
|
||||
│ │ │ │ │ ├── settings.router.js
|
||||
│ │ │ │ │ ├── shard.router.js
|
||||
│ │ │ │ │ ├── shardAtlas.controller.js
|
||||
│ │ │ │ │ ├── shardClilocs.controller.js
|
||||
│ │ │ │ │ ├── shardOps.controller.js
|
||||
│ │ │ │ │ ├── shardVisibility.controller.js
|
||||
│ │ │ │ │ ├── uoLink.controller.js
|
||||
│ │ │ │ │ ├── uoLink.router.js
|
||||
│ │ │ │ │ ├── uploads.router.js
|
||||
│ │ │ │ │ ├── users.router.js
|
||||
│ │ │ │ │ ├── usersShard.controller.js
|
||||
│ │ │ │ │ └── wiki.router.js
|
||||
│ │ │ │ ├── auth/
|
||||
│ │ │ │ │ ├── auth.controller.js
|
||||
│ │ │ │ │ ├── auth.routes.js
|
||||
│ │ │ │ │ ├── index.js
|
||||
│ │ │ │ │ ├── invite.controller.js
|
||||
│ │ │ │ │ ├── invite.router.js
|
||||
│ │ │ │ │ ├── login.router.js
|
||||
│ │ │ │ │ ├── loginGuards.js
|
||||
│ │ │ │ │ ├── me.routes.js
|
||||
│ │ │ │ │ ├── mobile.controller.js
|
||||
│ │ │ │ │ ├── mobile.routes.js
|
||||
@@ -431,7 +466,10 @@ website/
|
||||
│ │ │ │ │ ├── mobileSso.routes.js
|
||||
│ │ │ │ │ ├── notifications.controller.js
|
||||
│ │ │ │ │ ├── notifications.routes.js
|
||||
│ │ │ │ │ ├── password.router.js
|
||||
│ │ │ │ │ ├── passwordReset.controller.js
|
||||
│ │ │ │ │ ├── register.router.js
|
||||
│ │ │ │ │ ├── session.router.js
|
||||
│ │ │ │ │ ├── sso.controller.js
|
||||
│ │ │ │ │ ├── sso.routes.js
|
||||
│ │ │ │ │ └── trustDevice.helper.js
|
||||
@@ -439,13 +477,23 @@ website/
|
||||
│ │ │ │ │ ├── internal.controller.js
|
||||
│ │ │ │ │ └── internal.routes.js
|
||||
│ │ │ │ ├── player/
|
||||
│ │ │ │ │ ├── account.router.js
|
||||
│ │ │ │ │ ├── appeals.controller.js
|
||||
│ │ │ │ │ ├── player.routes.js
|
||||
│ │ │ │ │ └── shard.controller.js
|
||||
│ │ │ │ │ ├── appeals.router.js
|
||||
│ │ │ │ │ ├── index.js
|
||||
│ │ │ │ │ ├── shard.controller.js
|
||||
│ │ │ │ │ └── shard.router.js
|
||||
│ │ │ │ ├── public/
|
||||
│ │ │ │ │ ├── atlas.controller.js
|
||||
│ │ │ │ │ ├── atlas.router.js
|
||||
│ │ │ │ │ ├── index.js
|
||||
│ │ │ │ │ ├── pages.router.js
|
||||
│ │ │ │ │ ├── posts.router.js
|
||||
│ │ │ │ │ ├── public.controller.js
|
||||
│ │ │ │ │ ├── public.routes.js
|
||||
│ │ │ │ │ └── shard.controller.js
|
||||
│ │ │ │ │ ├── shard.controller.js
|
||||
│ │ │ │ │ ├── shard.router.js
|
||||
│ │ │ │ │ ├── site.router.js
|
||||
│ │ │ │ │ └── wiki.router.js
|
||||
│ │ │ │ └── v1.router.js
|
||||
│ │ │ ├── api.router.js
|
||||
│ │ │ ├── cspReport.controller.js
|
||||
@@ -455,6 +503,8 @@ website/
|
||||
│ │ │ ├── auth.js
|
||||
│ │ │ ├── botInternalClient.js
|
||||
│ │ │ ├── botInternalKey.js
|
||||
│ │ │ ├── clilocParse.js
|
||||
│ │ │ ├── clilocSource.js
|
||||
│ │ │ ├── db.js
|
||||
│ │ │ ├── logger.js
|
||||
│ │ │ ├── mailer.js
|
||||
@@ -465,6 +515,9 @@ website/
|
||||
│ │ │ ├── shardBroadcast.js
|
||||
│ │ │ ├── shardIngest.js
|
||||
│ │ │ ├── shardSales.js
|
||||
│ │ │ ├── shardVisibility.js
|
||||
│ │ │ ├── spawnAtlasParse.js
|
||||
│ │ │ ├── spawnAtlasSource.js
|
||||
│ │ │ ├── totp.js
|
||||
│ │ │ ├── trustProxy.js
|
||||
│ │ │ ├── uoLinkClient.js
|
||||
@@ -483,11 +536,14 @@ website/
|
||||
│ │ ├── appeals.pure.test.js
|
||||
│ │ ├── appeals.test.js
|
||||
│ │ ├── appLinks.test.js
|
||||
│ │ ├── atlasController.test.js
|
||||
│ │ ├── authController.test.js
|
||||
│ │ ├── authMe.test.js
|
||||
│ │ ├── authTrustedDevice.test.js
|
||||
│ │ ├── botInternalKey.test.js
|
||||
│ │ ├── botScore.test.js
|
||||
│ │ ├── clilocParse.test.js
|
||||
│ │ ├── clilocSource.test.js
|
||||
│ │ ├── csp.test.js
|
||||
│ │ ├── emailConfig.model.test.js
|
||||
│ │ ├── honeypot.test.js
|
||||
@@ -521,17 +577,32 @@ website/
|
||||
│ │ ├── secretBox.test.js
|
||||
│ │ ├── selfTrustedDevices.test.js
|
||||
│ │ ├── session.test.js
|
||||
│ │ ├── shardBroadcast.visibility.test.js
|
||||
│ │ ├── shardControllerPublic.test.js
|
||||
│ │ ├── shardIngest.champsPages.test.js
|
||||
│ │ ├── shardIngest.market.test.js
|
||||
│ │ ├── shardIngest.points.test.js
|
||||
│ │ ├── shardIngest.protocol2.test.js
|
||||
│ │ ├── shardIngest.ruleset.test.js
|
||||
│ │ ├── shardMarket.model.test.js
|
||||
│ │ ├── shardState.governorTerms.test.js
|
||||
│ │ ├── shardState.model.test.js
|
||||
│ │ ├── shardVisibility.test.js
|
||||
│ │ ├── spawnAtlas.parse.test.js
|
||||
│ │ ├── spawnAtlas.source.test.js
|
||||
│ │ ├── ssoCallback.test.js
|
||||
│ │ ├── ssoState.test.js
|
||||
│ │ ├── ssoTrustedDevice.test.js
|
||||
│ │ ├── totp.test.js
|
||||
│ │ ├── trustedDevices.test.js
|
||||
│ │ ├── trustProxy.test.js
|
||||
│ │ ├── uoLinkClient.test.js
|
||||
│ │ └── usernamePolicy.test.js
|
||||
│ ├── tools/
|
||||
│ │ └── cliloc-export/
|
||||
│ │ ├── clilocexport.csproj
|
||||
│ │ ├── Program.cs
|
||||
│ │ └── README.md
|
||||
│ ├── .env.example
|
||||
│ ├── package-lock.json
|
||||
│ ├── package.json
|
||||
|
||||
@@ -60,12 +60,32 @@ board while holding back one column. See the table in §3.
|
||||
| **Shard rules** | Skill/stat caps, house limits, vet rewards, the ruleset | Everyone | Connect address → Everyone |
|
||||
| **Spawn atlas** | Bestiary and spawn locations (static content) | Everyone | — |
|
||||
| **Leaderboards** | Point and loyalty standings | Everyone | Character names → Everyone |
|
||||
| **Marketplace** | The shard-wide player-vendor index | Everyone, **live updates off** | Vendor owner name → Everyone · Location → Everyone |
|
||||
| **Marketplace** | The shard-wide player-vendor index | Everyone, **live updates off** | Vendor owner name → Everyone · Vendor owner character id → Everyone · In-game location → Everyone |
|
||||
|
||||
**Why the marketplace ships with live updates off.** A live feed of every vendor's full inventory
|
||||
would be the single largest thing the site sends. No page needs it — the marketplace is a search over
|
||||
stored data with a “prices last refreshed N minutes ago” stamp. Turn it on only if you want it.
|
||||
|
||||
**Why the marketplace's fields default to Everyone.** A vendor's shop name, its owner's character
|
||||
name and where it is standing are *already* visible to every player in game: the stock Vendor Search
|
||||
gump surfaces exactly that set to anyone who opens it. Publishing them on the site is not a new
|
||||
disclosure. They stay configurable because a shard may still prefer to keep its economy behind a
|
||||
login — and because "already public in game" is a judgement about your shard, not ours.
|
||||
|
||||
**Location is one setting covering four things.** Hiding it removes the facet, the coordinates, the
|
||||
region *and* the house name together. That is deliberate: those are four ways of saying the same
|
||||
thing, and a setting that hid the coordinates while publishing the house name would not have hidden
|
||||
anything.
|
||||
|
||||
**Hiding the owner name also hides the owner character id.** They are separate settings so you can
|
||||
be explicit, but leaving the id published while hiding the name achieves nothing — the leaderboards
|
||||
and guild boards resolve that same id back to a character name. Set both.
|
||||
|
||||
**What hiding a vendor cannot do.** Only vendors whose owner left the in-game *Vendor Search* flag ON
|
||||
are ever sent to the site, so a player who hides their shop in game is hidden here too — and no
|
||||
setting on this page can override that. It works the other way as well: these settings control who
|
||||
sees the index, not whether players can find each other's shops in game.
|
||||
|
||||
**Why house owner/price default to Staff.** The public Houses page has always been a "where are the
|
||||
falling houses" board — location only. Owner and price are the staff view. That split is preserved.
|
||||
|
||||
|
||||
@@ -223,7 +223,8 @@ The atlas is fully functional as text. `shard_spawn_creatures.art` is nullable
|
||||
and is NULL on every fresh import; pages render without images, which is the
|
||||
normal and supported state, not a degraded one.
|
||||
|
||||
An operator who wants art:
|
||||
An operator who wants art — step-by-step, with the UOFiddler side spelled out, in
|
||||
[`UOFIDDLER.md`](UOFIDDLER.md) §Part 2:
|
||||
|
||||
1. Extracts it from **their own** client files (UOFiddler, ClassicUO tooling, or
|
||||
any art extractor).
|
||||
|
||||
282
website/UOFIDDLER.md
Normal file
282
website/UOFIDDLER.md
Normal file
@@ -0,0 +1,282 @@
|
||||
# Extracting from your own UO client (UOFiddler)
|
||||
|
||||
**Audience:** the shard operator, once, at setup time.
|
||||
**Related:** [`CLILOCS.md`](CLILOCS.md) (why the cliloc conversion is unavoidable),
|
||||
[`SPAWN_ATLAS.md`](SPAWN_ATLAS.md) (where creature art fits).
|
||||
|
||||
Two features read data that **only exists inside a UO client**, and a UO client's
|
||||
files are EA's, not ours to redistribute. So neither this repo nor any image we
|
||||
publish can ship them — the operator extracts from **their own** client, once,
|
||||
and points the site at the result.
|
||||
|
||||
| Feature | What it needs | Required? | Without it |
|
||||
|---|---|---|---|
|
||||
| **Item / title names** ([`CLILOCS.md`](CLILOCS.md)) | `Cliloc.enu`, converted | No | Names render as raw ids — `id 1023721` instead of *quarter staff* |
|
||||
| **Creature art** ([`SPAWN_ATLAS.md`](SPAWN_ATLAS.md)) | Sprites from `.mul`/`.uop` | No | Atlas pages render as text, which is the normal state |
|
||||
|
||||
**Both are optional and neither is load-bearing.** A shard that never does any of
|
||||
this is fully supported. Do part one and skip part two if art is not worth your
|
||||
time — they share only the tool.
|
||||
|
||||
Everything you extract stays **outside the repository**: the converted cliloc
|
||||
file lives at a path you choose, and `spawnAtlas.art.json` plus `server/uploads/`
|
||||
are gitignored, so none of it can be committed by accident.
|
||||
|
||||
---
|
||||
|
||||
## Part 0 — Get UOFiddler
|
||||
|
||||
[UOFiddler](https://github.com/polserver/UOFiddler) is the community client-file
|
||||
editor. We use it because its `Ultima.dll` already contains the cliloc
|
||||
decompressor, maintained by people who do this for a living.
|
||||
|
||||
1. Download the latest release zip from
|
||||
<https://github.com/polserver/UOFiddler/releases/latest> — one asset, named
|
||||
`UOFiddler-<version>.zip` (4.22.2 is ~2 MB).
|
||||
2. Extract it. The zip contains a single top-level folder, and the two files that
|
||||
matter are at **its root**:
|
||||
|
||||
```
|
||||
UOFiddler-4.22.2/
|
||||
Ultima.dll ← the decompressor (Part 1 needs this path)
|
||||
UoFiddler.exe ← the GUI (Part 2 needs this)
|
||||
plugins/
|
||||
…
|
||||
```
|
||||
|
||||
3. **Runtime:** UOFiddler 4.22.2 is built for **.NET 10**. Running `UoFiddler.exe`
|
||||
needs the .NET 10 **Desktop** Runtime (Windows only); loading `Ultima.dll` from
|
||||
the converter in Part 1 needs the .NET 10 runtime. Install from
|
||||
<https://dotnet.microsoft.com/download/dotnet/10.0>.
|
||||
|
||||
### Finding your client files
|
||||
|
||||
The cliloc file is in your **UO client installation directory**, not in your
|
||||
ServUO tree — the shard server has no copy of it. Look for `Cliloc.enu` (English;
|
||||
the other seven are `chs`, `cht`, `deu`, `esp`, `fra`, `jpn`, `kor`) beside
|
||||
`art.mul` / `artLegacyMUL.uop`. The EA Classic Client's default location is:
|
||||
|
||||
```
|
||||
C:\Program Files (x86)\Electronic Arts\Ultima Online Classic\
|
||||
```
|
||||
|
||||
**If your shard distributes its own patched client to players, use that copy.**
|
||||
Any cliloc edits you shipped to players are then already in the base table and
|
||||
you need no overlay for them (see [`CLILOCS.md`](CLILOCS.md) §Shard-added and
|
||||
shard-edited items).
|
||||
|
||||
---
|
||||
|
||||
## Part 1 — Convert the cliloc table
|
||||
|
||||
**Goal:** turn the client's compressed `Cliloc.enu` into a file the site can
|
||||
read, and point the site at it.
|
||||
|
||||
The site cannot read `Cliloc.enu` directly. Every modern client compresses it
|
||||
(the "Mythic" container), and so does ServUO's own bundled `Ultima.StringList` —
|
||||
which is why the shard cannot supply names on our behalf either. The full
|
||||
reasoning is in [`CLILOCS.md`](CLILOCS.md) §Why the operator has to convert the
|
||||
file; this section is just the procedure.
|
||||
|
||||
Two routes. **The bundled tool is the recommended one** — the GUI export needs a
|
||||
fixup step, described below.
|
||||
|
||||
### Route A — the bundled converter (recommended)
|
||||
|
||||
Needs a .NET SDK (any version 8 or newer — the project targets `net8.0` and rolls
|
||||
forward, so whatever you have works) **plus** the .NET 10 runtime from Part 0,
|
||||
which is what actually loads `Ultima.dll`.
|
||||
|
||||
```bash
|
||||
cd website/server/tools/cliloc-export
|
||||
dotnet build -c Release
|
||||
|
||||
# plain binary — recommended, exact
|
||||
dotnet run -c Release -- \
|
||||
"/path/to/UOFiddler-4.22.2/Ultima.dll" \
|
||||
"/path/to/UO client/Cliloc.enu" \
|
||||
/srv/uo-data/clilocs.plain
|
||||
|
||||
# or tab-delimited text, if you want to eyeball or hand-edit it
|
||||
dotnet run -c Release -- \
|
||||
"/path/to/UOFiddler-4.22.2/Ultima.dll" \
|
||||
"/path/to/UO client/Cliloc.enu" \
|
||||
/srv/uo-data/clilocs.tsv --tsv
|
||||
```
|
||||
|
||||
Expected output for a stock English client:
|
||||
|
||||
```
|
||||
wrote 123490 entries to /srv/uo-data/clilocs.plain (maxTextBytes=12150, skippedOversize=0)
|
||||
```
|
||||
|
||||
**Sanity-check that number.** A stock `Cliloc.enu` is ~123,000 entries. A few
|
||||
hundred means it read something else and you should not ship the result. The
|
||||
tool exits non-zero and says `no entries were written — is that a cliloc file?`
|
||||
when it gets nothing at all.
|
||||
|
||||
The conversion runs on whatever machine has the client (usually Windows), and the
|
||||
site reads the output wherever it runs — so **copy the output file to the server**
|
||||
if those are different machines. It is a single self-contained file (~5 MB); the
|
||||
`--tsv` form is larger but diff-able.
|
||||
|
||||
<details>
|
||||
<summary>Errors you may hit</summary>
|
||||
|
||||
| Message | Cause |
|
||||
|---|---|
|
||||
| `Ultima.StringList not found — is that really UOFiddler's Ultima.dll?` | First argument points at some other `Ultima.dll` (ServUO ships one too — it is **not** the same assembly and cannot do this) |
|
||||
| `You must install .NET to run this application` | Missing the .NET 10 runtime from Part 0 step 3 |
|
||||
| `Unexpected Ultima.StringList API` | UOFiddler older than 4.21 |
|
||||
| `usage: clilocexport …` | Fewer than three arguments |
|
||||
|
||||
</details>
|
||||
|
||||
### Route B — the UOFiddler GUI
|
||||
|
||||
Use this if you would rather not install a .NET SDK. **It needs one extra step**,
|
||||
so do not skip the fixup.
|
||||
|
||||
1. Launch `UoFiddler.exe` and point it at your client directory when it asks
|
||||
(or **Options → Path Settings**).
|
||||
2. Open the **Cliloc** tab and use its **export to CSV** action.
|
||||
3. It writes `CliLoc.csv` to UOFiddler's configured output path, in **three**
|
||||
columns with a header row:
|
||||
|
||||
```
|
||||
Number;Text;Flag
|
||||
1023721;quarter staff;0
|
||||
```
|
||||
|
||||
4. **Strip the trailing flag column.** The site's text parser reads
|
||||
`number<TAB|,|;>text`, so that third field is otherwise absorbed into the name
|
||||
and every item on the site renders as `quarter staff;0`.
|
||||
|
||||
```bash
|
||||
sed -E 's/;[0-9]+$//' CliLoc.csv > clilocs.csv
|
||||
```
|
||||
|
||||
```powershell
|
||||
Get-Content CliLoc.csv |
|
||||
ForEach-Object { $_ -replace ';\d+$','' } |
|
||||
Set-Content -Encoding utf8 clilocs.csv
|
||||
```
|
||||
|
||||
The header row needs no removal — a line whose first field is not an integer
|
||||
is skipped. Blank entries (`1005008;`) survive the fixup correctly and are
|
||||
dropped at import, as intended.
|
||||
|
||||
5. Copy `clilocs.csv` to the server.
|
||||
|
||||
**Why the fixup is not just done for us:** the parser already handles
|
||||
`number,flag,text` — the flag in the *middle*, which is what several exports
|
||||
emit. UOFiddler puts it at the *end*, where it is indistinguishable from a name
|
||||
that genuinely ends in `;0`. One `sed` on the operator's side beats a parser
|
||||
heuristic that would corrupt real names.
|
||||
|
||||
### Point the site at it
|
||||
|
||||
Two ways, the setting winning over the environment:
|
||||
|
||||
| Where | How |
|
||||
|---|---|
|
||||
| **Admin → Shard → cliloc path** | Takes effect on the next refresh, no redeploy |
|
||||
| `UO_CLIENT_PATH` env var | The deploy-time default |
|
||||
|
||||
The value may be **the file itself or a directory to search** — both are natural
|
||||
answers to "where is it", and overlays are picked up either way.
|
||||
|
||||
Setting the path deliberately does **not** import as a side effect. Click
|
||||
**Import** (or `POST /api/v1/admin/shard/clilocs/import`) to load it.
|
||||
|
||||
### Verify
|
||||
|
||||
`GET /api/v1/admin/shard/clilocs`, or the Admin → Shard panel, reports what each
|
||||
source contributed:
|
||||
|
||||
```json
|
||||
"sources": [
|
||||
{ "label": "clilocs.plain", "kind": "base", "entries": 123490, "added": 123490, "overrode": 0 }
|
||||
]
|
||||
```
|
||||
|
||||
Roughly **67,500 rows stored** from a stock table is correct — about half a
|
||||
cliloc table is empty strings for ids the client reserves and never uses.
|
||||
|
||||
Then load any character sheet with equipment: items should show names rather than
|
||||
`id 1023721`.
|
||||
|
||||
<details>
|
||||
<summary>What a refusal means</summary>
|
||||
|
||||
A bad file answers `200` with a `status` and a named reason, not a `500` — you
|
||||
need to be told *which file* to fix.
|
||||
|
||||
| `code` | Meaning |
|
||||
|---|---|
|
||||
| `COMPRESSED` | You pointed at the raw client `Cliloc.enu`. Convert it — this whole page. |
|
||||
| `TRUNCATED` | Half-copied file. Re-copy; the loaded table is untouched. |
|
||||
| `EMPTY` | A text source with no parseable rows — the file is named in the reason. |
|
||||
| `status: needsReview` + `missingSources` | A previously-loaded source has vanished (unmounted volume? deliberate deletion?). Nothing changes until you re-import with `{ "approve": true }`. |
|
||||
|
||||
</details>
|
||||
|
||||
### Custom items — do *not* re-export for these
|
||||
|
||||
Shard-added items carry ids no client table has. Drop a small delimited file in a
|
||||
`custom/` directory beside the base file and re-import:
|
||||
|
||||
```
|
||||
/srv/uo-data/
|
||||
clilocs.plain ← base, from this guide
|
||||
custom/
|
||||
01-uomysticmoon.tsv ← your additions and overrides
|
||||
```
|
||||
|
||||
Files are read in sorted order and **later sources win**, so an overlay both adds
|
||||
new ids and overrides stock ones you have re-purposed. **Adding one item never
|
||||
means re-exporting a 5 MB client file.** Details in [`CLILOCS.md`](CLILOCS.md).
|
||||
|
||||
---
|
||||
|
||||
## Part 2 — Creature art for the spawn atlas (optional)
|
||||
|
||||
**Goal:** put sprites on atlas pages. Purely cosmetic — the atlas is fully
|
||||
functional as text, and `art` is NULL on every fresh import.
|
||||
|
||||
**This project ships no art and no art-extraction tooling, and never will.**
|
||||
|
||||
1. In `UoFiddler.exe` (paths configured as in Route B step 1), open the
|
||||
**Animations** tab for creature sprites — or **Items** for object art — find
|
||||
the creature, and export as PNG. Right-click an entry for its export options,
|
||||
or use the tab's *Export All* action for a batch. (4.22.2 added an export
|
||||
option to the Animation tab's thumbnail list, which is the convenient one
|
||||
here.)
|
||||
2. Put the images under `server/uploads/atlas/`.
|
||||
3. Copy `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json` in
|
||||
the same directory and map creature slugs to file names:
|
||||
|
||||
```json
|
||||
{
|
||||
"lizardman": "lizardman.png",
|
||||
"orc": "orc.png"
|
||||
}
|
||||
```
|
||||
|
||||
**Keys are the slugs the atlas API reports**, derived from the type names in
|
||||
your own shard's `Spawns/*.xml` — read them off the atlas rather than guessing.
|
||||
A creature with no entry renders without art, which is the default.
|
||||
|
||||
4. Restart, or `npm run atlas:import -- --force`.
|
||||
|
||||
The art map is re-read on every atlas refresh, so adding one image is an edit plus
|
||||
a refresh. Both `spawnAtlas.art.json` and `server/uploads/` are gitignored.
|
||||
|
||||
---
|
||||
|
||||
## Licensing, briefly
|
||||
|
||||
UO's strings and sprites are EA's. Extracting from **your own** client for
|
||||
**your own** shard is the arrangement here; redistributing the extracted files is
|
||||
not something this project does or can advise on. That is the whole reason this
|
||||
page exists instead of a download link.
|
||||
Reference in New Issue
Block a user