wtclaude
f6a22734d3
Both parts are built: Android-app#30 (the visibility rules + read-model adds) and #31 (the four screens, stacked on it). The on-device five-rung walk against a website on the cutover branch is what edge->main is now actually waiting on. Co-Authored-By: Claude <noreply@anthropic.com>
1018 lines
68 KiB
Markdown
1018 lines
68 KiB
Markdown
# Protocol 3.0 — Shard content, standings & the visibility framework
|
||
|
||
**Status:** Feature-complete on `edge`; the cutover (order 6) is in review. All work lands on an `edge` branch in each repo; `edge` → `main` is the v3 cutover.
|
||
**Date:** 2026-07-28
|
||
**Codebase:** ServUO 57.4, `<servuo>`, net48 / x64, Expansion **EJ**.
|
||
**Companion to** [`PLAN.md`](PLAN.md) (1.0 read/event plane), [`PROTOCOL_2.md`](PROTOCOL_2.md) (2.0 provisioning + world-state streams), [`ADMIN_CONTROLS.md`](ADMIN_CONTROLS.md) (staff write plane), [`INTEGRATION.md`](INTEGRATION.md) (website API).
|
||
|
||
### Progress
|
||
|
||
Each part is marked off here as it lands on `edge`. §9 carries the same state per sequencing row.
|
||
|
||
| Order | Part | State | Landed on `edge` |
|
||
|---|---|---|---|
|
||
| 1 | **A** — visibility framework + actor-leak fix (§3) | ✅ **Done** | website [#109](https://gitea.whitlocktech.com/RunicGateway/website/pulls/109) + [#110](https://gitea.whitlocktech.com/RunicGateway/website/pulls/110), docs [#64](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/64) + [#65](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/65) |
|
||
| 2 | **B/1** — `world.ruleset` (§5) | ✅ **Done** | servuo-plugins [#3](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/3), link [#17](https://gitea.whitlocktech.com/RunicGateway/link/pulls/17), website [#111](https://gitea.whitlocktech.com/RunicGateway/website/pulls/111), docs [#66](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/66) |
|
||
| 3 | **C** — spawn atlas (§6) | ✅ **Done** | website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112) (parsers + CLI + tables) + [#113](https://gitea.whitlocktech.com/RunicGateway/website/pulls/113) (API + pages + admin panel), docs [#67](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/67) + [#68](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/68) |
|
||
| 4 | **B/2** — `points.board` (§7) | ✅ **Done** | servuo-plugins [#4](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/4), link [#18](https://gitea.whitlocktech.com/RunicGateway/link/pulls/18), website [#114](https://gitea.whitlocktech.com/RunicGateway/website/pulls/114), docs [#69](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/69) |
|
||
| 5a | **B/3 dependency** — cliloc table (§8.6) | ✅ **Done** | website [#115](https://gitea.whitlocktech.com/RunicGateway/website/pulls/115), docs [#70](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/70) |
|
||
| 5b | **B/3** — `vendor.listing` (§8) | ✅ **Done** | servuo-plugins [#5](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/5), link [#19](https://gitea.whitlocktech.com/RunicGateway/link/pulls/19), website [#116](https://gitea.whitlocktech.com/RunicGateway/website/pulls/116), docs [#71](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/71) |
|
||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3 (§4) | 🟨 In review | the bump: link [#20](https://gitea.whitlocktech.com/RunicGateway/link/pulls/20), website [#117](https://gitea.whitlocktech.com/RunicGateway/website/pulls/117), docs [#72](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/72) — then `edge` → `main`: servuo-plugins [#6](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/6), link [#21](https://gitea.whitlocktech.com/RunicGateway/link/pulls/21), website [#118](https://gitea.whitlocktech.com/RunicGateway/website/pulls/118), docs [#73](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/73) |
|
||
|
||
Order 5 split in two once §8.6's cliloc dependency turned out to be a client-format problem rather
|
||
than a parser (see §8.6). 5a is website-only and lands first so the marketplace ships with real item
|
||
names; 5b is the four-repo wire change.
|
||
|
||
**The `edge` → `main` half of order 6 is held for Android parity** (decided 2026-07-30, see §10): the
|
||
app sees none of the four new features and gates shard nav on session role alone, so merging the
|
||
cutover first would ship a shard whose app client silently disagrees with the web client about what is
|
||
public. The **bump** PRs into `edge` are unaffected and merge normally.
|
||
|
||
---
|
||
|
||
## 1. Why 3.0
|
||
|
||
A survey of the live ServUO tree against everything the bridge already surfaces end-to-end found that
|
||
**the bridge covers live *activity* well and covers shard *content and standings* almost not at all.**
|
||
|
||
Covered by 1.0 + 2.0: presence/online, region transitions, char vitals + profile + roster, house
|
||
registry + IDOC decay, champion spawns, guild board, city governors + term history, help-page queue,
|
||
total gold supply, player-vendor sales log, deaths/murders/kills, skill gains, fame/karma, quest
|
||
completes, staff/cheat audit, account linking + creation, town crier + news.
|
||
|
||
Not covered by anything: every leaderboard, every ruleset fact, every "where do I find X", and the
|
||
entire player economy outside a player's own vendors.
|
||
|
||
3.0 has **three scope areas**:
|
||
|
||
- **A — The visibility framework (§3).** Admin-configurable, per-feature and per-field audience
|
||
control over every shard-derived surface on the website. Ships first; the rest depends on it.
|
||
- **B — Three new wire streams (§5, §7, §8).** `world.ruleset`, `points.board`,
|
||
`vendor.listing`/`vendor.listing.remove`.
|
||
- **C — One website-only feature (§6).** The spawn atlas, built from static ServUO data files with no
|
||
wire involvement at all.
|
||
|
||
---
|
||
|
||
## 2. Survey: the full gap list
|
||
|
||
Recorded so the items *not* scoped for 3.0 aren't re-derived later.
|
||
|
||
| # | Gap | Source on the shard | Value | Cost | Status |
|
||
|---|---|---|---|---|---|
|
||
| 1 | **Points/loyalty leaderboards** — 25 point currencies | `Scripts/Services/PointsSystems/PointsSystem.cs` → `static List<PointsSystem> Systems`, each `List<PointsEntry>{Player,Points}` | Very high | Low | **3.0 §7** |
|
||
| 2 | **Shard ruleset page** | `Config/*.cfg` via `Server.Config.Get` | High | Very low | **3.0 §5** |
|
||
| 3 | **Shard-wide marketplace** | `PlayerVendor.PlayerVendors` + `VendorSearch.cs` | Very high | High | **3.0 §8** |
|
||
| 4 | **Spawn atlas / bestiary** | `Spawns/*.xml` (6,455 spawners), `RevampedSpawns/*.xml` (333), `Data/Regions.xml`, `Data/Locations/*.xml`, `Config/ChampionSpawns.xml`, `Data/teleporters.csv`, `Data/HarvestLocs/*` | High | Medium | **3.0 §6** |
|
||
| 5 | VvV standings + battle status | `Services/ViceVsVirtue/{ViceVsVirtueSystem,GuildStats,VvVBattle}.cs` | High | Medium | `PROTOCOL_2.md` §10.4 deferred this pending "which PvP system does this shard run?" — **now answered: `VvV.cfg Enabled=True`, `Factions.cfg` off.** Unblocked, not scoped here |
|
||
| 6 | Skill leaderboards + shard census | `Services/Reports/Reports.cs` → `GetSkillDistribution()`, `CompileGeneralStats()`, `StaffHistory` | High | Low | Spec'd `PROTOCOL_2.md` §14 (Part B phase 6), unbuilt. Shares §7's UI — fold in after |
|
||
| 7 | Custom mounts/pets codex — ~35 across 4 tiers | `Scripts/Custom/{Companions,Legendary,Named,New Legacy}` | Medium-high | Very low | Pure wiki/CMS content, zero bridge work. The shard's most distinctive content, with zero site presence |
|
||
| 8 | Community Collections progress | `Services/CommunityCollections/CollectionsSystem.cs` | Medium | Low | Natural public "community goal" widget |
|
||
| 9 | Seasonal/holiday event calendar | `Services/Seasonal Events/SeasonalEventSystem.cs`, Krampus, Forsaken Foes | Medium | Low | "What's live now / what's next" |
|
||
| 10 | Crafting / taming / harvesting feeds | `EventSink.CraftSuccess` / `TameCreature` / `ResourceHarvestSuccess` | Medium | Low | Spec'd `PROTOCOL_2.md` §11 #3/#4/#5, unbuilt |
|
||
| 11 | Virtue progression | `EventSink.VirtueLevelChange`, `Services/Ethics/` | Medium | Low | Spec'd §11 #6, unbuilt |
|
||
| 12 | Bulk Order Deeds + reward tables | `Services/BulkOrders/`, `Data/Bulk Orders/*` | Medium | Low | Feed spec'd §11 #7; the static reward tables are a free wiki page |
|
||
| 13 | Guild wars | war state on `Guild` | Low-medium | Low | Spec'd §11 #8, unbuilt |
|
||
| 14 | Astronomy discovery log | `Services/Astronomy/AstronomySystem.cs` (104 KB save) | Low | Low | Niche completion leaderboard |
|
||
| 15 | In-game chat relay | `Services/Chat/`, `Logs/Chat/{General,Help,Trade,LFG}` | Low | Medium | Privacy-sensitive; staff-only at most |
|
||
| 16 | Shard health telemetry | Crash logs, `LayerConflict.log`, `throttle.log`, `world.save.after` counts, AutoSave/AutoRestart schedule | Low-medium | Low | `world.save.after` is already ingested but never charted — world-size-over-time is nearly free |
|
||
| 17 | Ultima Store / Sovereigns balance | `Store.cfg Enabled=True, CurrencyName=Sovereigns`; `UltimaStore.GetCurrency` | ? | Medium | Only worth it if sovereigns are actually sold |
|
||
|
||
**Excluded permanently** — see [`ADMIN_CONTROLS.md`](ADMIN_CONTROLS.md) Tier H/N: firewall/IP-block,
|
||
kill/resurrect, jail, item/gold grants, set-access-level, arbitrary `[set`/`[add`.
|
||
|
||
**Noticed during the survey, out of scope:** `Scripts/Custom/PerryOwnerFix.cs` hardcodes an
|
||
`EventSink.Login` hook granting `AccessLevel.Owner` to account `"ShardOnwerPerry"`. Worth reviewing
|
||
independently of this work.
|
||
|
||
---
|
||
|
||
## 3. Part A — The visibility framework ✅ Done
|
||
|
||
*Landed on `edge`: website [#109](https://gitea.whitlocktech.com/RunicGateway/website/pulls/109) (the framework)
|
||
and [#110](https://gitea.whitlocktech.com/RunicGateway/website/pulls/110) (the REST-projection gap §3.6.1
|
||
records), docs [#64](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/64) + [#65](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/65).
|
||
Smoke-tested across all five rungs per §11.*
|
||
|
||
### 3.1 The leak this replaces (verified 2026-07-28)
|
||
|
||
`BridgeJson.Actor()` (`BridgeJson.cs:85-117`) writes `serial`, `name`, **`acct`**, **`webId`**,
|
||
`player`. `shardState.model.js:346 shapeGuild()` returns `r.payload` verbatim, and
|
||
`GET /api/v1/public/shard/guilds` (anonymous, `shard.controller.js:131`) serves it. **A guild
|
||
leader's game account name and website user id are readable on an anonymous public endpoint today.**
|
||
The same path exists for `shapeGovernor` → `/public/shard/governors`. `Actor` also feeds
|
||
`guild.join`, `city.update` and `region.enter`, all three in `PUBLIC_KINDS` on the anonymous SSE
|
||
stream.
|
||
|
||
The framework below is the vehicle for the fix, and the reason it ships before anything else.
|
||
|
||
### 3.2 Where visibility lives
|
||
|
||
**On the website, never in the sidecar.** The sidecar's job for 3.0 is unchanged in character: accept
|
||
frames, persist them to its SQLite store, forward them verbatim over WS, and serve store-backed reads
|
||
that survive a shard outage. It defines no access parameters, no audiences, no field projection, and
|
||
advertises no capabilities.
|
||
|
||
### 3.3 The audience ladder
|
||
|
||
`anonymous → logged_in → player → staff → admin`, each rung implying the ones below it.
|
||
|
||
`viewerLevel(req)` resolves: no session ⇒ `anonymous`; authenticated ⇒ `logged_in`; authenticated
|
||
with a linked shard account ⇒ `player`; moderator/admin role ⇒ `staff`/`admin`. **Staff always
|
||
satisfy the `player` rung** even without a linked game account, consistent with the existing rule
|
||
that `/player/*` is role-agnostic self-service.
|
||
|
||
### 3.4 Two limits an admin cannot override
|
||
|
||
1. **`acct` and `webId` are admin-only, always.** They are not in-game-visible and are not exposed as
|
||
configurable fields.
|
||
2. **A kind absent from the kind→feature map is never broadcast below `admin`.** Fail closed. This
|
||
preserves the property that today's static `PUBLIC_KINDS` allowlist is a security boundary rather
|
||
than a convenience filter.
|
||
|
||
### 3.5 Configuration
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS shard_feature_visibility (
|
||
feature VARCHAR(48) NOT NULL PRIMARY KEY,
|
||
enabled TINYINT(1) NOT NULL DEFAULT 1,
|
||
audience VARCHAR(20) NOT NULL DEFAULT 'anonymous',
|
||
field_rules JSON NULL, -- {"<field>": "<rung>"} for sensitive fields only
|
||
updated_by INT NULL,
|
||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
```
|
||
|
||
**Not** seeded on boot (this changed during implementation): an **absent row means "use the compiled
|
||
default"**, so the table starts empty and only ever holds rows an admin has actually touched. The
|
||
defaults live in one place — `FEATURES` in `shardVisibility.js` — instead of being duplicated into a
|
||
seeder that could drift from it, and a DB blip degrades to those same defaults rather than to
|
||
"everything is public". **All ten shard features are covered — the four new ones and the six that
|
||
already ship — and every default reproduces today's behavior, so the retrofit is a no-op until an
|
||
admin changes something.**
|
||
|
||
| Feature | Default audience | Sensitive fields (default rung) |
|
||
|---|---|---|
|
||
| `status`, `activity`, `champs`, `guilds`, `governors` | `anonymous` | guilds/governors: `leaderAcct` / `leaderWebId` → **admin (locked)** |
|
||
| `houses` | `anonymous` | `owner` → `staff`, `price` → `staff` (matches today's IDOC-only public view) |
|
||
| `presence` | `anonymous` | `location` → `staff` (matches today's staff-only, location-gated `/online`) |
|
||
| `ruleset` (new) | `anonymous` | `connect` → `anonymous` |
|
||
| `atlas` (new) | `anonymous` | — |
|
||
| `leaderboards` (new) | `anonymous` | `characterName` → `anonymous` |
|
||
| `market` (new) | `anonymous` | `ownerName` → `anonymous`, `location` → `anonymous` |
|
||
|
||
### 3.6 Enforcement — three points, one config
|
||
|
||
New `website/server/src/utils/shardVisibility.js`:
|
||
|
||
- `LADDER = ['anonymous','logged_in','player','staff','admin']`, `rank()`, `meets(viewer, required)`
|
||
- `viewerLevel(req)` (§3.3)
|
||
- `KIND_FEATURE` — every event kind → its feature; unmapped ⇒ admin-only (§3.4)
|
||
- `getConfig()` — DB-backed, cached ~5 s like `uoLinkClient`'s config cache, busted on admin `PUT`
|
||
- `requireFeature(name)` — **404 when disabled** (don't leak existence), **403 when enabled but the
|
||
viewer is below the audience**
|
||
- `projectFeature(name, payload, viewerLevel)` — strips fields whose rung the viewer doesn't meet;
|
||
`acct`/`webId` always stripped below `admin`
|
||
|
||
Applied at:
|
||
|
||
1. **Routes** — `requireFeature(…)` on every `/public/shard/*`, `/public/atlas/*` and the
|
||
shard-derived player routes; `projectFeature` in the controllers, replacing the ad-hoc
|
||
`shapeGuild`-returns-payload-verbatim path.
|
||
2. **SSE** — `shardBroadcast.js` moves from *"one public channel with a static `PUBLIC_KINDS`
|
||
allowlist plus one admin channel"* to **per-connection filtering**: each subscriber carries its
|
||
`viewerLevel`; each frame is mapped kind→feature, gated on `enabled && meets(...)`, then passed
|
||
through `projectFeature` before write. `PUBLIC_KINDS` becomes the seed data for `KIND_FEATURE`
|
||
rather than a hardcoded gate. **This is the largest single change in Part A and where the security
|
||
boundary now lives.**
|
||
3. **Nav** — `GET /api/v1/public/shard/features` returns only the features the calling viewer can
|
||
see, so the SPA hides nav entries rather than rendering links that 403.
|
||
|
||
### 3.6.1 What the first implementation missed (found by the §11 smoke test, fixed)
|
||
|
||
Part A shipped enforcement on the SSE path and on `/guilds` + `/governors`, but the **remaining public
|
||
REST reads never called into it** — so the same event was projected live and served verbatim from
|
||
history. Recorded because each miss is a shape the next phase can repeat:
|
||
|
||
- **`/public/shard/feed` returned the stored payload as-is.** `actor.acct` / `actor.webId` were
|
||
readable *anonymously* for every logged kind (`player.death`, `mob.killed`, `skill.gain`,
|
||
`guild.join`, …) — broader than the §3.1 leak, which was limited to board holders.
|
||
- **`/public/shard/idoc` returned `ownerAcct`.** Rule 1 keyed on the exact strings `acct`/`webId`,
|
||
but `shapeHouse` flattens the actor into `ownerAcct` / `ownerName` / `ownerSerial`. The lock is now
|
||
on the field's **meaning** — a key that is or ends in `acct`/`webId`, case-insensitively — so
|
||
flattened spellings are covered and unwritten shapes fail closed.
|
||
- **The `houses` field rules were dead config.** Neither `getIdoc` nor `getHouses` projected, so the
|
||
panel offered toggles that did nothing. **Every feature's declared fields must name the keys the
|
||
read model actually emits**, not just the wire frame's.
|
||
- **`/feed` filtered on `PUBLIC_KINDS`**, a module-load constant derived from the compiled defaults,
|
||
so live audience changes never reached it. `visibleKinds(level, config)` resolves the readable set
|
||
from live config; it deliberately ignores the `stream` flag, which governs SSE fan-out only (market
|
||
history stays readable with its firehose off).
|
||
- **`shardEvents.db.list` treated an empty `kinds` array as "no filter"** and fell through to an
|
||
unfiltered `SELECT`. A fully-gated config would have dumped the whole event log, staff audit
|
||
included. An empty allowlist now serves nothing.
|
||
- **`projectValue` recursed into every object**, so a `Date` column came back as `{}`. It walks
|
||
arrays and plain objects only. The unit tests used JSON fixtures and could not have caught this —
|
||
the live read did, which is the argument for §11's smoke test over tests alone.
|
||
|
||
**The rule this leaves behind:** *a read path that returns shard data and does not call
|
||
`projectFeature` is a bug.* Every new surface in Parts B and C — `/ruleset`, `/points`, `/market`,
|
||
`/atlas` — must project, and must gate its kind set on live config rather than on `PUBLIC_KINDS`.
|
||
|
||
### 3.7 Admin surface
|
||
|
||
`GET` / `PUT /api/v1/admin/shard/visibility` (admin-only). Validate feature names against the known
|
||
set and rungs against the ladder; reject any attempt to set a locked field below `admin` — including
|
||
its flattened spellings (`ownerAcct`, `leaderWebId`), see §3.6.1. Writes an
|
||
`admin.audit`-style row so visibility changes are traceable. New client panel
|
||
`routes/admin/ShardVisibility.jsx` at `/admin/shard-visibility`, linked from `ShardAdmin.jsx`.
|
||
|
||
---
|
||
|
||
## 4. The version bump and the rollout
|
||
|
||
`PROTOCOL_VERSION` **2 → 3** in `link/sidecar/src/main.rs:27`. v3 is defined as *"adds
|
||
`world.ruleset`, `points.board`, `vendor.listing` / `vendor.listing.remove`"*.
|
||
|
||
A bump is an operator-visible hard cutover — `web.rs::gate` returns 409 on every protected route on
|
||
mismatch, `uoLinkSocket.js::handleHello` closes the WS, and the website's declared version is the
|
||
admin-set `uo_link_config.protocol` column — so it happens **exactly once**, at the end:
|
||
|
||
- Cut an **`edge`** branch from `main` in each of `website/`, `link/`, `servuo-plugins/`, `docs/`.
|
||
- Every phase PRs into `edge`, never `main`. Feature branches are cut from `edge`.
|
||
- Part A lands first, alone.
|
||
- When all parts are built and tested, one `edge` → `main` PR per repo, merged together. **That merge
|
||
is the v3 cutover.**
|
||
- A schema migration sets `uo_link_config.protocol` (the existing row **and** the column default)
|
||
from 2 to 3, so the cutover doesn't require a manual admin edit. `UOLINK_PROTOCOL` still overrides.
|
||
- No feature-negotiation array anywhere — v3 implies all three kinds.
|
||
|
||
### 4.1 What the bump actually touches
|
||
|
||
The version lives in five places, and all five move together:
|
||
|
||
| Where | Change |
|
||
|---|---|
|
||
| `link/sidecar/src/main.rs` | `PROTOCOL_VERSION` 2 → 3 (with the v3 note beside the v2 one), plus the sidecar README's worked example |
|
||
| `website/server/db/schema.sql` | `uo_link_config.protocol` column default 1 → 3, plus the boot migration below |
|
||
| `website/server/src/model/uoLinkConfig/uoLinkConfig.model.js` | `DEFAULT_PROTOCOL` — what a site with nothing saved yet declares |
|
||
| `website/server/src/utils/uoLinkClient.js` + `uoLinkSocket.js` | the `config.protocol || …` fallbacks, so an unset value can never quietly send `1` and 409 with a confusing message |
|
||
| `website/client/.../ShardAdmin.jsx`, `website/.env.example` | the admin form's initial value and the documented env default |
|
||
|
||
**The migration has to be one-shot, and that is the only subtle part.** `schema.sql` is re-run on
|
||
*every* boot (`utils/db.js::ensureSchema`), and every other statement in its migration block is an
|
||
idempotent `ADD COLUMN IF NOT EXISTS` / `MODIFY`. A bare `UPDATE uo_link_config SET protocol = 3`
|
||
would not be idempotent in the sense that matters: `protocol` is **admin-editable**, so an operator
|
||
who deliberately pins an older sidecar in Admin → Shard would silently be un-pinned on the next
|
||
restart. It is therefore gated on a marker row in `settings`:
|
||
|
||
```sql
|
||
ALTER TABLE uo_link_config MODIFY COLUMN protocol INT NOT NULL DEFAULT 3;
|
||
UPDATE uo_link_config SET protocol = 3
|
||
WHERE id = 1 AND protocol < 3
|
||
AND NOT EXISTS (SELECT 1 FROM settings WHERE `key` = 'uo_link_protocol_3_migrated');
|
||
INSERT IGNORE INTO settings (`key`, value) VALUES ('uo_link_protocol_3_migrated', '1');
|
||
```
|
||
|
||
The marker is written *after* the `UPDATE`, so the first boot on the new build migrates and every
|
||
later boot is a no-op. A fresh install has no `uo_link_config` row to update and simply gets the
|
||
marker plus the new column default. `protocol < 3` rather than `= 2` so an install that never left
|
||
the old default of `1` is carried across too — it could not have been talking to a v2 sidecar
|
||
anyway.
|
||
|
||
---
|
||
|
||
## 5. Part B/1 — `world.ruleset` ✅ Done
|
||
|
||
*Landed on `edge`: servuo-plugins [#3](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/3), link [#17](https://gitea.whitlocktech.com/RunicGateway/link/pulls/17), website [#111](https://gitea.whitlocktech.com/RunicGateway/website/pulls/111), docs [#66](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/66). Implementation notes worth keeping:*
|
||
|
||
- ***`shadowguard` is derived, not configured.*** `Shadowguard.cfg` carries only `ReadyDuration` and
|
||
`RandomizeInstances` — there is no `Enabled` key — so the systems block reports `Core.TOL`
|
||
(the expansion gate) instead. Same shape for `factions`: `Factions.cfg` has no `Enabled` either, and
|
||
`Services/Factions/Core/Faction.cs` sets `Settings.Enabled = !ViceVsVirtueSystem.Enabled`, so the
|
||
frame reads that static rather than inventing a key. **Where a system's on/off state is derived, read
|
||
the system's own static; only read `Config.Get` where the .cfg key IS the truth.**
|
||
- **`caps.skill` / `caps.totalSkill` are in tenths** (1000 = 100.0), the way ServUO stores them.
|
||
Documented in `INTEGRATION.md` and converted in the client, because the raw number is actively
|
||
misleading rather than merely unhelpful.
|
||
- **`Config.Get` re-parses when the cached type differs.** `InternalGet<T>` caches the parsed value on
|
||
the entry and re-parses if `entry.Object is T` fails, so reading `PlayerCaps.SkillCap` as an `int`
|
||
where ServUO reads it as a `double` is correct (both parse) — it just re-parses. Harmless, but worth
|
||
knowing before assuming a shared cache.
|
||
- **The plugin CAN be compile-verified**, contrary to "no standalone build": point Roslyn
|
||
(`dotnet sdk/*/Roslyn/bincore/csc.dll`, `/langversion:7.3`, net48 reference assemblies) at the whole
|
||
ServUO `Scripts` tree with `overlay/Scripts/Custom/Bridge/*.cs` substituted for the deployed copy,
|
||
excluding `Scripts/obj` and `Scripts/bin`. 6,205 files, ~40 s, and it catches every signature error
|
||
a boot would. Worth doing before every plugin PR.
|
||
|
||
`PROTOCOL_2.md` §10.4 sketches a `world.systems` capability frame that was never implemented
|
||
(`grep` returns nothing across all four repos). **`world.ruleset` subsumes it**, carrying a `systems`
|
||
sub-object with the `cityLoyalty` / `vvv` / `factions` booleans §10.4 asked for. §10.4 is marked
|
||
superseded; no orphan kind is left behind.
|
||
|
||
### 5.1 Plugin
|
||
|
||
NEW `servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeRuleset.cs`, modelled on
|
||
`BridgeBoot.EmitHello` — **not** a sweep. Subscribes `BridgeLink.Connected_Core += Emit` so a sidecar
|
||
that comes up second still learns the ruleset.
|
||
|
||
Built from an **explicit allowlist** of `Server.Config.Get<T>` calls. **Never enumerate
|
||
`Config.Entries`** (`Server/Config.cs:162`) — it would sweep in secrets. An FNV-1a `rev` over the body
|
||
makes an unchanged reconnect a site-side no-op (`String.GetHashCode()` is not stable across runs and
|
||
must not be used).
|
||
|
||
`BridgeConfig.cs` + `overlay/Config/Bridge.cfg`: `RulesetEnabled=true`, `PublicConnectAddress=""`,
|
||
`RulesetIncludeSchedule=true`. `BridgeBoot.cs`: `reload` → re-emit, `status` → rev/bytes. Not wired
|
||
to `sweepnow`; it isn't a sweep.
|
||
|
||
### 5.2 Payload
|
||
|
||
Every block optional, omitted when its system is off:
|
||
|
||
`shard`, `expansion`, `connect` (only from `PublicConnectAddress`),
|
||
`systems{cityLoyalty,vvv,factions,siege,chat,store,dailyRares,honesty,shadowguard,treasureMaps,vetRewards,testCenter}`,
|
||
`caps{skill:1000,totalSkill:7000,stat:225,str/dex/int:125,strMax/dexMax/intMax:150}`,
|
||
`housing{accountHouseLimit:1}`, `accounts{perIp:3,charSlots:7,autoCreate}`,
|
||
`vetRewards{enabled,rewardIntervalDays:30}`,
|
||
`loot{feluccaLuckBonus:1000,feluccaBudgetBonus:100,feluccaMaxProps:11}`,
|
||
`vendors{restockDelayMinutes,maxSell,economyStockAmount}`,
|
||
`champions{powerScrolls:6,statScrolls:16,scrollChance,transcendenceChance,rankThresholds}`,
|
||
`treasureMaps`, `vvv{enabled,startSilver:2000,enhancedRules}`, `store{enabled,currencyName}`,
|
||
`schedule{autoSaveFrequencyMinutes,autoRestart*}`.
|
||
|
||
**Excluded by name — in a code comment and here:** `Server.cfg` (Address/Listen/Port; only
|
||
`PublicConnectAddress` is published), `Staff.cfg`, `Email.cfg`, `DataPath.cfg`, `Bridge.cfg`,
|
||
`Compiler.cfg`, `Reports.cfg`, `Client.cfg`.
|
||
|
||
### 5.3 Sidecar and website
|
||
|
||
Sidecar — `store.rs`: singleton `ruleset(id CHECK(id=1), rev, json, updated_t)` + upsert/get;
|
||
`main.rs`: new arm in the board-projection match; `web.rs`: `GET /ruleset` served from the store, so
|
||
it answers during a shard outage (`PROTOCOL_2.md` §12.2).
|
||
|
||
Website — `uoLinkClient.getRuleset()`; `uoLinkSocket.backfill()` (object-shaped, so follow the
|
||
`getPresence()` block's explicit form, not the array-only `snapshot()` helper); `shardIngest.js` →
|
||
`shardState.setRuleset`, **not** in `LOGGED_KINDS` (it re-arrives every reconnect and `server.hello`
|
||
already marks those); `KIND_FEATURE['world.ruleset'] = 'ruleset'`; `shard_ruleset` singleton table
|
||
(`rev`, `expansion`, `payload JSON`, `t`); `GET /public/shard/ruleset` behind
|
||
`requireFeature('ruleset')`, returning `null` ⇒ "not published yet".
|
||
|
||
Client — NEW `routes/public/Rules.jsx` at `/site/rules`, alongside
|
||
`/site/champs|guilds|governors|houses`; live via `useShardFeed({ filter: new Set(['world.ruleset']) })`.
|
||
|
||
### 5.4 Risk
|
||
|
||
Perf is nil (~3 KB per connect). The only real risk is publishing a secret, mitigated by the explicit
|
||
allowlist, the no-`Config.Entries` rule, the named exclusion list, and a manual eyeball of the emitted
|
||
frame during verification.
|
||
|
||
---
|
||
|
||
## 6. Part C — Spawn atlas / bestiary (website-only)
|
||
|
||
**No plugin, no sidecar, no `Bridge.cfg` knob, no new kinds.** Not part of the v3 wire change.
|
||
|
||
> **Status:** complete on `edge` — website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112)
|
||
> (parsers, import CLI, tables) and [#113](https://gitea.whitlocktech.com/RunicGateway/website/pulls/113)
|
||
> (the six public routes, the five admin ones, `/site/atlas` + `/site/atlas/:slug`, and the
|
||
> Admin → Spawn Atlas panel).
|
||
> Part C ships as **two** website PRs, not one: the parsing half is where the correctness risk
|
||
> lives, and burying it under routes and React would have meant reviewing it in a 10k-line diff.
|
||
> Full operator documentation: [`docs/website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md).
|
||
>
|
||
> **§6 below is the original design and is partly superseded.** §6.1 records two decisions that were
|
||
> rejected in review and replaced (the committed artifact, and the fixed facet list); §6.2 records
|
||
> the corrections the real ServUO data forced. Read both before trusting §6.
|
||
|
||
**Decision (revised at implementation time): the shard's ServUO tree is the single source of truth,
|
||
re-derived on every server boot.** The original plan here was a committed generated artifact plus an
|
||
idempotent import. That was rejected in review for two reasons, recorded in §6.1: a snapshot in the
|
||
repo goes stale as a shard's maps change, and the design leaned on a fixed facet list that no shard
|
||
is obliged to keep. Still not a browser-served blob; still parsed server-side only.
|
||
|
||
New in `website/server/`:
|
||
|
||
- `src/utils/spawnAtlasParse.js` — **pure functions, no fs**, so they are unit-testable in CI without
|
||
a ServUO tree: `parseObjects2()`, `parsePoints()`, `parseRegions()`, `parseLocations()`,
|
||
`resolveRegion()`.
|
||
- ~~`scripts/buildSpawnAtlas.js` and a committed `db/data/spawnAtlas.*.json` artifact~~ — dropped,
|
||
see §6.1 R1. Replaced by `src/utils/spawnAtlasSource.js` (the only thing that reads a ServUO tree,
|
||
shared by the boot path and the CLI) and a `scripts/importSpawnAtlas.js` that is a thin CLI over
|
||
the model. `package.json` gains `atlas:import` only.
|
||
- `src/model/shardAtlas/{shardAtlas.db.js,shardAtlas.model.js}` following the `shardState` split.
|
||
- `src/router/v1/public/atlas.{router,controller}.js`; `test/spawnAtlas.parse.test.js`.
|
||
|
||
Two parsing notes that matter:
|
||
|
||
- `<Objects2>` is `Type:MX=n:SB=…` segments joined by `:OBJ=` — verified against `trammel.xml`, where
|
||
a single point carries six types. Split on `:OBJ=`; the token before the first `:` is the type.
|
||
- **The high-value transform:** point-in-rect each spawn against the facet's `Regions.xml` rects
|
||
(highest `priority` wins), falling back to the nearest `Data/Locations` landmark, else
|
||
`"Wilderness"`. This is what turns *"lizardman at 5411,1234"* into ***"Despise, Felucca"*** and is
|
||
the entire reason the page is worth building. `Regions.xml` is genuinely nested and needs a ~120-line
|
||
recursive tokenizer **or** one devDependency (`fast-xml-parser`) — the server has zero XML deps
|
||
today, so that is an explicit call to make at implementation time. The flat `<Points>` files need
|
||
only regex/streaming; **do not** put 10.5 MB through a DOM parser.
|
||
|
||
Tables: `shard_spawn_creatures` (slug PK, name, total, facets JSON), `shard_spawn_points` (slug,
|
||
facet, x, y, region, landmark, max_count, tod_*), `shard_regions`, `shard_landmarks`,
|
||
`shard_champion_spawns`, `shard_atlas_meta`. Plain `INDEX` on name, **not `FULLTEXT`** — ~1,500
|
||
creature rows makes a `LIKE` scan free, and FULLTEXT brings min-token-length trouble for names like
|
||
"orc". No FKs, consistent with every existing `shard_*` table.
|
||
|
||
Routes at `/api/v1/public/atlas`, **not** under `/shard` — the atlas is static shard *content*, not
|
||
live shard *state*; it must not look sidecar-dependent, and unlike `/shard/*` it *should* be
|
||
`siteMode`-gated like `/posts` and `/wiki`. `GET /creatures?q=&facet=`, `/creatures/:slug`,
|
||
`/regions`, `/landmarks`, `/champions`, `/meta`, all behind `requireFeature('atlas')`. Admin:
|
||
`GET /admin/shard/atlas/status` (artifact-vs-DB drift) and `POST /admin/shard/atlas/import`. **Build
|
||
stays CLI-only.**
|
||
|
||
Client: `routes/public/Atlas.jsx` (`/site/atlas`) and `AtlasCreature.jsx` (`/site/atlas/:slug`).
|
||
|
||
**Payload risk** — *superseded by §6.1 R1; nothing is committed.* The field selection it describes
|
||
still applies at parse time: every `<Points>` field the site cannot use (`UniqueId`, all
|
||
trigger/refractory/proximity/sequential fields, sound ids) is dropped, keeping
|
||
Name/Map/X/Y/W/H/Range/MaxCount/MinDelay/MaxDelay/TOD*/types. Parsed data never reaches the browser;
|
||
the browser sees only paginated API responses.
|
||
|
||
**Operator re-run story** — *revised by §6.1 R1.* Spawns changed → restart, or
|
||
`npm run atlas:import` / `POST /admin/shard/atlas/import` to apply without one. `shard_atlas_meta`
|
||
holds a sha256 per source file, so the server can tell on boot whether anything changed, and
|
||
`GET /admin/shard/atlas/status` reports drift. If the change would remove a facet it is staged for
|
||
approval rather than applied (§6.1 R3). Full detail in `docs/website/SPAWN_ATLAS.md`.
|
||
|
||
### 6.1 What implementation changed
|
||
|
||
Two design decisions in §6 were rejected in review and replaced; the rest are corrections the real
|
||
ServUO data forced. Kept as a diff rather than edited in place, because each is a trap the next
|
||
person would otherwise re-enter.
|
||
|
||
**R1. The committed artifact is gone — the tree is re-parsed on every boot.** §6 proposed building a
|
||
generated artifact, committing it, and importing it. Two problems. A shard's maps change over its
|
||
life, so a snapshot in the repo silently drifts from the world players actually see; and the build/
|
||
import split existed only to work around the website container not having a tree, which is a
|
||
deployment question (mount it) rather than a reason to freeze data. The server now hashes the source
|
||
files on boot and re-derives the atlas when they differ. `scripts/buildSpawnAtlas.js`, the 1.41 MB
|
||
artifact, and the whole encode/decode seam it needed are deleted.
|
||
|
||
**R2. Nothing may name a facet.** The first implementation carried a lookup table of the six stock
|
||
UO facets to reconcile the spelling drift between sources. A shard may add facets, replace them
|
||
outright, or rename them when its maps are updated, and a built-in list mishandles all three
|
||
silently. Reconciliation is now by *matching* against the facet set discovered from the shard's own
|
||
spawn and region data — exact key, then prefix in either direction — with an unmatched name keeping
|
||
its own rather than being forced into a wrong bucket.
|
||
|
||
**R3. Two contracts on the boot path.** It never blocks startup: no path, an unreadable mount, a
|
||
malformed file or a database error is caught and logged, and the site comes up serving whatever
|
||
atlas it had. And a refresh that would REMOVE a facet is never applied automatically — facet loss
|
||
is indistinguishable at boot from a half-copied or mid-update tree, so it is staged in
|
||
`shard_atlas_pending` for an admin to approve or reject. Only the decision is stored (source hashes
|
||
+ the facet diff, a few KB); approving re-parses, so what lands matches the tree at approval time.
|
||
A rejection is remembered against those hashes so it does not re-prompt every restart.
|
||
|
||
### 6.2 What the build against real data changed
|
||
|
||
Six corrections to the design above, from running it against stock ServUO 57.4. Kept as a diff
|
||
rather than edited in place, because each one is a trap the next person would otherwise re-enter.
|
||
|
||
**1. Six facets, not thirteen.** The design said `spawnAtlas.<facet>.json ×13`, assuming one facet
|
||
per spawn file. There are 13 files but only **6** facets — `Eodon.xml`, `GravewaterLake.xml`,
|
||
`TreasuresOfKotl.xml` and the other named-area files carry TerMur/Trammel points. The facet comes
|
||
from each record's own `<Map>`, never the file name, and the artifact shards 6 ways.
|
||
|
||
**2. The XML dependency call: hand-rolled, zero deps.** §6 left `fast-xml-parser` vs a ~120-line
|
||
tokenizer open. Resolved as the tokenizer — a deliberate *subset* parser covering only what these
|
||
files use. The server keeps zero XML dependencies at any tier.
|
||
|
||
**3. Facet names disagree between sources — a silent failure.** `Data/Locations/*.xml` spells them
|
||
`Ter Mur` and `Tokuno Islands`; `<Map>` and `<Facet name>` say `TerMur` and `Tokuno`. Unreconciled,
|
||
the landmark bucket is keyed differently from the points looking it up, so the fallback never fires
|
||
and **every unregioned spawn in Ter Mur and Tokuno reads "Wilderness"** — a plausible-looking atlas
|
||
that is quietly wrong for two facets. All facet names now pass through `normalizeFacet()`.
|
||
|
||
**4. Spawn type tokens carry XmlSpawner directives.** `<Objects2>` types are not always bare class
|
||
names: `Fairy,{RND,4,8}`, `alchemist/z/-50`, `Agralem/Name/Agralem`, `greatape,true`. Taken literally
|
||
they invent creatures that do not exist *and* split real ones in two, since `Fairy` and
|
||
`Fairy,{RND,4,8}` slug apart. 71 of 845 entries were affected; stripping at the first `/` or `,`
|
||
leaves **800** real creatures. (The design's "~1,500 creature rows" estimate was high; 800 only
|
||
reinforces the plain-`INDEX`-not-`FULLTEXT` call.)
|
||
|
||
**5. The artifact would have been 1.41 MB, not "well under 1 MB" — and is now moot.** Dropping the
|
||
unused `<Points>` fields as the design directed still left 4.40 MB; three further encodings brought
|
||
it to 1.41 MB, and getting under 1 MB would have meant dropping the spawner `name`. The size budget
|
||
in §6 was simply optimistic for 6,455 points. Superseded by §6.1 R1: there is no artifact, so there
|
||
is no payload to budget and no encode/decode seam to keep in sync.
|
||
|
||
**6. `DELETE`, not `TRUNCATE`.** The design said "TRUNCATE + batched INSERT in one transaction",
|
||
which does not hold: `TRUNCATE` is DDL in MariaDB and implicitly commits, so a mid-import failure
|
||
would leave the atlas half-loaded. `DELETE` is transactional, and at ~7k rows the cost is
|
||
irrelevant. Point ids are also assigned explicitly rather than by `AUTO_INCREMENT`, because the
|
||
join rows need them and `conn.batch()` reports no usable `insertId`.
|
||
|
||
**Measured result:** 6,455 points, 800 creatures, 23,927 point/type rows, 387 regions, 558
|
||
landmarks, 25 champion altars. The placement transform resolves **83.2%** of points (3,689 by
|
||
region, 1,690 by landmark, 1,086 Wilderness).
|
||
|
||
**One thing the design got exactly right:** the point-in-rect transform really is the reason to
|
||
build this. "Where does a lizardman spawn?" answers *Shrines, Isamu-Jima, Yew* across three facets.
|
||
|
||
### 6.3 What the API/client half added
|
||
|
||
The second website PR built the six public routes, the five admin ones, `/site/atlas` +
|
||
`/site/atlas/:slug`, and the Admin → Spawn Atlas panel. Three things it changed or established:
|
||
|
||
**1. Respawn delays were being read in the wrong unit — sometimes.** XmlSpawner writes
|
||
`MinDelay`/`MaxDelay` in minutes and switches to seconds only when a delay does not divide into
|
||
whole minutes, flagging that per record with `DelayInSec`
|
||
(`XmlSpawner2.cs:7462-7480`, read back at `:6345-6358`). So a `5` means five *minutes* on one
|
||
spawner and five *seconds* on the next, both plausible, and the pipeline stored the raw number.
|
||
170 of 6,455 stock spawners are second-flagged — few enough to look like noise on a page and be
|
||
believed. The parser now normalises to **seconds**, and the API and UI carry seconds throughout.
|
||
*This is the class of bug §6.2 is a list of: the atlas still builds, it is just quietly wrong.*
|
||
|
||
**2. The hash gate needed a parser version, and this generalises.** Fixing the parse exposed that
|
||
"has the tree changed?" is the wrong question on its own — an install whose maps never change would
|
||
have kept serving the old readings forever, because the only thing compared was the tree.
|
||
`spawnAtlasSource.PARSER_VERSION` is stored in `shard_atlas_meta` beside the source hashes, and a
|
||
mismatch counts as drift. Any future parse correction lands on the next boot without an operator
|
||
having to know it happened. **Bump it whenever the parser derives different data from identical
|
||
files.**
|
||
|
||
**3. `points` is a count; `spawners` is the list.** The first cut of the detail route spread the
|
||
creature row and then set `points` to the array of spawn points — the same key meaning a number on
|
||
the search route and an array on the detail route. Renamed before it shipped, and worth recording
|
||
because the two names are one letter apart in meaning and it reads as correct.
|
||
|
||
**On projection.** The `atlas` feature declares no sensitive fields, so `projectFeature` is a no-op
|
||
on every one of these routes today. Every handler calls it anyway, per §3.6.1's rule — the point of
|
||
the rule is that the *first* field that needs gating is covered by construction rather than by a
|
||
retrofit nobody remembers to do.
|
||
|
||
---
|
||
|
||
## 7. Part B/2 — `points.board` ✅ Done
|
||
|
||
*Landed on `edge`: servuo-plugins [#4](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/4),
|
||
link [#18](https://gitea.whitlocktech.com/RunicGateway/link/pulls/18), website [#114](https://gitea.whitlocktech.com/RunicGateway/website/pulls/114),
|
||
docs [#69](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/69). Verified against the real ServUO tree
|
||
per §11 — see §7.5 for what that run changed.*
|
||
|
||
Two deliverables: a diff sweep for the boards, and a `points` block folded into `char.profile` —
|
||
the `PROTOCOL_2.md` §10.3 `titles` precedent (read-model enrichment, no new request kind).
|
||
|
||
### 7.1 Plugin
|
||
|
||
NEW `BridgePoints.cs`, copying the `BridgeHousing.cs` diff-sweep shape (`Initialize` →
|
||
`ServerStarted`, `Connected_Core += OnConnected` clearing `_last` + `Rearm()`, `SweepOnce()`,
|
||
`Status()`, skip when `!BridgeLink.Connected`, try/catch throughout).
|
||
|
||
Which systems: default to `PointsSystem.Systems` filtered to `ShowOnLoyaltyGump == true` — reuse the
|
||
shard's own "this is player-facing" signal rather than inventing one. `Bridge.cfg PointsSystems=`
|
||
overrides. Null-guard `PointsSystem.Systems`; it is a mutable static populated by 25 separate
|
||
subsystem constructors.
|
||
|
||
**The perf trap.** `PlayerTable` is a plain `List<PointsEntry>`, and `QueensLoyalty` has `AutoAdd`, so
|
||
it can hold an entry for every `PlayerMobile` that ever existed. A naive
|
||
`.OrderByDescending().Take(N)` across 25 systems is 25 full sorts — at 20,000 historical characters,
|
||
~7.5 M comparisons, tens of ms on the Core thread. `BRIDGE_PLUGIN_PLAN.md` §1 found that nothing
|
||
except bulk profile generation comes close to a frame budget; this would be the second thing that
|
||
does.
|
||
|
||
**Mitigation — single-pass bounded selection** into a fixed N-element sorted array (N=10): O(n·N) with
|
||
tiny constants and one allocation. Skip `Player == null || Deleted` and `Points <= 0`. ~500 k cheap
|
||
iterations at a 300 s interval.
|
||
|
||
Diff signature per system: `concat(serial + ":" + (long)points)` over the top N, plus the entry count.
|
||
**No `points.remove`** — the system set is fixed, the same argument `city.update` already uses.
|
||
|
||
### 7.2 Payload — one frame per system
|
||
|
||
25 × ~600 B rather than one 12 KB frame, matching `champ.update` / `guild.update`:
|
||
|
||
```jsonc
|
||
{"t":…,"kind":"points.board","system":"QueensLoyalty",
|
||
"nameString":"Queen's Loyalty","nameNumber":1114938,
|
||
"maxPoints":30000,"showOnGump":true,"players":842,
|
||
"top":[{"rank":1,"serial":"0x1A2B","name":"Darrow","points":29500}, …]}
|
||
```
|
||
|
||
`nameString` **and** `nameNumber` are both emitted (a `TextDefinition` may be a cliloc), resolved
|
||
website-side — the contract `titles.reward` already documents at `BridgeProfile.cs:107-110`.
|
||
|
||
**Entries are written inline as `{serial, name}` — never via `BridgeJson.Actor`.** Deliberate even
|
||
though the website can now reveal fields by rung: `acct`/`webId` are not needed here, because the
|
||
website resolves serial→user from its own `shard_account_links` mirror for staff views. Keep the wire
|
||
minimal.
|
||
|
||
### 7.3 `char.profile` enrichment
|
||
|
||
`BridgeProfile.cs` gains `WritePoints(sb, m)` alongside `WriteTitles`:
|
||
`"points":[{system,nameString,points,maxPoints}]`, omitting systems with no entry or 0 points.
|
||
|
||
**Deliberately no `rank`** — computing it means scanning each system's `PlayerTable` once per profile
|
||
(25 × n), which would dominate the measured 0.069 ms/profile budget. The website derives rank from
|
||
the board when the character appears in the top N. Gate behind `PointsProfileRank=false` if it is
|
||
ever wanted.
|
||
|
||
### 7.4 Config, sidecar, website
|
||
|
||
`Bridge.cfg`: `PointsSweepSeconds=300`, `PointsLeaderboardEnabled=true`, `PointsTopN=10`,
|
||
`PointsSystems=` (blank ⇒ auto), `PointsProfileEnabled=true`, `PointsProfileRank=false`.
|
||
`BridgeBoot.cs`: `Rearm()` in `reload`, `SweepOnce()` in `sweepnow`, `Status()` in both.
|
||
|
||
Sidecar — `points_boards(system PK, name, json, updated_t)`; `main.rs` arm keyed on `system`;
|
||
`GET /points` and `GET /points/:system`.
|
||
|
||
Website — `shard_points_boards(system PK, name, name_cliloc, max_points, players, show_on_gump,
|
||
payload JSON, t)`. **The top-N list stays in `payload`** — a fixed-size list read whole, exactly like
|
||
`shard_governors.candidates`. Do not normalize into a `shard_points_entries` table until a
|
||
per-character reverse lookup is actually needed. `shardIngest.js` → `upsertPointsBoard`, **not** in
|
||
`LOGGED_KINDS` (board state, like `guild.update`). `KIND_FEATURE['points.board'] = 'leaderboards'`,
|
||
with `characterName` as its per-field rule. `GET /public/shard/points` and `/points/:system` behind
|
||
`requireFeature('leaderboards')`; validate `system` ≤ 48 chars.
|
||
|
||
**No new player route** — per-character points ride inside `char.profile`, already served by
|
||
`GET /player/shard/char/:serial` with its `shardLinks.ownsAccount` check.
|
||
|
||
Client — NEW `routes/public/Leaderboards.jsx` at `/site/leaderboards`; a "Loyalty & Points" section
|
||
added to `components/CharacterSheet.jsx`, one edit serving both `PlayerCharacter.jsx` and
|
||
`AdminCharacter.jsx`.
|
||
|
||
### 7.5 What the run against a real shard changed
|
||
|
||
The plan above was written from reading `PointsSystem.cs`. Booting the actual shard (ServUO 57.4, a
|
||
43,011-mobile world) and letting one sweep run corrected four things — all of them invisible to a
|
||
fake-shard test, because a fake shard emits whatever the spec says it should.
|
||
|
||
1. **`maxPoints` overflowed to `long.MinValue`.** `MaxPoints` is a `double`, and ServUO's idiom for an
|
||
uncapped system is `double.MaxValue` — which `DespiseCrystals`, `ShameCrystals` and `VoidPool` all
|
||
use. `(long)double.MaxValue` in C# is an **unchecked** conversion: it does not throw, it yields
|
||
`long.MinValue`, and the first real sweep published
|
||
`"maxPoints": -9223372036854775808` for three of the five live boards. Fixed with `Cap()` /
|
||
`Score()` converters that normalise anything unrepresentable to `0`, which is now the wire's
|
||
documented **"uncapped"** value. Worth stating plainly because it inverts the obvious reading:
|
||
**on a real shard, `maxPoints: 0` is the common case, not an edge case**, so any UI dividing by it
|
||
must special-case it.
|
||
2. **`nameString` is usually `null`.** Most systems define their `Name` as a cliloc rather than a
|
||
literal: four of the five boards on the live shard came back `nameString: null` with only
|
||
`nameNumber` set. The humanise-the-`system`-key fallback is therefore the *primary* display path,
|
||
not a defensive nicety, and both the leaderboards page and the character sheet lead with it.
|
||
3. **`GetEntry`/`GetPoints` cannot be used in the read model.** `GetEntry(from, create: false)` still
|
||
calls `AddEntry` when the system has `AutoAdd` (`PointsSystem.cs:207`) — it **mutates the world**.
|
||
Ten of the ~25 systems have `AutoAdd = true`, so a profile built with the obvious accessor would
|
||
have appended up to ten rows to the points save file every time anyone viewed a character sheet.
|
||
`BridgeProfile.WritePoints` hand-rolls a read-only scan instead, and says so loudly.
|
||
4. **`players` had to be redefined.** §7.2 called for "the entry count", but those same ten `AutoAdd`
|
||
systems hold a zero-point row per character ever created — so the raw count reports the shard's
|
||
whole census as one system's participants. It is now the number of players actually holding points,
|
||
which is both the honest number and a strictly better diff signal (it moves when someone scores,
|
||
not when someone logs in for the first time).
|
||
|
||
One deviation from the plan as written, for the same class of reason: §7.4 named the per-field
|
||
visibility rule `characterName`, but `projectValue` matches on the **literal JSON key**, and the wire
|
||
key is `name`. A rule under the descriptive name would have been silently inert — an admin tightening
|
||
character names would have got no enforcement and no error, exactly the failure §3.6.1 records for the
|
||
flattened `ownerAcct`. `FEATURES.leaderboards.fields` therefore keys on `name`, with a test that fails
|
||
if it is renamed back.
|
||
|
||
---
|
||
|
||
## 8. Part B/3 — `vendor.listing` 🟨 In review
|
||
|
||
### 8.1 It cannot be an RPC, and this is load-bearing
|
||
|
||
`rpc.rs::try_route` correlates on the **first** frame carrying a matching `reqId` and resolves a
|
||
single `oneshot`. A chunked reply sharing one `reqId` would deliver chunk 1 to the HTTP caller and
|
||
**leak chunks 2..N onto the broadcast feed**. `REPLY_TIMEOUT` is 10 s (the client waits 12 s), so a
|
||
whole-world snapshot could not fit regardless.
|
||
|
||
⇒ **a per-vendor diff sweep on the broadcast stream**, like `champ.update` / `house.update`. The
|
||
existing per-account `vendor.snapshot` RPC is untouched; the player portal keeps using it.
|
||
|
||
Kinds: `vendor.listing` (one frame per vendor, authoritative for that vendor) and
|
||
`vendor.listing.remove`. Payload: `serial, shopName, owner:{serial,name}, map, x, y, region, house,
|
||
count, truncated, items:[{serial,itemId,hue,amount,price,name,cliloc,child}]`.
|
||
|
||
### 8.2 Two perf traps
|
||
|
||
Measured baseline (`BRIDGE_PLUGIN_PLAN.md` §1): 30 vendors / 1,200 listings = 0.343 ms via
|
||
`pack.Items` + `GetVendorItem`; extrapolated to 500 vendors / 40,000 listings ≈ 12 ms per full pass.
|
||
Except:
|
||
|
||
1. **`VendorSearch.GetItemName(Item)` is a packet builder, not a field read.** It constructs an
|
||
`ObjectPropertyList`, calls `GetProperties`, serialises, then byte-parses the packet
|
||
(`VendorSearch.cs:681-789`) — per item. Across 40,000 items in one tick that is a
|
||
multi-hundred-millisecond stall. **Mandatory: never call it in the sweep.** Emit `itemId`, `hue`,
|
||
`amount`, `price`, `item.Name` (the plain field, null for most) and `item.LabelNumber`, resolving
|
||
display names website-side — exactly what `char.profile.equipment` already does
|
||
(`BridgeProfile.cs:173`).
|
||
2. **`VendorSearch.GetItems(PlayerVendor)` is private** (`:791`). The reusable public API is
|
||
`GetItems(Container, List<Item>)` (`:807`), which recurses into sub-containers, so real item counts
|
||
run above the top-level `pack.Items` the 0.343 ms measurement used. Budget accordingly.
|
||
|
||
### 8.3 Mitigations
|
||
|
||
- **Amortized round-robin sweep** — `MarketSweepSeconds=60`, at most `MarketSweepBatch=25` vendors per
|
||
tick, with a persistent cursor over `PlayerVendor.PlayerVendors`. Full coverage in
|
||
`ceil(vendors/25) × 60 s`, with **per-tick cost bounded independent of world size**. This is the one
|
||
genuinely new pattern versus the existing sweeps and should be flagged in review.
|
||
- **Per-vendor signature diff** (`count | Σ(serial ^ price) | x | y | shopName`), as `BridgeHousing`
|
||
does — most vendors are static, so steady-state emission is near zero.
|
||
- **`MarketMaxListings=250`**, then `"truncated":true`. `BridgeJson.Parse` caps *inbound* at 1 MB;
|
||
outbound is uncapped and `shard.rs::read_line` will allocate whatever arrives.
|
||
- On `Connected_Core`, clear `_last` **and reset the cursor**; the re-emit is self-throttled by the
|
||
round-robin window.
|
||
|
||
### 8.4 Player opt-out and privacy
|
||
|
||
**Honour `pv.VendorSearch`** — ServUO's own per-vendor opt-out, which `DoSearch` filters on (`:62`).
|
||
Skip opted-out vendors entirely; the seen-set removal then drops them from the board, so **a player
|
||
who hid their vendor in game is hidden on the website too.** Also skip `Map == null || Map.Internal`
|
||
and `Backpack == null`, matching `DoSearch`.
|
||
|
||
A vendor's shop name, owner character name and location are **already globally visible in-game** — the
|
||
stock Vendor Search gump surfaces exactly this set to any player — which is why they default to
|
||
`anonymous`. They remain per-field configurable (`ownerName`, `location`) so an admin can tighten
|
||
them. Account name and website user id never go on the wire.
|
||
|
||
### 8.5 Sidecar and website
|
||
|
||
Sidecar — one table `vendors(serial PK, shop_name, owner_name, map, x, y, region, count, json,
|
||
updated_t)` storing the whole-vendor blob. **No `vendor_items` table** — the sidecar's job here is
|
||
outage resilience (`PROTOCOL_2.md` §12.2), not search; search lives in MariaDB. Endpoint is
|
||
**`GET /market`**, not `/vendors` — axum would route the latter fine, but the collision with the
|
||
per-account RPC is a readability trap.
|
||
|
||
Website — `shard_vendors` + `shard_vendor_items` (indexes on `vendor_serial`, `price`, `item_id`,
|
||
`display_name`; delete-then-insert per vendor in one transaction; no FKs). `shardIngest.js` handles
|
||
both kinds; **not** in `LOGGED_KINDS`.
|
||
|
||
`KIND_FEATURE['vendor.listing'] = 'market'`, but the market feature's **SSE mapping is disabled by
|
||
default**: a live firehose of full vendor inventories would be the site's single biggest bandwidth
|
||
consumer, and no page needs it live. The page is a paginated DB query with a staleness stamp; an
|
||
admin can turn the stream on. `uoLinkSocket` paginates `/market` on reconnect, bounded by
|
||
`MARKET_SNAPSHOT_MAX = 5000` vendors so a pathological world cannot hang startup.
|
||
|
||
`GET /public/shard/market?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=`
|
||
(limit 1..100, default 50; `q` ≤ 60 chars; `sort ∈ {price_asc, price_desc, recent}`) and
|
||
`/market/vendors/:serial`, behind `requireFeature('market')`. **Rate-limit it** — this is the first
|
||
genuinely expensive public endpoint; `express-rate-limit` is already a dependency.
|
||
|
||
### 8.6 The open dependency — cliloc names ✅ Resolved (shipped ahead of §8)
|
||
|
||
`CharacterSheet.jsx:14-15` documented the gap ("without a cliloc table on the site we can only
|
||
show literals") and rendered equipment as `id {itemId}`. Search-by-name needs that table.
|
||
|
||
**Resolved as its own website-only change, landed BEFORE the market so `/site/market` ships with real
|
||
item names.** Full design and operator guide: [`docs/website/CLILOCS.md`](../website/CLILOCS.md).
|
||
Ingest denormalizes the resolved name into `shard_vendor_items.display_name` as planned.
|
||
|
||
Two things in the original recommendation above turned out to be wrong, and both are worth recording
|
||
because the reasoning generalises.
|
||
|
||
**1. The committed `db/data/clilocs.json` artifact was dropped.** It predates the two Part C
|
||
corrections (§6.1) and violates both: no committed snapshot of derived content, and nothing
|
||
EA-derived ever shipped. UO's strings are EA's, exactly as the creature sprites are. Replaced with
|
||
the §6 pattern instead — parse on every boot from an operator-configured path, hash-gated, output
|
||
gitignored, `PARSER_VERSION` counted as drift.
|
||
|
||
**2. `scripts/buildClilocs.js reads the UO client's Cliloc.enu` is not possible, and the reason
|
||
matters.** **Every current client ships its cliloc files COMPRESSED** — all eight `Cliloc.*` files
|
||
open with a DWORD whose high byte is `0x8E`, the "Mythic" container. The plain layout (`02 00 00 00
|
||
01 00`, then `{int32 number, byte flag, uint16 length, UTF-8}`) is what those files looked like
|
||
*before* that change. Parsing a compressed file as plain does not fail cleanly: it yields ~19k
|
||
"records" with negative ids, 1,722 distinct keys out of 19,508, one 62 KB "string", and a truncation
|
||
somewhere in the middle.
|
||
|
||
Decompressing means porting an inverse-BWT coder with a 1 KB frequency header — a few hundred lines
|
||
whose failure mode is plausible-looking garbage rather than an error. Two facts closed off the
|
||
alternatives:
|
||
|
||
- **ServUO cannot read it either.** Its bundled `Ultima.StringList` implements only the plain layout,
|
||
so on a modern client `VendorSearch.StringList` is null and `VendorSearch.GetItemName` returns
|
||
`item.Name`. **The in-game Vendor Search gump has the same gap** — which also means §8.2's warning
|
||
never to call `GetItemName` in the sweep costs us nothing we could otherwise have had.
|
||
- The shard therefore cannot supply names on our behalf, so this could not be pushed to the plugin.
|
||
|
||
⇒ **the operator converts once, from their own client, and the site reads the result.** Accepted
|
||
shapes are the plain binary layout and a `number<TAB|,|;>text` export; the site sniffs which.
|
||
`server/tools/cliloc-export/` drives UOFiddler's `Ultima.dll` (the decompressor that already exists)
|
||
and writes the plain form. A shard that never converts is fully supported — names render as ids,
|
||
exactly as before.
|
||
|
||
**Shards edit items and add new ones**, and those carry ids no stock client table has — so this reads
|
||
a **set** of sources, not one file, hash-gated together and re-read on every boot exactly as §6 reads
|
||
the ServUO tree: a base (the converted client table) plus every overlay under `custom/`, later
|
||
winning. Adding one custom item therefore never means re-exporting a 5 MB client file. Measured on
|
||
the live shard for scale: its script tree references **16,434** cliloc ids and only **37** are absent
|
||
from stock — tens against a 67k base, which is why an overlay and not a second table. `custom/` is the
|
||
one convention here that is ours rather than the shard's, because **ServUO has no server-side notion
|
||
of a custom cliloc**: they live in the patched client a shard distributes, and nothing in the tree
|
||
declares them.
|
||
|
||
That set also brings back a hazard a single file did not have, and §8.6 answers it the way §6 does. A
|
||
corrupt source fails the parse loudly, but a source that has **vanished** parses perfectly and imports
|
||
a table quietly missing everything it contributed — an unmounted volume is indistinguishable from a
|
||
deliberate deletion. So it is **staged, not applied** (`status: 'needsReview'`), reported by both the
|
||
import and `status()`, and accepted with `{approve:true}`. It is a flag rather than §6's
|
||
approve/reject pair because the atlas stores a pending decision *so that approving re-parses*; here
|
||
nothing is stored, so re-reading at approval time is automatic.
|
||
|
||
Five traps found by building it, all recorded in `CLILOCS.md`:
|
||
|
||
- **`StringList.SaveStringList` RE-COMPRESSES on save.** It looks exactly like the export path and is
|
||
not; its output is byte-identical to its compressed input, because its purpose is round-tripping a
|
||
file back into the client.
|
||
- **Trimming a text line before splitting silently drops half the table.** Roughly half of a real
|
||
cliloc table is empty strings (ids the client reserves), exported as `1005008<TAB>`. Trimming eats
|
||
the trailing separator, leaving a bare number that then looks like a header row — 55,994 of 123,490
|
||
entries vanished, and the import still looked successful.
|
||
- **`Number('')` is `0`, not `NaN`.** A line starting with a separator imports as a bogus cliloc 0
|
||
unless the empty field is rejected explicitly.
|
||
- **Tidying punctuation unconditionally corrupts real names.** Stripping leftover brackets is right
|
||
after a placeholder is removed (`[~1_stuff~]` → nothing) and wrong otherwise: a shard's custom
|
||
`"Runic Gateway Sigil (v2)"` rendered as `"(v2"`. Same shape as the `%` rule. **Found only by
|
||
running a shard-style overlay through it** — every stock-table fixture passed.
|
||
- **Source labels must be forward-slashed and root-relative**, or the same directory fingerprints
|
||
differently on Windows and Linux and every boot looks like a change. The identical bug §6 records.
|
||
|
||
Blank entries are dropped at import (123,490 parsed → **67,496** stored), which also makes the binary
|
||
and text paths converge on identical content.
|
||
|
||
### 8.7 Client
|
||
|
||
`routes/public/Market.jsx` at `/site/market`, with a *"prices last refreshed N minutes ago"* banner
|
||
driven by `staleAt` (the oldest `shard_vendors.updated_at`). The round-robin sweep means data is
|
||
inherently up to one full cycle old, and the UI must say so.
|
||
|
||
Shipped with a second page, `routes/public/MarketVendor.jsx` at `/site/market/vendors/:serial` —
|
||
where a search result points. It is the only surface that can render the two states the result list
|
||
cannot: a `truncated` shop (*"showing 250 of 3,104 — this shop holds more than the shard
|
||
publishes"*) and a `location` an admin has gated away, which is a real answer rather than an empty
|
||
coordinate.
|
||
|
||
### 8.8 What the build changed
|
||
|
||
Four things the implementation settled differently from §8 as written, all of them found by building
|
||
against the live shard.
|
||
|
||
**1. `location` is a nested object, not flat `map`/`x`/`y`/`region`.** §8.1's payload sketch had them
|
||
flat, and it would have made `market.location` — a rule Part A pre-wired — **inert**, exactly like
|
||
the `characterName` miss §7.5 records: `projectValue` matches literal JSON keys, so there is no
|
||
`location` key for the rule to match. Flat keys would have needed five rules that could drift apart.
|
||
Nesting makes one rule hide the facet, the coordinates, the region and the house together, on the
|
||
live frame and the stored read model alike, because both now spell it the same way.
|
||
|
||
The other pre-wired rule, `market.ownerName`, checked out — it is a real key on the frame. Owner is
|
||
written as flat `ownerSerial`/`ownerName` rather than through `BridgeJson.Actor`, which would add
|
||
`acct` and `webId`; same argument `points.board` makes. `ownerSerial` was **added** to the
|
||
configurable fields alongside `ownerName`, because an admin who hides the owner's name and leaves a
|
||
serial every other board resolves back to that name has not hidden anything.
|
||
|
||
**2. The per-vendor diff signature is the full listing set, not §8.3's `count | Σ(serial ^ price)`.**
|
||
That hash collides on the single most common change a shop makes: two items swapping prices, which
|
||
is what re-pricing looks like. The signature is built over the same buffer the frame is written
|
||
from, in the same order, so a match really does mean an identical frame.
|
||
|
||
**3. There is no `payload` column on `shard_vendors`.** §8.5 implied the board pattern (whole frame
|
||
in JSON, columns hoisted for display). It does not apply here: the items ARE the searchable rows, so
|
||
they are normalized into `shard_vendor_items` and there is nothing left worth duplicating. The
|
||
sidecar keeps the whole blob, because outage resilience is its job and search is not.
|
||
|
||
**4. Sweep cost is reported, and a slow tick warns.** The batch cap is a *claim* about per-tick cost,
|
||
and an operator tuning `MarketSweepBatch` was otherwise tuning blind. `[bridge status` now carries
|
||
`lastMs`/`maxMs`, and a tick over 50 ms prints a rate-limited warning naming the knob.
|
||
|
||
Measured on the live shard (27 vendors × 40 listings, 209k items / 43k mobiles):
|
||
|
||
| | |
|
||
|---|---|
|
||
| First tick — 25 vendors emitted cold | **15.4 ms** |
|
||
| Second tick — the remaining 2 | **3.4 ms** |
|
||
| Steady state — nothing changed | **0.3 ms** |
|
||
| Website `/market` search over 1,040 listings | 1,040 total, names resolved |
|
||
| Cliloc re-resolution pass over 1,040 rows | **50 ms** |
|
||
|
||
The diff is what makes the steady state ~free; the batch cap is what bounds the cold case. Note the
|
||
arithmetic the warning exists for: at the default cap of 250 listings, a batch of 25 **full** shops
|
||
is 6,250 items ≈ 95 ms — over budget. Real shops hold tens, which is why 25 is the default, but a
|
||
shard of commodity resellers should lower the batch, and now it will be told to.
|
||
|
||
Two smaller things worth not rediscovering:
|
||
|
||
- **`BridgeJson.Escape` takes a NON-NULL string** — it dereferences `value.Length` immediately — and
|
||
`BridgeJson.Str` writes its own `,"key":` prefix, so neither serves a value inside a hand-built
|
||
object. Nearly everything this frame writes is legitimately null (an item's plain `Name` is null
|
||
for almost every item; a vendor in the street has no house), so that is the common path, not an
|
||
edge case. `BridgeMarket.Text()` is the two-line writer that was missing.
|
||
- **The ServUO console writes in the OS code page**, so an em dash in a `Console.WriteLine` renders
|
||
as `???` in the log an operator would paste into an issue. Bridge console output is ASCII.
|
||
|
||
Search-side, one thing the site had to fix rather than inherit: `%` and `_` in a user's query are
|
||
**LIKE** metacharacters, not SQL ones, so parameterization does not neutralize them — a search for
|
||
`%` would otherwise match every listing on the shard. `shardMarket.db.js` escapes them. (The atlas's
|
||
`LIKE` searches predate this and have the same shape over a much smaller table; worth a follow-up,
|
||
not a blocker here.)
|
||
|
||
---
|
||
|
||
## 9. Sequencing
|
||
|
||
| Order | Part | Repos touched | Wire change | State |
|
||
|---|---|---|---|---|
|
||
| 1 | **A** — visibility framework + actor-leak fix | website, docs | none | ✅ Done |
|
||
| 2 | **B/1** — `world.ruleset` (§5) | all four | new kind | ✅ Done |
|
||
| 3 | **C** — spawn atlas (§6) | website, docs | none | ✅ Done |
|
||
| 4 | **B/2** — `points.board` (§7) | all four | new kind + `char.profile` field | ✅ Done |
|
||
| 5a | **B/3 dependency** — cliloc table (§8.6) | website, docs | none | ✅ Done |
|
||
| 5b | **B/3** — `vendor.listing` (§8) | all four | new kinds | ✅ Done |
|
||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3, `edge` → `main` | all four | the bump | 🟨 In review — `edge` → `main` held for Android parity (§10) |
|
||
|
||
---
|
||
|
||
## 10. Documentation obligations
|
||
|
||
- This file (`link/v3.md`) is the canonical 3.0 design.
|
||
- `PROTOCOL_2.md` §10.4 gains a note that `world.systems` is superseded by `world.ruleset`, and that
|
||
the deferred VvV question is answered (`VvV.cfg Enabled=True`, Factions off).
|
||
- `INTEGRATION.md` — catalog entries and §6 consumer sections for each new kind, plus the v2→v3
|
||
upgrade note for operators.
|
||
- `PLAN.md` — phasing.
|
||
- `website/BACKEND_DESIGN.md` — every new table and route, and **the visibility framework as a
|
||
security contract**: the audience ladder, the two locked rules, and the fail-closed kind map belong
|
||
in the security section.
|
||
- NEW `website/SHARD_VISIBILITY.md` — admin-facing: what each feature exposes, what each rung means,
|
||
what cannot be loosened.
|
||
- NEW `website/SPAWN_ATLAS.md`, NEW `website/CLILOCS.md`, NEW `website/MARKETPLACE.md`.
|
||
- `PROJECT_TREE.md` in each touched repo.
|
||
- `npm run swagger` **and** `npm run routes:manifest` on every route-touching PR — both are committed
|
||
artifacts, and `test/routeManifest.test.js` fails on drift.
|
||
|
||
**Android parity — now scoped, and it gates the cutover (decided 2026-07-30).** This was written as a
|
||
"track separately" follow-up. It was re-examined before the cutover and the gap is wider than nav
|
||
hiding: the app consumes the same public/player shard API but has **no consumer for any of the four new
|
||
features** (`ruleset`, `leaderboards`, `market`, `atlas`), no `points` block on its character sheet, no
|
||
cliloc-resolved item names (§8.6), and — the part that matters for §3 — **it gates shard navigation on
|
||
session role alone**, so an admin who disables a feature or raises its audience leaves the app
|
||
rendering entries that `404`/`403` into a generic error where the web client hides them.
|
||
|
||
Two things were verified as already correct and are recorded so they are not re-derived: the app's SSE
|
||
request rides the same authenticated OkHttp client as every other call, so an app session resolves to
|
||
the same audience rung as the same account on the web; and every shard DTO in the app is
|
||
nullable-with-defaults, so field projection strips fields without a deserialization failure.
|
||
|
||
Scoped as **M11 in [`../android/PLAN.md`](../android/PLAN.md) §9**, two PRs: the visibility rules +
|
||
read-model adds ([Android-app #30](https://gitea.whitlocktech.com/RunicGateway/Android-app/pulls/30))
|
||
and the four screens ([#31](https://gitea.whitlocktech.com/RunicGateway/Android-app/pulls/31), stacked
|
||
on it). Both are **built and in review**; the on-device five-rung walk (§11) against a website on the
|
||
cutover branch is the remaining gate. `edge` → `main` is held until both land, so web and app surface
|
||
the same shard on the same day. Neither PR is coupled to the merge order — on a pre-v3 website
|
||
every new route and `/public/shard/features` `404`s and the app falls back to today's behavior — so
|
||
holding the cutover is a schedule decision, not a technical dependency.
|
||
|
||
---
|
||
|
||
## 11. Verification
|
||
|
||
**Plugin** — `servuo-plugins\deploy.ps1 -ServerPath <servuo> -Verify`, inspect the ADD/CHANGE list,
|
||
then re-run without `-Verify` (ServUO must be stopped). Boot with `tools/stub_sidecar.ps1` listening
|
||
and **confirm the compile banner in the console, not merely the absence of errors** —
|
||
`BRIDGE_PLUGIN_PLAN.md` §1 warns that a failing build is silently ignored and the previous
|
||
`Scripts.dll` reloads. Then `[bridge status`, `[bridge sweepnow`, `[bridge reload`.
|
||
|
||
- §5: eyeball the emitted `world.ruleset` frame for anything sourced from `Server.cfg`, `Staff.cfg`,
|
||
`Email.cfg`, `DataPath.cfg` or `Bridge.cfg`.
|
||
- §8: with a seeded world, time one sweep tick and confirm the batch cap holds it under ~1 ms.
|
||
|
||
**Sidecar** — `cargo build && cargo clippy`; `curl -H "Authorization: Bearer <token>"
|
||
localhost:8080/ruleset` (and `/points`, `/market`); confirm `X-UOLink-Version: 3` and that a client
|
||
declaring 2 receives a 409.
|
||
|
||
**Website server** — `DB_HOST=127.0.0.1 DB_PORT=59999 node --test`. New tests, each modelled on an
|
||
existing sibling: `test/shardVisibility.test.js`, `test/shardBroadcast.visibility.test.js`,
|
||
`test/shardIngest.{ruleset,points,market}.test.js` (after `shardIngest.protocol2.test.js` — stubbed
|
||
deps, asserting routing and `logged` flags), `test/spawnAtlas.parse.test.js` (pure functions, inline
|
||
fixtures). Then `npm run routes:manifest` and `npm run swagger`, committing both.
|
||
|
||
**Full stack** — against a local MariaDB: apply `db/schema.sql` (idempotent), start the server,
|
||
confirm `uoLinkSocket` backfill logs the new snapshot lines and that `uo_link_config.protocol`
|
||
migrated to 3, then load `/site/rules`, `/site/atlas`, `/site/leaderboards`, `/site/market`.
|
||
|
||
**Visibility smoke test** — for each of the five rungs, walk every shard page and confirm gating and
|
||
field projection match the configured matrix. Same shape as the 200-routes × 5-access-levels sweep
|
||
already run for the domain split.
|
||
|
||
---
|
||
|
||
## 12. Critical files
|
||
|
||
| File | Why |
|
||
|---|---|
|
||
| `website/server/src/utils/shardBroadcast.js` | The security boundary; reworked from a static allowlist to per-connection audience filtering. **The highest-risk file in 3.0.** |
|
||
| `website/server/src/utils/shardVisibility.js` (new) | Ladder, kind→feature map, projection |
|
||
| `website/server/src/utils/shardIngest.js` | The dispatcher every new kind routes through |
|
||
| `website/server/src/model/shardState/shardState.model.js` | The `shape*` projections, including the `shapeGuild` leak §3.1 fixes |
|
||
| `servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeHousing.cs` | Cleanest copy of the diff-sweep pattern; template for `BridgePoints.cs` and `BridgeMarket.cs` |
|
||
| `link/sidecar/src/main.rs` | `PROTOCOL_VERSION` 2→3 and the board-projection match |
|
||
| `website/server/db/schema.sql` | All new `shard_*` tables plus the `uo_link_config.protocol` migration |
|