Compare commits
38 Commits
b7244a24b0
...
docs/spawn
| Author | SHA1 | Date | |
|---|---|---|---|
| 1b7da860b5 | |||
| ff1c2064a5 | |||
| 10ae129b94 | |||
| 3fb3f63f25 | |||
| e9ecdc0ecb | |||
| 09467c67b0 | |||
| b0a2207c6a | |||
| bd9718a859 | |||
| 4c0ceb1c41 | |||
| 35ad440bad | |||
| 5cb77595aa | |||
| 8b4fc439ee | |||
| bf41105ec0 | |||
| 7e8cbe1916 | |||
| 6622afe4bd | |||
| b523336313 | |||
| e4bec0caba | |||
| b2c27fb285 | |||
| 2b93529ef6 | |||
| 9f3f014f34 | |||
| 257ed2166c | |||
| bab70a3f6f | |||
| 9c0c8f902c | |||
| a3e3ca817e | |||
| 45fce7cb9f | |||
|
|
2efc32c022 | ||
| e892d80aa8 | |||
| 43d01fde03 | |||
| 9f6ad6888d | |||
| d515d42b7c | |||
|
|
c47c89c023 | ||
| d034c6f673 | |||
| a4d03bd956 | |||
| 7b699301e7 | |||
|
|
bd8adf1c54 | ||
| fec3aa0d5d | |||
| a7186e2fb9 | |||
| 6cea1c24c4 |
@@ -19,6 +19,7 @@ ci/ cross-cutting CI/quality notes
|
||||
| [BACKEND_DESIGN.md](website/BACKEND_DESIGN.md) | API contract, DB schema, security model |
|
||||
| [HERO_EDITOR.md](website/HERO_EDITOR.md) | Hero canvas editor feature spec |
|
||||
| [WIKI_UPGRADE.md](website/WIKI_UPGRADE.md) | Wiki subsystem upgrade notes |
|
||||
| [SHARD_VISIBILITY.md](website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework |
|
||||
| [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) |
|
||||
| [PROJECT_TREE.md](website/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout |
|
||||
|
||||
@@ -27,6 +28,7 @@ ci/ cross-cutting CI/quality notes
|
||||
|---|---|
|
||||
| [INTEGRATION.md](link/INTEGRATION.md) | How the website integrates with the uo-link sidecar |
|
||||
| [PROTOCOL_2.md](link/PROTOCOL_2.md) | Protocol 2.0 / 2.1 design |
|
||||
| [v3.md](link/v3.md) | Protocol 3.0 design — shard content/standings streams + the visibility framework |
|
||||
| [ADMIN_CONTROLS.md](link/ADMIN_CONTROLS.md) | Staff write-plane (kick/ban/broadcast, page queue) |
|
||||
| [SHARD_PREREQS.md](link/SHARD_PREREQS.md) | Shard-side prerequisites for the bridge |
|
||||
| [PLAN.md](link/PLAN.md) | uo-link build plan |
|
||||
|
||||
@@ -43,6 +43,20 @@ Pin the version you built against and compare it to the header (or `/health.prot
|
||||
|
||||
**v2 (Protocol 2.0)** added the account-provisioning surface (§6.x: `POST /accounts/create`, `DELETE /link/{account}`) and the `account.*` events. Outbound event kinds are **additive** — a v1 client that ignores unknown kinds keeps working against the live feed — but the new *endpoints* require a v2 sidecar. If you send `X-UOLink-Version: 1`, calls to the new endpoints are refused with the 409 above.
|
||||
|
||||
**v3 (Protocol 3.0) is being built and the version has not been bumped yet.** It is defined as *adds
|
||||
`world.ruleset`, `points.board`, `vendor.listing` / `vendor.listing.remove`*, and the bump to
|
||||
`X-UOLink-Version: 3` happens **exactly once**, at the end, when [`v3.md`](v3.md) §4's `edge` → `main`
|
||||
cutover lands — because a bump is an operator-visible hard break (409 on every protected route, and
|
||||
the website's WS closes on the `ws.hello` mismatch), so doing it per phase would break the site
|
||||
repeatedly.
|
||||
|
||||
Until then, sidecars on `edge` still report `2` while already carrying some v3 kinds and endpoints.
|
||||
That is safe in the direction that matters: event kinds are additive, and a client that ignores
|
||||
unknown kinds and tolerates a `404` on a not-yet-present endpoint keeps working. What you must **not**
|
||||
do is infer feature availability from the version number during this window — probe the endpoint, or
|
||||
treat a missing `world.ruleset` as "this shard hasn't published one". There is deliberately **no
|
||||
feature-negotiation array**: v3 implies all three kinds.
|
||||
|
||||
---
|
||||
|
||||
## 3. Health
|
||||
@@ -315,6 +329,58 @@ The house registry — one row per house, complementing the `house.decay` *trans
|
||||
|
||||
Render from `GET /houses` (§6) on connect, then keep live with these events.
|
||||
|
||||
#### Shard ruleset (Protocol 3.0)
|
||||
|
||||
How the shard is actually configured, published by the shard itself. **Not a sweep** — it changes only
|
||||
when an operator edits `Config/*.cfg`, so it is emitted once per shard↔sidecar connect (and on
|
||||
`[bridge reload`), exactly like `server.hello`.
|
||||
|
||||
| kind | fields | notes |
|
||||
|------|--------|-------|
|
||||
| `world.ruleset` | `rev`, `shard`, `expansion`, `connect?`, `systems`, `caps`, `housing`, `accounts`, `vetRewards`, `loot`, `vendors`, `champions?`, `treasureMaps`, `vvv?`, `store`, `schedule?` | The whole ruleset, always complete — **never a delta**, so the latest frame replaces the previous one outright. Every block except `shard`/`expansion` is optional and is **omitted when its system is off**, so absence means "not applicable here", not "unknown". |
|
||||
|
||||
`rev` is the shard's FNV-1a of the body: identical `rev` means the ruleset is unchanged and this frame
|
||||
is just a reconnect re-send, so a consumer can skip the write. It is deliberately **not**
|
||||
`String.GetHashCode()`, which is seeded per process and would change on every shard restart.
|
||||
|
||||
```json
|
||||
{"kind":"world.ruleset","rev":"1a2b3c4d","shard":"UOMysticmoon","expansion":"EJ",
|
||||
"systems":{"cityLoyalty":true,"vvv":true,"factions":false,"siege":false,"chat":true,
|
||||
"store":true,"dailyRares":true,"honesty":true,"shadowguard":true,
|
||||
"treasureMaps":true,"vetRewards":true,"testCenter":false},
|
||||
"caps":{"skill":1000,"totalSkill":7000,"stat":225,"str":125,"dex":125,"int":125,
|
||||
"strMax":150,"dexMax":150,"intMax":150},
|
||||
"housing":{"accountHouseLimit":1},
|
||||
"accounts":{"perIp":3,"charSlots":7,"autoCreate":true},
|
||||
"vetRewards":{"enabled":true,"rewardIntervalDays":30},
|
||||
"loot":{"feluccaLuckBonus":1000,"feluccaBudgetBonus":100,"feluccaMaxProps":11},
|
||||
"vendors":{"restockDelayMinutes":60,"maxSell":500,"economyStockAmount":500},
|
||||
"champions":{"powerScrolls":6,"statScrolls":16,"scrollChance":0.1,
|
||||
"transcendenceChance":50.0,"rankThresholds":[5,10,13]},
|
||||
"treasureMaps":{"enabled":true,"lootChance":0.01,"resetDays":30},
|
||||
"vvv":{"enabled":true,"startSilver":2000,"enhancedRules":false},
|
||||
"store":{"enabled":true,"currencyName":"Sovereigns"},
|
||||
"schedule":{"autoSaveEnabled":true,"autoSaveFrequencyMinutes":15,"autoRestartEnabled":false},
|
||||
"t":1752489280000}
|
||||
```
|
||||
|
||||
**Two things consumers get wrong.**
|
||||
|
||||
1. **`caps.skill` and `caps.totalSkill` are in tenths**, the way ServUO stores them: `1000` is `100.0`
|
||||
skill and `7000` is `700.0` total. Rendering the raw number is actively misleading. The other caps
|
||||
(`stat`, `str`, …) are plain integers.
|
||||
2. **`connect` is present only if the operator set `Bridge.PublicConnectAddress`.** The shard's real
|
||||
listen address (`Server.cfg`) is never published; nor are `Staff.cfg`, `Email.cfg`, `DataPath.cfg`,
|
||||
`Bridge.cfg`, `Compiler.cfg`, `Reports.cfg` or `Client.cfg`. The frame is built from an explicit
|
||||
allowlist in `BridgeRuleset.cs` — `Config.Entries` is never enumerated, because that would sweep in
|
||||
every key on the server.
|
||||
|
||||
Absent entirely if the shard runs `Bridge.RulesetEnabled=false` or an older plugin. Render from
|
||||
`GET /ruleset` (§6) on connect, then keep live with this event.
|
||||
|
||||
This **supersedes the `world.systems` frame** sketched in [`PROTOCOL_2.md`](PROTOCOL_2.md) §10.4 and
|
||||
never implemented; the `systems` block above is what that asked for.
|
||||
|
||||
---
|
||||
|
||||
## 5. REST — read queries
|
||||
@@ -658,6 +724,22 @@ GET /houses
|
||||
|
||||
Every house's latest snapshot — owner→houses map. Served from the sidecar's projection, kept current by the `house.*` stream (§4). Ordered by name. Survives a sidecar restart.
|
||||
|
||||
### Shard ruleset (Protocol 3.0)
|
||||
|
||||
```
|
||||
GET /ruleset
|
||||
→ { "ruleset": {"kind":"world.ruleset","rev":"1a2b3c4d","shard":"UOMysticmoon",
|
||||
"expansion":"EJ","systems":{...},"caps":{...},"accounts":{...}, ... } }
|
||||
```
|
||||
|
||||
The shard's published ruleset (§4 for the full frame and its two gotchas). Served from the sidecar's
|
||||
store, so it **answers while the shard is down** — a rules page that goes blank during a restart is
|
||||
worse than one that is briefly stale. Keep it current with the `world.ruleset` stream.
|
||||
|
||||
`{"ruleset": null}` means the shard has never published one — an older plugin, or
|
||||
`Bridge.RulesetEnabled=false`. That is a real answer distinct from a published ruleset, and worth
|
||||
rendering differently ("not published yet") rather than as an empty ruleset.
|
||||
|
||||
---
|
||||
|
||||
## 7. Status codes
|
||||
|
||||
13
link/PLAN.md
13
link/PLAN.md
@@ -315,6 +315,14 @@ Counts in `hello` are a live snapshot taken on the Core thread, not a cached val
|
||||
7. **Core edit: `PlayerVendorSale`** (§6). Then the cheat-detection feed.
|
||||
8. **Cheat signals.** `FastWalk`, `OnPropertyChanged` audit, vendor-sale anomaly detection in the sidecar.
|
||||
|
||||
**Beyond 1.0.** Phases above are the 1.0 read/event plane. Protocol 2.0's phasing (provisioning +
|
||||
world-state boards) is [`PROTOCOL_2.md`](PROTOCOL_2.md) §13; Protocol 3.0's (visibility framework,
|
||||
shard content and standings) is [`v3.md`](v3.md) §9, which also tracks what has landed. Shipped from
|
||||
3.0 so far: **Part A** — the visibility framework — and **`world.ruleset`** ([`v3.md`](v3.md) §5),
|
||||
`BridgeRuleset.cs`, the first bridge stream that is neither an event subscription nor a sweep: it is
|
||||
emitted once per connect, like `server.hello`, because shard config changes only when an operator
|
||||
edits a file.
|
||||
|
||||
### Config keys (`Config/Bridge.cfg`)
|
||||
|
||||
```ini
|
||||
@@ -328,6 +336,11 @@ EconomySweepSeconds=300
|
||||
|
||||
Read in `Configure()` via `Config.Get<T>("Bridge.<Key>", default)`. Key scope is the filename: `Bridge.cfg` + `StatSweepSeconds` → `Bridge.StatSweepSeconds`.
|
||||
|
||||
The set above is the 1.0 sample, not the current one — every later phase added keys (sweep intervals
|
||||
for each board, the town-crier/news caps, the admin write plane, account provisioning, and 3.0's
|
||||
`RulesetEnabled` / `PublicConnectAddress` / `RulesetIncludeSchedule`). **`servuo-plugins/overlay/Config/Bridge.cfg`
|
||||
is the authoritative, commented list**; `BridgeConfig.cs` holds the defaults.
|
||||
|
||||
---
|
||||
|
||||
## 11. Phase 1 acceptance
|
||||
|
||||
@@ -285,6 +285,18 @@ City titles and faction/VvV merchant titles (`CityLoyaltySystem.ApplyCityTitle`,
|
||||
|
||||
### 10.4 Factions / Vice vs Virtue
|
||||
|
||||
> **Status update (Protocol 3.0, 2026-07-28).**
|
||||
>
|
||||
> - **The deferred question is answered.** This shard runs **Vice vs Virtue** (`VvV.cfg Enabled=True`);
|
||||
> old Factions is off, and in stock ServUO that is not a coincidence —
|
||||
> `Services/Factions/Core/Faction.cs` sets `Settings.Enabled = !ViceVsVirtueSystem.Enabled`, so the
|
||||
> two are mutually exclusive by construction. The `vvv.standings` / `vvv.battle` streams below are
|
||||
> therefore unblocked, but are **not** scoped for 3.0 (see [`v3.md`](v3.md) §2 row 5).
|
||||
> - **`world.systems` is superseded by `world.ruleset`** ([`v3.md`](v3.md) §5), which shipped in 3.0.
|
||||
> It was never implemented under this name. `world.ruleset` carries the same
|
||||
> `systems{cityLoyalty, vvv, factions, …}` sub-object this section asked for, plus the rest of the
|
||||
> shard's published ruleset, so no orphan kind is left behind. Do not implement `world.systems`.
|
||||
|
||||
**Which system is live is a shard decision — verify before building.** Two exist:
|
||||
|
||||
- **Old Factions** (`Scripts/Services/Factions`): `Faction.Commander` (leader, `Faction.cs:160`), `Faction.Election`, `Faction.Members` (`List<PlayerState>`), and faction-controlled **Towns** (`Town.cs` — each town has an owning faction, a sheriff, and finance). Config-gated and, on most modern shards, **off**.
|
||||
@@ -300,7 +312,10 @@ City titles and faction/VvV merchant titles (`CityLoyaltySystem.ApplyCityTitle`,
|
||||
{"kind":"vvv.standings","order":142000,"chaos":138500,"leaderSide":"Order"}
|
||||
```
|
||||
|
||||
> Start by detecting which system is enabled at boot and streaming only that one; emit a one-time `world.systems` frame (what's on: cityLoyalty, vvv, factions) so the website renders the right panels instead of guessing.
|
||||
> Start by detecting which system is enabled at boot and streaming only that one. ~~emit a one-time
|
||||
> `world.systems` frame (what's on: cityLoyalty, vvv, factions) so the website renders the right panels
|
||||
> instead of guessing.~~ — **superseded: `world.ruleset` already carries that `systems` block** (see the
|
||||
> status note at the top of this section).
|
||||
|
||||
## 11. Further integration points — a menu to pick from
|
||||
|
||||
|
||||
740
link/v3.md
Normal file
740
link/v3.md
Normal file
@@ -0,0 +1,740 @@
|
||||
# Protocol 3.0 — Shard content, standings & the visibility framework
|
||||
|
||||
**Status:** In progress. All work lands on an `edge` branch in each repo; `edge` → `main` is the v3 cutover.
|
||||
**Date:** 2026-07-28
|
||||
**Codebase:** ServUO 57.4, `<servuo>`, net48 / x64, Expansion **EJ**.
|
||||
**Companion to** [`PLAN.md`](PLAN.md) (1.0 read/event plane), [`PROTOCOL_2.md`](PROTOCOL_2.md) (2.0 provisioning + world-state streams), [`ADMIN_CONTROLS.md`](ADMIN_CONTROLS.md) (staff write plane), [`INTEGRATION.md`](INTEGRATION.md) (website API).
|
||||
|
||||
### Progress
|
||||
|
||||
Each part is marked off here as it lands on `edge`. §9 carries the same state per sequencing row.
|
||||
|
||||
| Order | Part | State | Landed on `edge` |
|
||||
|---|---|---|---|
|
||||
| 1 | **A** — visibility framework + actor-leak fix (§3) | ✅ **Done** | website [#109](https://gitea.whitlocktech.com/RunicGateway/website/pulls/109) + [#110](https://gitea.whitlocktech.com/RunicGateway/website/pulls/110), docs [#64](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/64) + [#65](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/65) |
|
||||
| 2 | **B/1** — `world.ruleset` (§5) | ✅ **Done** | servuo-plugins [#3](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/3), link [#17](https://gitea.whitlocktech.com/RunicGateway/link/pulls/17), website [#111](https://gitea.whitlocktech.com/RunicGateway/website/pulls/111), docs [#66](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/66) |
|
||||
| 3 | **C** — spawn atlas (§6) | 🟡 **Data pipeline done** | website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112) (parsers + CLI + tables); API/client PR next |
|
||||
| 4 | **B/2** — `points.board` (§7) | ⬜ Not started | — |
|
||||
| 5 | **B/3** — `vendor.listing` (§8) | ⬜ Not started | — |
|
||||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3 (§4) | ⬜ Not started | — |
|
||||
|
||||
---
|
||||
|
||||
## 1. Why 3.0
|
||||
|
||||
A survey of the live ServUO tree against everything the bridge already surfaces end-to-end found that
|
||||
**the bridge covers live *activity* well and covers shard *content and standings* almost not at all.**
|
||||
|
||||
Covered by 1.0 + 2.0: presence/online, region transitions, char vitals + profile + roster, house
|
||||
registry + IDOC decay, champion spawns, guild board, city governors + term history, help-page queue,
|
||||
total gold supply, player-vendor sales log, deaths/murders/kills, skill gains, fame/karma, quest
|
||||
completes, staff/cheat audit, account linking + creation, town crier + news.
|
||||
|
||||
Not covered by anything: every leaderboard, every ruleset fact, every "where do I find X", and the
|
||||
entire player economy outside a player's own vendors.
|
||||
|
||||
3.0 has **three scope areas**:
|
||||
|
||||
- **A — The visibility framework (§3).** Admin-configurable, per-feature and per-field audience
|
||||
control over every shard-derived surface on the website. Ships first; the rest depends on it.
|
||||
- **B — Three new wire streams (§5, §7, §8).** `world.ruleset`, `points.board`,
|
||||
`vendor.listing`/`vendor.listing.remove`.
|
||||
- **C — One website-only feature (§6).** The spawn atlas, built from static ServUO data files with no
|
||||
wire involvement at all.
|
||||
|
||||
---
|
||||
|
||||
## 2. Survey: the full gap list
|
||||
|
||||
Recorded so the items *not* scoped for 3.0 aren't re-derived later.
|
||||
|
||||
| # | Gap | Source on the shard | Value | Cost | Status |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | **Points/loyalty leaderboards** — 25 point currencies | `Scripts/Services/PointsSystems/PointsSystem.cs` → `static List<PointsSystem> Systems`, each `List<PointsEntry>{Player,Points}` | Very high | Low | **3.0 §7** |
|
||||
| 2 | **Shard ruleset page** | `Config/*.cfg` via `Server.Config.Get` | High | Very low | **3.0 §5** |
|
||||
| 3 | **Shard-wide marketplace** | `PlayerVendor.PlayerVendors` + `VendorSearch.cs` | Very high | High | **3.0 §8** |
|
||||
| 4 | **Spawn atlas / bestiary** | `Spawns/*.xml` (6,455 spawners), `RevampedSpawns/*.xml` (333), `Data/Regions.xml`, `Data/Locations/*.xml`, `Config/ChampionSpawns.xml`, `Data/teleporters.csv`, `Data/HarvestLocs/*` | High | Medium | **3.0 §6** |
|
||||
| 5 | VvV standings + battle status | `Services/ViceVsVirtue/{ViceVsVirtueSystem,GuildStats,VvVBattle}.cs` | High | Medium | `PROTOCOL_2.md` §10.4 deferred this pending "which PvP system does this shard run?" — **now answered: `VvV.cfg Enabled=True`, `Factions.cfg` off.** Unblocked, not scoped here |
|
||||
| 6 | Skill leaderboards + shard census | `Services/Reports/Reports.cs` → `GetSkillDistribution()`, `CompileGeneralStats()`, `StaffHistory` | High | Low | Spec'd `PROTOCOL_2.md` §14 (Part B phase 6), unbuilt. Shares §7's UI — fold in after |
|
||||
| 7 | Custom mounts/pets codex — ~35 across 4 tiers | `Scripts/Custom/{Companions,Legendary,Named,New Legacy}` | Medium-high | Very low | Pure wiki/CMS content, zero bridge work. The shard's most distinctive content, with zero site presence |
|
||||
| 8 | Community Collections progress | `Services/CommunityCollections/CollectionsSystem.cs` | Medium | Low | Natural public "community goal" widget |
|
||||
| 9 | Seasonal/holiday event calendar | `Services/Seasonal Events/SeasonalEventSystem.cs`, Krampus, Forsaken Foes | Medium | Low | "What's live now / what's next" |
|
||||
| 10 | Crafting / taming / harvesting feeds | `EventSink.CraftSuccess` / `TameCreature` / `ResourceHarvestSuccess` | Medium | Low | Spec'd `PROTOCOL_2.md` §11 #3/#4/#5, unbuilt |
|
||||
| 11 | Virtue progression | `EventSink.VirtueLevelChange`, `Services/Ethics/` | Medium | Low | Spec'd §11 #6, unbuilt |
|
||||
| 12 | Bulk Order Deeds + reward tables | `Services/BulkOrders/`, `Data/Bulk Orders/*` | Medium | Low | Feed spec'd §11 #7; the static reward tables are a free wiki page |
|
||||
| 13 | Guild wars | war state on `Guild` | Low-medium | Low | Spec'd §11 #8, unbuilt |
|
||||
| 14 | Astronomy discovery log | `Services/Astronomy/AstronomySystem.cs` (104 KB save) | Low | Low | Niche completion leaderboard |
|
||||
| 15 | In-game chat relay | `Services/Chat/`, `Logs/Chat/{General,Help,Trade,LFG}` | Low | Medium | Privacy-sensitive; staff-only at most |
|
||||
| 16 | Shard health telemetry | Crash logs, `LayerConflict.log`, `throttle.log`, `world.save.after` counts, AutoSave/AutoRestart schedule | Low-medium | Low | `world.save.after` is already ingested but never charted — world-size-over-time is nearly free |
|
||||
| 17 | Ultima Store / Sovereigns balance | `Store.cfg Enabled=True, CurrencyName=Sovereigns`; `UltimaStore.GetCurrency` | ? | Medium | Only worth it if sovereigns are actually sold |
|
||||
|
||||
**Excluded permanently** — see [`ADMIN_CONTROLS.md`](ADMIN_CONTROLS.md) Tier H/N: firewall/IP-block,
|
||||
kill/resurrect, jail, item/gold grants, set-access-level, arbitrary `[set`/`[add`.
|
||||
|
||||
**Noticed during the survey, out of scope:** `Scripts/Custom/PerryOwnerFix.cs` hardcodes an
|
||||
`EventSink.Login` hook granting `AccessLevel.Owner` to account `"ShardOnwerPerry"`. Worth reviewing
|
||||
independently of this work.
|
||||
|
||||
---
|
||||
|
||||
## 3. Part A — The visibility framework ✅ Done
|
||||
|
||||
*Landed on `edge`: website [#109](https://gitea.whitlocktech.com/RunicGateway/website/pulls/109) (the framework)
|
||||
and [#110](https://gitea.whitlocktech.com/RunicGateway/website/pulls/110) (the REST-projection gap §3.6.1
|
||||
records), docs [#64](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/64) + [#65](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/65).
|
||||
Smoke-tested across all five rungs per §11.*
|
||||
|
||||
### 3.1 The leak this replaces (verified 2026-07-28)
|
||||
|
||||
`BridgeJson.Actor()` (`BridgeJson.cs:85-117`) writes `serial`, `name`, **`acct`**, **`webId`**,
|
||||
`player`. `shardState.model.js:346 shapeGuild()` returns `r.payload` verbatim, and
|
||||
`GET /api/v1/public/shard/guilds` (anonymous, `shard.controller.js:131`) serves it. **A guild
|
||||
leader's game account name and website user id are readable on an anonymous public endpoint today.**
|
||||
The same path exists for `shapeGovernor` → `/public/shard/governors`. `Actor` also feeds
|
||||
`guild.join`, `city.update` and `region.enter`, all three in `PUBLIC_KINDS` on the anonymous SSE
|
||||
stream.
|
||||
|
||||
The framework below is the vehicle for the fix, and the reason it ships before anything else.
|
||||
|
||||
### 3.2 Where visibility lives
|
||||
|
||||
**On the website, never in the sidecar.** The sidecar's job for 3.0 is unchanged in character: accept
|
||||
frames, persist them to its SQLite store, forward them verbatim over WS, and serve store-backed reads
|
||||
that survive a shard outage. It defines no access parameters, no audiences, no field projection, and
|
||||
advertises no capabilities.
|
||||
|
||||
### 3.3 The audience ladder
|
||||
|
||||
`anonymous → logged_in → player → staff → admin`, each rung implying the ones below it.
|
||||
|
||||
`viewerLevel(req)` resolves: no session ⇒ `anonymous`; authenticated ⇒ `logged_in`; authenticated
|
||||
with a linked shard account ⇒ `player`; moderator/admin role ⇒ `staff`/`admin`. **Staff always
|
||||
satisfy the `player` rung** even without a linked game account, consistent with the existing rule
|
||||
that `/player/*` is role-agnostic self-service.
|
||||
|
||||
### 3.4 Two limits an admin cannot override
|
||||
|
||||
1. **`acct` and `webId` are admin-only, always.** They are not in-game-visible and are not exposed as
|
||||
configurable fields.
|
||||
2. **A kind absent from the kind→feature map is never broadcast below `admin`.** Fail closed. This
|
||||
preserves the property that today's static `PUBLIC_KINDS` allowlist is a security boundary rather
|
||||
than a convenience filter.
|
||||
|
||||
### 3.5 Configuration
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS shard_feature_visibility (
|
||||
feature VARCHAR(48) NOT NULL PRIMARY KEY,
|
||||
enabled TINYINT(1) NOT NULL DEFAULT 1,
|
||||
audience VARCHAR(20) NOT NULL DEFAULT 'anonymous',
|
||||
field_rules JSON NULL, -- {"<field>": "<rung>"} for sensitive fields only
|
||||
updated_by INT NULL,
|
||||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
```
|
||||
|
||||
**Not** seeded on boot (this changed during implementation): an **absent row means "use the compiled
|
||||
default"**, so the table starts empty and only ever holds rows an admin has actually touched. The
|
||||
defaults live in one place — `FEATURES` in `shardVisibility.js` — instead of being duplicated into a
|
||||
seeder that could drift from it, and a DB blip degrades to those same defaults rather than to
|
||||
"everything is public". **All ten shard features are covered — the four new ones and the six that
|
||||
already ship — and every default reproduces today's behavior, so the retrofit is a no-op until an
|
||||
admin changes something.**
|
||||
|
||||
| Feature | Default audience | Sensitive fields (default rung) |
|
||||
|---|---|---|
|
||||
| `status`, `activity`, `champs`, `guilds`, `governors` | `anonymous` | guilds/governors: `leaderAcct` / `leaderWebId` → **admin (locked)** |
|
||||
| `houses` | `anonymous` | `owner` → `staff`, `price` → `staff` (matches today's IDOC-only public view) |
|
||||
| `presence` | `anonymous` | `location` → `staff` (matches today's staff-only, location-gated `/online`) |
|
||||
| `ruleset` (new) | `anonymous` | `connect` → `anonymous` |
|
||||
| `atlas` (new) | `anonymous` | — |
|
||||
| `leaderboards` (new) | `anonymous` | `characterName` → `anonymous` |
|
||||
| `market` (new) | `anonymous` | `ownerName` → `anonymous`, `location` → `anonymous` |
|
||||
|
||||
### 3.6 Enforcement — three points, one config
|
||||
|
||||
New `website/server/src/utils/shardVisibility.js`:
|
||||
|
||||
- `LADDER = ['anonymous','logged_in','player','staff','admin']`, `rank()`, `meets(viewer, required)`
|
||||
- `viewerLevel(req)` (§3.3)
|
||||
- `KIND_FEATURE` — every event kind → its feature; unmapped ⇒ admin-only (§3.4)
|
||||
- `getConfig()` — DB-backed, cached ~5 s like `uoLinkClient`'s config cache, busted on admin `PUT`
|
||||
- `requireFeature(name)` — **404 when disabled** (don't leak existence), **403 when enabled but the
|
||||
viewer is below the audience**
|
||||
- `projectFeature(name, payload, viewerLevel)` — strips fields whose rung the viewer doesn't meet;
|
||||
`acct`/`webId` always stripped below `admin`
|
||||
|
||||
Applied at:
|
||||
|
||||
1. **Routes** — `requireFeature(…)` on every `/public/shard/*`, `/public/atlas/*` and the
|
||||
shard-derived player routes; `projectFeature` in the controllers, replacing the ad-hoc
|
||||
`shapeGuild`-returns-payload-verbatim path.
|
||||
2. **SSE** — `shardBroadcast.js` moves from *"one public channel with a static `PUBLIC_KINDS`
|
||||
allowlist plus one admin channel"* to **per-connection filtering**: each subscriber carries its
|
||||
`viewerLevel`; each frame is mapped kind→feature, gated on `enabled && meets(...)`, then passed
|
||||
through `projectFeature` before write. `PUBLIC_KINDS` becomes the seed data for `KIND_FEATURE`
|
||||
rather than a hardcoded gate. **This is the largest single change in Part A and where the security
|
||||
boundary now lives.**
|
||||
3. **Nav** — `GET /api/v1/public/shard/features` returns only the features the calling viewer can
|
||||
see, so the SPA hides nav entries rather than rendering links that 403.
|
||||
|
||||
### 3.6.1 What the first implementation missed (found by the §11 smoke test, fixed)
|
||||
|
||||
Part A shipped enforcement on the SSE path and on `/guilds` + `/governors`, but the **remaining public
|
||||
REST reads never called into it** — so the same event was projected live and served verbatim from
|
||||
history. Recorded because each miss is a shape the next phase can repeat:
|
||||
|
||||
- **`/public/shard/feed` returned the stored payload as-is.** `actor.acct` / `actor.webId` were
|
||||
readable *anonymously* for every logged kind (`player.death`, `mob.killed`, `skill.gain`,
|
||||
`guild.join`, …) — broader than the §3.1 leak, which was limited to board holders.
|
||||
- **`/public/shard/idoc` returned `ownerAcct`.** Rule 1 keyed on the exact strings `acct`/`webId`,
|
||||
but `shapeHouse` flattens the actor into `ownerAcct` / `ownerName` / `ownerSerial`. The lock is now
|
||||
on the field's **meaning** — a key that is or ends in `acct`/`webId`, case-insensitively — so
|
||||
flattened spellings are covered and unwritten shapes fail closed.
|
||||
- **The `houses` field rules were dead config.** Neither `getIdoc` nor `getHouses` projected, so the
|
||||
panel offered toggles that did nothing. **Every feature's declared fields must name the keys the
|
||||
read model actually emits**, not just the wire frame's.
|
||||
- **`/feed` filtered on `PUBLIC_KINDS`**, a module-load constant derived from the compiled defaults,
|
||||
so live audience changes never reached it. `visibleKinds(level, config)` resolves the readable set
|
||||
from live config; it deliberately ignores the `stream` flag, which governs SSE fan-out only (market
|
||||
history stays readable with its firehose off).
|
||||
- **`shardEvents.db.list` treated an empty `kinds` array as "no filter"** and fell through to an
|
||||
unfiltered `SELECT`. A fully-gated config would have dumped the whole event log, staff audit
|
||||
included. An empty allowlist now serves nothing.
|
||||
- **`projectValue` recursed into every object**, so a `Date` column came back as `{}`. It walks
|
||||
arrays and plain objects only. The unit tests used JSON fixtures and could not have caught this —
|
||||
the live read did, which is the argument for §11's smoke test over tests alone.
|
||||
|
||||
**The rule this leaves behind:** *a read path that returns shard data and does not call
|
||||
`projectFeature` is a bug.* Every new surface in Parts B and C — `/ruleset`, `/points`, `/market`,
|
||||
`/atlas` — must project, and must gate its kind set on live config rather than on `PUBLIC_KINDS`.
|
||||
|
||||
### 3.7 Admin surface
|
||||
|
||||
`GET` / `PUT /api/v1/admin/shard/visibility` (admin-only). Validate feature names against the known
|
||||
set and rungs against the ladder; reject any attempt to set a locked field below `admin` — including
|
||||
its flattened spellings (`ownerAcct`, `leaderWebId`), see §3.6.1. Writes an
|
||||
`admin.audit`-style row so visibility changes are traceable. New client panel
|
||||
`routes/admin/ShardVisibility.jsx` at `/admin/shard-visibility`, linked from `ShardAdmin.jsx`.
|
||||
|
||||
---
|
||||
|
||||
## 4. The version bump and the rollout
|
||||
|
||||
`PROTOCOL_VERSION` **2 → 3** in `link/sidecar/src/main.rs:27`. v3 is defined as *"adds
|
||||
`world.ruleset`, `points.board`, `vendor.listing` / `vendor.listing.remove`"*.
|
||||
|
||||
A bump is an operator-visible hard cutover — `web.rs::gate` returns 409 on every protected route on
|
||||
mismatch, `uoLinkSocket.js::handleHello` closes the WS, and the website's declared version is the
|
||||
admin-set `uo_link_config.protocol` column — so it happens **exactly once**, at the end:
|
||||
|
||||
- Cut an **`edge`** branch from `main` in each of `website/`, `link/`, `servuo-plugins/`, `docs/`.
|
||||
- Every phase PRs into `edge`, never `main`. Feature branches are cut from `edge`.
|
||||
- Part A lands first, alone.
|
||||
- When all parts are built and tested, one `edge` → `main` PR per repo, merged together. **That merge
|
||||
is the v3 cutover.**
|
||||
- A schema migration sets `uo_link_config.protocol` (the existing row **and** the column default)
|
||||
from 2 to 3, so the cutover doesn't require a manual admin edit. `UOLINK_PROTOCOL` still overrides.
|
||||
- No feature-negotiation array anywhere — v3 implies all three kinds.
|
||||
|
||||
---
|
||||
|
||||
## 5. Part B/1 — `world.ruleset` ✅ Done
|
||||
|
||||
*Landed on `edge`: servuo-plugins [#3](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/3), link [#17](https://gitea.whitlocktech.com/RunicGateway/link/pulls/17), website [#111](https://gitea.whitlocktech.com/RunicGateway/website/pulls/111), docs [#66](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/66). Implementation notes worth keeping:*
|
||||
|
||||
- ***`shadowguard` is derived, not configured.*** `Shadowguard.cfg` carries only `ReadyDuration` and
|
||||
`RandomizeInstances` — there is no `Enabled` key — so the systems block reports `Core.TOL`
|
||||
(the expansion gate) instead. Same shape for `factions`: `Factions.cfg` has no `Enabled` either, and
|
||||
`Services/Factions/Core/Faction.cs` sets `Settings.Enabled = !ViceVsVirtueSystem.Enabled`, so the
|
||||
frame reads that static rather than inventing a key. **Where a system's on/off state is derived, read
|
||||
the system's own static; only read `Config.Get` where the .cfg key IS the truth.**
|
||||
- **`caps.skill` / `caps.totalSkill` are in tenths** (1000 = 100.0), the way ServUO stores them.
|
||||
Documented in `INTEGRATION.md` and converted in the client, because the raw number is actively
|
||||
misleading rather than merely unhelpful.
|
||||
- **`Config.Get` re-parses when the cached type differs.** `InternalGet<T>` caches the parsed value on
|
||||
the entry and re-parses if `entry.Object is T` fails, so reading `PlayerCaps.SkillCap` as an `int`
|
||||
where ServUO reads it as a `double` is correct (both parse) — it just re-parses. Harmless, but worth
|
||||
knowing before assuming a shared cache.
|
||||
- **The plugin CAN be compile-verified**, contrary to "no standalone build": point Roslyn
|
||||
(`dotnet sdk/*/Roslyn/bincore/csc.dll`, `/langversion:7.3`, net48 reference assemblies) at the whole
|
||||
ServUO `Scripts` tree with `overlay/Scripts/Custom/Bridge/*.cs` substituted for the deployed copy,
|
||||
excluding `Scripts/obj` and `Scripts/bin`. 6,205 files, ~40 s, and it catches every signature error
|
||||
a boot would. Worth doing before every plugin PR.
|
||||
|
||||
`PROTOCOL_2.md` §10.4 sketches a `world.systems` capability frame that was never implemented
|
||||
(`grep` returns nothing across all four repos). **`world.ruleset` subsumes it**, carrying a `systems`
|
||||
sub-object with the `cityLoyalty` / `vvv` / `factions` booleans §10.4 asked for. §10.4 is marked
|
||||
superseded; no orphan kind is left behind.
|
||||
|
||||
### 5.1 Plugin
|
||||
|
||||
NEW `servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeRuleset.cs`, modelled on
|
||||
`BridgeBoot.EmitHello` — **not** a sweep. Subscribes `BridgeLink.Connected_Core += Emit` so a sidecar
|
||||
that comes up second still learns the ruleset.
|
||||
|
||||
Built from an **explicit allowlist** of `Server.Config.Get<T>` calls. **Never enumerate
|
||||
`Config.Entries`** (`Server/Config.cs:162`) — it would sweep in secrets. An FNV-1a `rev` over the body
|
||||
makes an unchanged reconnect a site-side no-op (`String.GetHashCode()` is not stable across runs and
|
||||
must not be used).
|
||||
|
||||
`BridgeConfig.cs` + `overlay/Config/Bridge.cfg`: `RulesetEnabled=true`, `PublicConnectAddress=""`,
|
||||
`RulesetIncludeSchedule=true`. `BridgeBoot.cs`: `reload` → re-emit, `status` → rev/bytes. Not wired
|
||||
to `sweepnow`; it isn't a sweep.
|
||||
|
||||
### 5.2 Payload
|
||||
|
||||
Every block optional, omitted when its system is off:
|
||||
|
||||
`shard`, `expansion`, `connect` (only from `PublicConnectAddress`),
|
||||
`systems{cityLoyalty,vvv,factions,siege,chat,store,dailyRares,honesty,shadowguard,treasureMaps,vetRewards,testCenter}`,
|
||||
`caps{skill:1000,totalSkill:7000,stat:225,str/dex/int:125,strMax/dexMax/intMax:150}`,
|
||||
`housing{accountHouseLimit:1}`, `accounts{perIp:3,charSlots:7,autoCreate}`,
|
||||
`vetRewards{enabled,rewardIntervalDays:30}`,
|
||||
`loot{feluccaLuckBonus:1000,feluccaBudgetBonus:100,feluccaMaxProps:11}`,
|
||||
`vendors{restockDelayMinutes,maxSell,economyStockAmount}`,
|
||||
`champions{powerScrolls:6,statScrolls:16,scrollChance,transcendenceChance,rankThresholds}`,
|
||||
`treasureMaps`, `vvv{enabled,startSilver:2000,enhancedRules}`, `store{enabled,currencyName}`,
|
||||
`schedule{autoSaveFrequencyMinutes,autoRestart*}`.
|
||||
|
||||
**Excluded by name — in a code comment and here:** `Server.cfg` (Address/Listen/Port; only
|
||||
`PublicConnectAddress` is published), `Staff.cfg`, `Email.cfg`, `DataPath.cfg`, `Bridge.cfg`,
|
||||
`Compiler.cfg`, `Reports.cfg`, `Client.cfg`.
|
||||
|
||||
### 5.3 Sidecar and website
|
||||
|
||||
Sidecar — `store.rs`: singleton `ruleset(id CHECK(id=1), rev, json, updated_t)` + upsert/get;
|
||||
`main.rs`: new arm in the board-projection match; `web.rs`: `GET /ruleset` served from the store, so
|
||||
it answers during a shard outage (`PROTOCOL_2.md` §12.2).
|
||||
|
||||
Website — `uoLinkClient.getRuleset()`; `uoLinkSocket.backfill()` (object-shaped, so follow the
|
||||
`getPresence()` block's explicit form, not the array-only `snapshot()` helper); `shardIngest.js` →
|
||||
`shardState.setRuleset`, **not** in `LOGGED_KINDS` (it re-arrives every reconnect and `server.hello`
|
||||
already marks those); `KIND_FEATURE['world.ruleset'] = 'ruleset'`; `shard_ruleset` singleton table
|
||||
(`rev`, `expansion`, `payload JSON`, `t`); `GET /public/shard/ruleset` behind
|
||||
`requireFeature('ruleset')`, returning `null` ⇒ "not published yet".
|
||||
|
||||
Client — NEW `routes/public/Rules.jsx` at `/site/rules`, alongside
|
||||
`/site/champs|guilds|governors|houses`; live via `useShardFeed({ filter: new Set(['world.ruleset']) })`.
|
||||
|
||||
### 5.4 Risk
|
||||
|
||||
Perf is nil (~3 KB per connect). The only real risk is publishing a secret, mitigated by the explicit
|
||||
allowlist, the no-`Config.Entries` rule, the named exclusion list, and a manual eyeball of the emitted
|
||||
frame during verification.
|
||||
|
||||
---
|
||||
|
||||
## 6. Part C — Spawn atlas / bestiary (website-only)
|
||||
|
||||
**No plugin, no sidecar, no `Bridge.cfg` knob, no new kinds.** Not part of the v3 wire change.
|
||||
|
||||
> **Status:** data pipeline landed on `edge` — website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112)
|
||||
> (parsers, build/import CLI, tables, artifact). API + client pages are the second website PR.
|
||||
> Part C ships as **two** website PRs, not one: the parsing half is where the correctness risk
|
||||
> lives, and burying it under routes and React would have meant reviewing it in a 10k-line diff.
|
||||
> Full operator documentation: [`docs/website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md).
|
||||
>
|
||||
> **§6 below is the original design and is partly superseded.** §6.1 records two decisions that were
|
||||
> rejected in review and replaced (the committed artifact, and the fixed facet list); §6.2 records
|
||||
> the corrections the real ServUO data forced. Read both before trusting §6.
|
||||
|
||||
**Decision (revised at implementation time): the shard's ServUO tree is the single source of truth,
|
||||
re-derived on every server boot.** The original plan here was a committed generated artifact plus an
|
||||
idempotent import. That was rejected in review for two reasons, recorded in §6.1: a snapshot in the
|
||||
repo goes stale as a shard's maps change, and the design leaned on a fixed facet list that no shard
|
||||
is obliged to keep. Still not a browser-served blob; still parsed server-side only.
|
||||
|
||||
New in `website/server/`:
|
||||
|
||||
- `src/utils/spawnAtlasParse.js` — **pure functions, no fs**, so they are unit-testable in CI without
|
||||
a ServUO tree: `parseObjects2()`, `parsePoints()`, `parseRegions()`, `parseLocations()`,
|
||||
`resolveRegion()`.
|
||||
- ~~`scripts/buildSpawnAtlas.js` and a committed `db/data/spawnAtlas.*.json` artifact~~ — dropped,
|
||||
see §6.1 R1. Replaced by `src/utils/spawnAtlasSource.js` (the only thing that reads a ServUO tree,
|
||||
shared by the boot path and the CLI) and a `scripts/importSpawnAtlas.js` that is a thin CLI over
|
||||
the model. `package.json` gains `atlas:import` only.
|
||||
- `src/model/shardAtlas/{shardAtlas.db.js,shardAtlas.model.js}` following the `shardState` split.
|
||||
- `src/router/v1/public/atlas.{router,controller}.js`; `test/spawnAtlas.parse.test.js`.
|
||||
|
||||
Two parsing notes that matter:
|
||||
|
||||
- `<Objects2>` is `Type:MX=n:SB=…` segments joined by `:OBJ=` — verified against `trammel.xml`, where
|
||||
a single point carries six types. Split on `:OBJ=`; the token before the first `:` is the type.
|
||||
- **The high-value transform:** point-in-rect each spawn against the facet's `Regions.xml` rects
|
||||
(highest `priority` wins), falling back to the nearest `Data/Locations` landmark, else
|
||||
`"Wilderness"`. This is what turns *"lizardman at 5411,1234"* into ***"Despise, Felucca"*** and is
|
||||
the entire reason the page is worth building. `Regions.xml` is genuinely nested and needs a ~120-line
|
||||
recursive tokenizer **or** one devDependency (`fast-xml-parser`) — the server has zero XML deps
|
||||
today, so that is an explicit call to make at implementation time. The flat `<Points>` files need
|
||||
only regex/streaming; **do not** put 10.5 MB through a DOM parser.
|
||||
|
||||
Tables: `shard_spawn_creatures` (slug PK, name, total, facets JSON), `shard_spawn_points` (slug,
|
||||
facet, x, y, region, landmark, max_count, tod_*), `shard_regions`, `shard_landmarks`,
|
||||
`shard_champion_spawns`, `shard_atlas_meta`. Plain `INDEX` on name, **not `FULLTEXT`** — ~1,500
|
||||
creature rows makes a `LIKE` scan free, and FULLTEXT brings min-token-length trouble for names like
|
||||
"orc". No FKs, consistent with every existing `shard_*` table.
|
||||
|
||||
Routes at `/api/v1/public/atlas`, **not** under `/shard` — the atlas is static shard *content*, not
|
||||
live shard *state*; it must not look sidecar-dependent, and unlike `/shard/*` it *should* be
|
||||
`siteMode`-gated like `/posts` and `/wiki`. `GET /creatures?q=&facet=`, `/creatures/:slug`,
|
||||
`/regions`, `/landmarks`, `/champions`, `/meta`, all behind `requireFeature('atlas')`. Admin:
|
||||
`GET /admin/shard/atlas/status` (artifact-vs-DB drift) and `POST /admin/shard/atlas/import`. **Build
|
||||
stays CLI-only.**
|
||||
|
||||
Client: `routes/public/Atlas.jsx` (`/site/atlas`) and `AtlasCreature.jsx` (`/site/atlas/:slug`).
|
||||
|
||||
**Payload risk** — *superseded by §6.1 R1; nothing is committed.* The field selection it describes
|
||||
still applies at parse time: every `<Points>` field the site cannot use (`UniqueId`, all
|
||||
trigger/refractory/proximity/sequential fields, sound ids) is dropped, keeping
|
||||
Name/Map/X/Y/W/H/Range/MaxCount/MinDelay/MaxDelay/TOD*/types. Parsed data never reaches the browser;
|
||||
the browser sees only paginated API responses.
|
||||
|
||||
**Operator re-run story** — *revised by §6.1 R1.* Spawns changed → restart, or
|
||||
`npm run atlas:import` / `POST /admin/shard/atlas/import` to apply without one. `shard_atlas_meta`
|
||||
holds a sha256 per source file, so the server can tell on boot whether anything changed, and
|
||||
`GET /admin/shard/atlas/status` reports drift. If the change would remove a facet it is staged for
|
||||
approval rather than applied (§6.1 R3). Full detail in `docs/website/SPAWN_ATLAS.md`.
|
||||
|
||||
### 6.1 What implementation changed
|
||||
|
||||
Two design decisions in §6 were rejected in review and replaced; the rest are corrections the real
|
||||
ServUO data forced. Kept as a diff rather than edited in place, because each is a trap the next
|
||||
person would otherwise re-enter.
|
||||
|
||||
**R1. The committed artifact is gone — the tree is re-parsed on every boot.** §6 proposed building a
|
||||
generated artifact, committing it, and importing it. Two problems. A shard's maps change over its
|
||||
life, so a snapshot in the repo silently drifts from the world players actually see; and the build/
|
||||
import split existed only to work around the website container not having a tree, which is a
|
||||
deployment question (mount it) rather than a reason to freeze data. The server now hashes the source
|
||||
files on boot and re-derives the atlas when they differ. `scripts/buildSpawnAtlas.js`, the 1.41 MB
|
||||
artifact, and the whole encode/decode seam it needed are deleted.
|
||||
|
||||
**R2. Nothing may name a facet.** The first implementation carried a lookup table of the six stock
|
||||
UO facets to reconcile the spelling drift between sources. A shard may add facets, replace them
|
||||
outright, or rename them when its maps are updated, and a built-in list mishandles all three
|
||||
silently. Reconciliation is now by *matching* against the facet set discovered from the shard's own
|
||||
spawn and region data — exact key, then prefix in either direction — with an unmatched name keeping
|
||||
its own rather than being forced into a wrong bucket.
|
||||
|
||||
**R3. Two contracts on the boot path.** It never blocks startup: no path, an unreadable mount, a
|
||||
malformed file or a database error is caught and logged, and the site comes up serving whatever
|
||||
atlas it had. And a refresh that would REMOVE a facet is never applied automatically — facet loss
|
||||
is indistinguishable at boot from a half-copied or mid-update tree, so it is staged in
|
||||
`shard_atlas_pending` for an admin to approve or reject. Only the decision is stored (source hashes
|
||||
+ the facet diff, a few KB); approving re-parses, so what lands matches the tree at approval time.
|
||||
A rejection is remembered against those hashes so it does not re-prompt every restart.
|
||||
|
||||
### 6.2 What the build against real data changed
|
||||
|
||||
Six corrections to the design above, from running it against stock ServUO 57.4. Kept as a diff
|
||||
rather than edited in place, because each one is a trap the next person would otherwise re-enter.
|
||||
|
||||
**1. Six facets, not thirteen.** The design said `spawnAtlas.<facet>.json ×13`, assuming one facet
|
||||
per spawn file. There are 13 files but only **6** facets — `Eodon.xml`, `GravewaterLake.xml`,
|
||||
`TreasuresOfKotl.xml` and the other named-area files carry TerMur/Trammel points. The facet comes
|
||||
from each record's own `<Map>`, never the file name, and the artifact shards 6 ways.
|
||||
|
||||
**2. The XML dependency call: hand-rolled, zero deps.** §6 left `fast-xml-parser` vs a ~120-line
|
||||
tokenizer open. Resolved as the tokenizer — a deliberate *subset* parser covering only what these
|
||||
files use. The server keeps zero XML dependencies at any tier.
|
||||
|
||||
**3. Facet names disagree between sources — a silent failure.** `Data/Locations/*.xml` spells them
|
||||
`Ter Mur` and `Tokuno Islands`; `<Map>` and `<Facet name>` say `TerMur` and `Tokuno`. Unreconciled,
|
||||
the landmark bucket is keyed differently from the points looking it up, so the fallback never fires
|
||||
and **every unregioned spawn in Ter Mur and Tokuno reads "Wilderness"** — a plausible-looking atlas
|
||||
that is quietly wrong for two facets. All facet names now pass through `normalizeFacet()`.
|
||||
|
||||
**4. Spawn type tokens carry XmlSpawner directives.** `<Objects2>` types are not always bare class
|
||||
names: `Fairy,{RND,4,8}`, `alchemist/z/-50`, `Agralem/Name/Agralem`, `greatape,true`. Taken literally
|
||||
they invent creatures that do not exist *and* split real ones in two, since `Fairy` and
|
||||
`Fairy,{RND,4,8}` slug apart. 71 of 845 entries were affected; stripping at the first `/` or `,`
|
||||
leaves **800** real creatures. (The design's "~1,500 creature rows" estimate was high; 800 only
|
||||
reinforces the plain-`INDEX`-not-`FULLTEXT` call.)
|
||||
|
||||
**5. The artifact would have been 1.41 MB, not "well under 1 MB" — and is now moot.** Dropping the
|
||||
unused `<Points>` fields as the design directed still left 4.40 MB; three further encodings brought
|
||||
it to 1.41 MB, and getting under 1 MB would have meant dropping the spawner `name`. The size budget
|
||||
in §6 was simply optimistic for 6,455 points. Superseded by §6.1 R1: there is no artifact, so there
|
||||
is no payload to budget and no encode/decode seam to keep in sync.
|
||||
|
||||
**6. `DELETE`, not `TRUNCATE`.** The design said "TRUNCATE + batched INSERT in one transaction",
|
||||
which does not hold: `TRUNCATE` is DDL in MariaDB and implicitly commits, so a mid-import failure
|
||||
would leave the atlas half-loaded. `DELETE` is transactional, and at ~7k rows the cost is
|
||||
irrelevant. Point ids are also assigned explicitly rather than by `AUTO_INCREMENT`, because the
|
||||
join rows need them and `conn.batch()` reports no usable `insertId`.
|
||||
|
||||
**Measured result:** 6,455 points, 800 creatures, 23,927 point/type rows, 387 regions, 558
|
||||
landmarks, 25 champion altars. The placement transform resolves **83.2%** of points (3,689 by
|
||||
region, 1,690 by landmark, 1,086 Wilderness).
|
||||
|
||||
**One thing the design got exactly right:** the point-in-rect transform really is the reason to
|
||||
build this. "Where does a lizardman spawn?" answers *Shrines, Isamu-Jima, Yew* across three facets.
|
||||
|
||||
---
|
||||
|
||||
## 7. Part B/2 — `points.board`
|
||||
|
||||
Two deliverables: a diff sweep for the boards, and a `points` block folded into `char.profile` —
|
||||
the `PROTOCOL_2.md` §10.3 `titles` precedent (read-model enrichment, no new request kind).
|
||||
|
||||
### 7.1 Plugin
|
||||
|
||||
NEW `BridgePoints.cs`, copying the `BridgeHousing.cs` diff-sweep shape (`Initialize` →
|
||||
`ServerStarted`, `Connected_Core += OnConnected` clearing `_last` + `Rearm()`, `SweepOnce()`,
|
||||
`Status()`, skip when `!BridgeLink.Connected`, try/catch throughout).
|
||||
|
||||
Which systems: default to `PointsSystem.Systems` filtered to `ShowOnLoyaltyGump == true` — reuse the
|
||||
shard's own "this is player-facing" signal rather than inventing one. `Bridge.cfg PointsSystems=`
|
||||
overrides. Null-guard `PointsSystem.Systems`; it is a mutable static populated by 25 separate
|
||||
subsystem constructors.
|
||||
|
||||
**The perf trap.** `PlayerTable` is a plain `List<PointsEntry>`, and `QueensLoyalty` has `AutoAdd`, so
|
||||
it can hold an entry for every `PlayerMobile` that ever existed. A naive
|
||||
`.OrderByDescending().Take(N)` across 25 systems is 25 full sorts — at 20,000 historical characters,
|
||||
~7.5 M comparisons, tens of ms on the Core thread. `BRIDGE_PLUGIN_PLAN.md` §1 found that nothing
|
||||
except bulk profile generation comes close to a frame budget; this would be the second thing that
|
||||
does.
|
||||
|
||||
**Mitigation — single-pass bounded selection** into a fixed N-element sorted array (N=10): O(n·N) with
|
||||
tiny constants and one allocation. Skip `Player == null || Deleted` and `Points <= 0`. ~500 k cheap
|
||||
iterations at a 300 s interval.
|
||||
|
||||
Diff signature per system: `concat(serial + ":" + (long)points)` over the top N, plus the entry count.
|
||||
**No `points.remove`** — the system set is fixed, the same argument `city.update` already uses.
|
||||
|
||||
### 7.2 Payload — one frame per system
|
||||
|
||||
25 × ~600 B rather than one 12 KB frame, matching `champ.update` / `guild.update`:
|
||||
|
||||
```jsonc
|
||||
{"t":…,"kind":"points.board","system":"QueensLoyalty",
|
||||
"nameString":"Queen's Loyalty","nameNumber":1114938,
|
||||
"maxPoints":30000,"showOnGump":true,"players":842,
|
||||
"top":[{"rank":1,"serial":"0x1A2B","name":"Darrow","points":29500}, …]}
|
||||
```
|
||||
|
||||
`nameString` **and** `nameNumber` are both emitted (a `TextDefinition` may be a cliloc), resolved
|
||||
website-side — the contract `titles.reward` already documents at `BridgeProfile.cs:107-110`.
|
||||
|
||||
**Entries are written inline as `{serial, name}` — never via `BridgeJson.Actor`.** Deliberate even
|
||||
though the website can now reveal fields by rung: `acct`/`webId` are not needed here, because the
|
||||
website resolves serial→user from its own `shard_account_links` mirror for staff views. Keep the wire
|
||||
minimal.
|
||||
|
||||
### 7.3 `char.profile` enrichment
|
||||
|
||||
`BridgeProfile.cs` gains `WritePoints(sb, m)` alongside `WriteTitles`:
|
||||
`"points":[{system,nameString,points,maxPoints}]`, omitting systems with no entry or 0 points.
|
||||
|
||||
**Deliberately no `rank`** — computing it means scanning each system's `PlayerTable` once per profile
|
||||
(25 × n), which would dominate the measured 0.069 ms/profile budget. The website derives rank from
|
||||
the board when the character appears in the top N. Gate behind `PointsProfileRank=false` if it is
|
||||
ever wanted.
|
||||
|
||||
### 7.4 Config, sidecar, website
|
||||
|
||||
`Bridge.cfg`: `PointsSweepSeconds=300`, `PointsLeaderboardEnabled=true`, `PointsTopN=10`,
|
||||
`PointsSystems=` (blank ⇒ auto), `PointsProfileEnabled=true`, `PointsProfileRank=false`.
|
||||
`BridgeBoot.cs`: `Rearm()` in `reload`, `SweepOnce()` in `sweepnow`, `Status()` in both.
|
||||
|
||||
Sidecar — `points_boards(system PK, name, json, updated_t)`; `main.rs` arm keyed on `system`;
|
||||
`GET /points` and `GET /points/:system`.
|
||||
|
||||
Website — `shard_points_boards(system PK, name, name_cliloc, max_points, players, show_on_gump,
|
||||
payload JSON, t)`. **The top-N list stays in `payload`** — a fixed-size list read whole, exactly like
|
||||
`shard_governors.candidates`. Do not normalize into a `shard_points_entries` table until a
|
||||
per-character reverse lookup is actually needed. `shardIngest.js` → `upsertPointsBoard`, **not** in
|
||||
`LOGGED_KINDS` (board state, like `guild.update`). `KIND_FEATURE['points.board'] = 'leaderboards'`,
|
||||
with `characterName` as its per-field rule. `GET /public/shard/points` and `/points/:system` behind
|
||||
`requireFeature('leaderboards')`; validate `system` ≤ 48 chars.
|
||||
|
||||
**No new player route** — per-character points ride inside `char.profile`, already served by
|
||||
`GET /player/shard/char/:serial` with its `shardLinks.ownsAccount` check.
|
||||
|
||||
Client — NEW `routes/public/Leaderboards.jsx` at `/site/leaderboards`; a "Loyalty & Points" section
|
||||
added to `components/CharacterSheet.jsx`, one edit serving both `PlayerCharacter.jsx` and
|
||||
`AdminCharacter.jsx`.
|
||||
|
||||
---
|
||||
|
||||
## 8. Part B/3 — `vendor.listing`
|
||||
|
||||
### 8.1 It cannot be an RPC, and this is load-bearing
|
||||
|
||||
`rpc.rs::try_route` correlates on the **first** frame carrying a matching `reqId` and resolves a
|
||||
single `oneshot`. A chunked reply sharing one `reqId` would deliver chunk 1 to the HTTP caller and
|
||||
**leak chunks 2..N onto the broadcast feed**. `REPLY_TIMEOUT` is 10 s (the client waits 12 s), so a
|
||||
whole-world snapshot could not fit regardless.
|
||||
|
||||
⇒ **a per-vendor diff sweep on the broadcast stream**, like `champ.update` / `house.update`. The
|
||||
existing per-account `vendor.snapshot` RPC is untouched; the player portal keeps using it.
|
||||
|
||||
Kinds: `vendor.listing` (one frame per vendor, authoritative for that vendor) and
|
||||
`vendor.listing.remove`. Payload: `serial, shopName, owner:{serial,name}, map, x, y, region, house,
|
||||
count, truncated, items:[{serial,itemId,hue,amount,price,name,cliloc,child}]`.
|
||||
|
||||
### 8.2 Two perf traps
|
||||
|
||||
Measured baseline (`BRIDGE_PLUGIN_PLAN.md` §1): 30 vendors / 1,200 listings = 0.343 ms via
|
||||
`pack.Items` + `GetVendorItem`; extrapolated to 500 vendors / 40,000 listings ≈ 12 ms per full pass.
|
||||
Except:
|
||||
|
||||
1. **`VendorSearch.GetItemName(Item)` is a packet builder, not a field read.** It constructs an
|
||||
`ObjectPropertyList`, calls `GetProperties`, serialises, then byte-parses the packet
|
||||
(`VendorSearch.cs:681-789`) — per item. Across 40,000 items in one tick that is a
|
||||
multi-hundred-millisecond stall. **Mandatory: never call it in the sweep.** Emit `itemId`, `hue`,
|
||||
`amount`, `price`, `item.Name` (the plain field, null for most) and `item.LabelNumber`, resolving
|
||||
display names website-side — exactly what `char.profile.equipment` already does
|
||||
(`BridgeProfile.cs:173`).
|
||||
2. **`VendorSearch.GetItems(PlayerVendor)` is private** (`:791`). The reusable public API is
|
||||
`GetItems(Container, List<Item>)` (`:807`), which recurses into sub-containers, so real item counts
|
||||
run above the top-level `pack.Items` the 0.343 ms measurement used. Budget accordingly.
|
||||
|
||||
### 8.3 Mitigations
|
||||
|
||||
- **Amortized round-robin sweep** — `MarketSweepSeconds=60`, at most `MarketSweepBatch=25` vendors per
|
||||
tick, with a persistent cursor over `PlayerVendor.PlayerVendors`. Full coverage in
|
||||
`ceil(vendors/25) × 60 s`, with **per-tick cost bounded independent of world size**. This is the one
|
||||
genuinely new pattern versus the existing sweeps and should be flagged in review.
|
||||
- **Per-vendor signature diff** (`count | Σ(serial ^ price) | x | y | shopName`), as `BridgeHousing`
|
||||
does — most vendors are static, so steady-state emission is near zero.
|
||||
- **`MarketMaxListings=250`**, then `"truncated":true`. `BridgeJson.Parse` caps *inbound* at 1 MB;
|
||||
outbound is uncapped and `shard.rs::read_line` will allocate whatever arrives.
|
||||
- On `Connected_Core`, clear `_last` **and reset the cursor**; the re-emit is self-throttled by the
|
||||
round-robin window.
|
||||
|
||||
### 8.4 Player opt-out and privacy
|
||||
|
||||
**Honour `pv.VendorSearch`** — ServUO's own per-vendor opt-out, which `DoSearch` filters on (`:62`).
|
||||
Skip opted-out vendors entirely; the seen-set removal then drops them from the board, so **a player
|
||||
who hid their vendor in game is hidden on the website too.** Also skip `Map == null || Map.Internal`
|
||||
and `Backpack == null`, matching `DoSearch`.
|
||||
|
||||
A vendor's shop name, owner character name and location are **already globally visible in-game** — the
|
||||
stock Vendor Search gump surfaces exactly this set to any player — which is why they default to
|
||||
`anonymous`. They remain per-field configurable (`ownerName`, `location`) so an admin can tighten
|
||||
them. Account name and website user id never go on the wire.
|
||||
|
||||
### 8.5 Sidecar and website
|
||||
|
||||
Sidecar — one table `vendors(serial PK, shop_name, owner_name, map, x, y, region, count, json,
|
||||
updated_t)` storing the whole-vendor blob. **No `vendor_items` table** — the sidecar's job here is
|
||||
outage resilience (`PROTOCOL_2.md` §12.2), not search; search lives in MariaDB. Endpoint is
|
||||
**`GET /market`**, not `/vendors` — axum would route the latter fine, but the collision with the
|
||||
per-account RPC is a readability trap.
|
||||
|
||||
Website — `shard_vendors` + `shard_vendor_items` (indexes on `vendor_serial`, `price`, `item_id`,
|
||||
`display_name`; delete-then-insert per vendor in one transaction; no FKs). `shardIngest.js` handles
|
||||
both kinds; **not** in `LOGGED_KINDS`.
|
||||
|
||||
`KIND_FEATURE['vendor.listing'] = 'market'`, but the market feature's **SSE mapping is disabled by
|
||||
default**: a live firehose of full vendor inventories would be the site's single biggest bandwidth
|
||||
consumer, and no page needs it live. The page is a paginated DB query with a staleness stamp; an
|
||||
admin can turn the stream on. `uoLinkSocket` paginates `/market` on reconnect, bounded by
|
||||
`MARKET_SNAPSHOT_MAX = 5000` vendors so a pathological world cannot hang startup.
|
||||
|
||||
`GET /public/shard/market?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=`
|
||||
(limit 1..100, default 50; `q` ≤ 60 chars; `sort ∈ {price_asc, price_desc, recent}`) and
|
||||
`/market/vendors/:serial`, behind `requireFeature('market')`. **Rate-limit it** — this is the first
|
||||
genuinely expensive public endpoint; `express-rate-limit` is already a dependency.
|
||||
|
||||
### 8.6 The open dependency — cliloc names
|
||||
|
||||
`CharacterSheet.jsx:14-15` already documents the gap ("without a cliloc table on the site we can only
|
||||
show literals") and renders equipment as `id {itemId}`. Search-by-name needs that table.
|
||||
|
||||
- **Recommended:** `scripts/buildClilocs.js` reads the UO client's `Cliloc.enu` → committed
|
||||
`db/data/clilocs.json`; ingest denormalizes into `shard_vendor_items.display_name`. Same
|
||||
build-artifact pattern as §6, and it **also fixes the character sheet**.
|
||||
- **Fallback:** ship with item-art + price + region filters, and name search only over renamed items.
|
||||
|
||||
This decision is the reason §8 is sequenced last.
|
||||
|
||||
### 8.7 Client
|
||||
|
||||
`routes/public/Market.jsx` at `/site/market`, with a *"prices last refreshed N minutes ago"* banner
|
||||
driven by `staleAt` (the oldest `shard_vendors.updated_at`). The round-robin sweep means data is
|
||||
inherently up to one full cycle old, and the UI must say so.
|
||||
|
||||
---
|
||||
|
||||
## 9. Sequencing
|
||||
|
||||
| Order | Part | Repos touched | Wire change | State |
|
||||
|---|---|---|---|---|
|
||||
| 1 | **A** — visibility framework + actor-leak fix | website, docs | none | ✅ Done |
|
||||
| 2 | **B/1** — `world.ruleset` (§5) | all four | new kind | ✅ Done |
|
||||
| 3 | **C** — spawn atlas (§6) | website, docs | none | 🟡 Pipeline done, API/client next |
|
||||
| 4 | **B/2** — `points.board` (§7) | all four | new kind + `char.profile` field | ⬜ |
|
||||
| 5 | **B/3** — `vendor.listing` (§8) | all four | new kinds | ⬜ |
|
||||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3, `edge` → `main` | all four | the bump | ⬜ |
|
||||
|
||||
---
|
||||
|
||||
## 10. Documentation obligations
|
||||
|
||||
- This file (`link/v3.md`) is the canonical 3.0 design.
|
||||
- `PROTOCOL_2.md` §10.4 gains a note that `world.systems` is superseded by `world.ruleset`, and that
|
||||
the deferred VvV question is answered (`VvV.cfg Enabled=True`, Factions off).
|
||||
- `INTEGRATION.md` — catalog entries and §6 consumer sections for each new kind, plus the v2→v3
|
||||
upgrade note for operators.
|
||||
- `PLAN.md` — phasing.
|
||||
- `website/BACKEND_DESIGN.md` — every new table and route, and **the visibility framework as a
|
||||
security contract**: the audience ladder, the two locked rules, and the fail-closed kind map belong
|
||||
in the security section.
|
||||
- NEW `website/SHARD_VISIBILITY.md` — admin-facing: what each feature exposes, what each rung means,
|
||||
what cannot be loosened.
|
||||
- NEW `website/SPAWN_ATLAS.md`, NEW `website/MARKETPLACE.md`.
|
||||
- `PROJECT_TREE.md` in each touched repo.
|
||||
- `npm run swagger` **and** `npm run routes:manifest` on every route-touching PR — both are committed
|
||||
artifacts, and `test/routeManifest.test.js` fails on drift.
|
||||
|
||||
**Follow-up, not scoped for 3.0:** the Android app consumes the same public/player shard API and will
|
||||
need `/public/shard/features` to hide its own nav. Track separately against `android-app/`.
|
||||
|
||||
---
|
||||
|
||||
## 11. Verification
|
||||
|
||||
**Plugin** — `servuo-plugins\deploy.ps1 -ServerPath <servuo> -Verify`, inspect the ADD/CHANGE list,
|
||||
then re-run without `-Verify` (ServUO must be stopped). Boot with `tools/stub_sidecar.ps1` listening
|
||||
and **confirm the compile banner in the console, not merely the absence of errors** —
|
||||
`BRIDGE_PLUGIN_PLAN.md` §1 warns that a failing build is silently ignored and the previous
|
||||
`Scripts.dll` reloads. Then `[bridge status`, `[bridge sweepnow`, `[bridge reload`.
|
||||
|
||||
- §5: eyeball the emitted `world.ruleset` frame for anything sourced from `Server.cfg`, `Staff.cfg`,
|
||||
`Email.cfg`, `DataPath.cfg` or `Bridge.cfg`.
|
||||
- §8: with a seeded world, time one sweep tick and confirm the batch cap holds it under ~1 ms.
|
||||
|
||||
**Sidecar** — `cargo build && cargo clippy`; `curl -H "Authorization: Bearer <token>"
|
||||
localhost:8080/ruleset` (and `/points`, `/market`); confirm `X-UOLink-Version: 3` and that a client
|
||||
declaring 2 receives a 409.
|
||||
|
||||
**Website server** — `DB_HOST=127.0.0.1 DB_PORT=59999 node --test`. New tests, each modelled on an
|
||||
existing sibling: `test/shardVisibility.test.js`, `test/shardBroadcast.visibility.test.js`,
|
||||
`test/shardIngest.{ruleset,points,market}.test.js` (after `shardIngest.protocol2.test.js` — stubbed
|
||||
deps, asserting routing and `logged` flags), `test/spawnAtlas.parse.test.js` (pure functions, inline
|
||||
fixtures). Then `npm run routes:manifest` and `npm run swagger`, committing both.
|
||||
|
||||
**Full stack** — against a local MariaDB: apply `db/schema.sql` (idempotent), start the server,
|
||||
confirm `uoLinkSocket` backfill logs the new snapshot lines and that `uo_link_config.protocol`
|
||||
migrated to 3, then load `/site/rules`, `/site/atlas`, `/site/leaderboards`, `/site/market`.
|
||||
|
||||
**Visibility smoke test** — for each of the five rungs, walk every shard page and confirm gating and
|
||||
field projection match the configured matrix. Same shape as the 200-routes × 5-access-levels sweep
|
||||
already run for the domain split.
|
||||
|
||||
---
|
||||
|
||||
## 12. Critical files
|
||||
|
||||
| File | Why |
|
||||
|---|---|
|
||||
| `website/server/src/utils/shardBroadcast.js` | The security boundary; reworked from a static allowlist to per-connection audience filtering. **The highest-risk file in 3.0.** |
|
||||
| `website/server/src/utils/shardVisibility.js` (new) | Ladder, kind→feature map, projection |
|
||||
| `website/server/src/utils/shardIngest.js` | The dispatcher every new kind routes through |
|
||||
| `website/server/src/model/shardState/shardState.model.js` | The `shape*` projections, including the `shapeGuild` leak §3.1 fixes |
|
||||
| `servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeHousing.cs` | Cleanest copy of the diff-sweep pattern; template for `BridgePoints.cs` and `BridgeMarket.cs` |
|
||||
| `link/sidecar/src/main.rs` | `PROTOCOL_VERSION` 2→3 and the board-projection match |
|
||||
| `website/server/db/schema.sql` | All new `shard_*` tables plus the `uo_link_config.protocol` migration |
|
||||
@@ -1,6 +1,7 @@
|
||||
# Website API — router domain split + CSP hardening
|
||||
|
||||
Status: **in progress** — PR 0 (route manifest), CSP report-only, and split PR 1 of 5 have landed ·
|
||||
Status: **domain split complete** — PR 0 (route manifest), CSP report-only and split PRs 1–5 have all
|
||||
landed; only the CSP enforce PR remains, and it is blocked on soak data rather than on code ·
|
||||
Target repo: `website/` · Docs owner: this file + `BACKEND_DESIGN.md`
|
||||
|
||||
> **This file replaces the earlier "API v2" plan** (auth merge → CSP → domain split, with a parallel
|
||||
@@ -273,7 +274,8 @@ router/v1/
|
||||
posts.router.js pages.router.js wiki.router.js
|
||||
uploads.router.js shard.router.js uoLink.router.js
|
||||
email.router.js discordBot.router.js settings.router.js
|
||||
dashboard.router.js # + the /activity and /site-mode singletons
|
||||
activity.router.js # the staff audit log — landed in PR 2, not with dashboard
|
||||
dashboard.router.js # + the /site-mode singleton
|
||||
auth/
|
||||
login.router.js register.router.js password.router.js invite.router.js
|
||||
sso.router.js mobile.router.js me.routes.js (already split, 23 routes)
|
||||
@@ -341,6 +343,227 @@ Three findings worth carrying into PRs 2–5:
|
||||
|
||||
The OpenAPI spec was also byte-for-byte unchanged, which required a prerequisite fix — see below.
|
||||
|
||||
### PR 2 — as landed
|
||||
|
||||
`moderation`, `bot-activity` and `activity` — 18 routes, leaving 64 in the residual file.
|
||||
|
||||
| Router | Routes | Prefix | Extra gate |
|
||||
|---|---|---|---|
|
||||
| `moderation.router.js` | 15 | `/admin/moderation` | `modAccess` (admin + moderator) at router level |
|
||||
| `botActivity.router.js` | 2 | `/admin/bot-activity` | `adminOnly` per route |
|
||||
| `activity.router.js` | 1 | `/admin/activity` | none — staff-wide audit log |
|
||||
| `admin.routes.js` (residual) | 64 | group root, mounted last | unchanged |
|
||||
|
||||
All four gates came back zero-diff: `routes.manifest.json` (200 public + 2 internal),
|
||||
`routes.guards.json`, `swagger-output.json` (198 operations), and `docs/website/api-route-inventory.json`
|
||||
was already in sync. 434 server tests green.
|
||||
|
||||
Notes:
|
||||
|
||||
- **`/activity` gets its own file, deviating from the target tree above**, which parked it as a
|
||||
singleton inside `dashboard.router.js`. It is folded into PR 2 by the sequencing list, and PR 4 is
|
||||
where `dashboard` lands — so honouring the tree would have meant leaving one route in the residual
|
||||
file for two PRs to satisfy a filename. It is also a genuinely separate capability: `/activity` is
|
||||
the **staff audit log** (`activity.model.js`), while `/dashboard` is a stats overview and
|
||||
`/bot-activity` is the botScore middleware's in-memory ban state. Three different things that read
|
||||
alike. **PR 4 mounts `dashboard` and `site-mode` only.**
|
||||
- **`modAccess` moved to a router-level `use`; `adminOnly` on bot-activity deliberately did not.**
|
||||
Moderation was already gated by a *prefix* mount (`adminRouter.use('/moderation', modAccess)`), so
|
||||
`moderationRouter.use(modAccess)` is the exact equivalent (the PR 1 `users` case). Bot-activity's
|
||||
gate was per-route, and keeping it per-route is what holds the per-route handler count — the one
|
||||
number in `routes.guards.json` that would catch a dropped `adminOnly`, since `requireRole(...)`
|
||||
returns an anonymous arrow and never shows up by name. **Rule for PRs 3–5: move a gate to router
|
||||
level only where it was already a prefix mount; otherwise leave it on the route.**
|
||||
- **`modAccess` stays in the residual file** — the `/shard/*` in-game staff operations still use it
|
||||
and do not move until PR 4. Its comment there was retargeted rather than deleted.
|
||||
|
||||
### PR 3 — as landed
|
||||
|
||||
`posts`, `uploads`, `wiki` and `pages` — 31 routes, leaving 33 in the residual file. The content tier,
|
||||
and the first split PR where no gate moved at all: all four capabilities are editor-tier, so the shared
|
||||
`staffOnly` in `admin/index.js` is their whole gate.
|
||||
|
||||
| Router | Routes | Prefix | Extra gate |
|
||||
|---|---|---|---|
|
||||
| `posts.router.js` | 9 | `/admin/posts` | none — editor tier |
|
||||
| `uploads.router.js` | 1 | `/admin/uploads` | none — editor tier |
|
||||
| `wiki.router.js` | 14 | `/admin/wiki` | none — editor tier |
|
||||
| `pages.router.js` | 7 | `/admin/pages` | none — editor tier |
|
||||
| `admin.routes.js` (residual) | 33 | group root, mounted last | unchanged |
|
||||
|
||||
All four gates zero-diff: `routes.manifest.json` (200 public + 2 internal), `routes.guards.json`,
|
||||
`swagger-output.json` (198 operations), and `docs/website/api-route-inventory.json` was already in
|
||||
sync. 434 server tests green.
|
||||
|
||||
Notes:
|
||||
|
||||
- **The residual 33 is exactly PR 4's list** — `shard` 16, `email` 6, `uo-link` 5, `settings` 2,
|
||||
`discord-bot` 2, `dashboard` 1, `site-mode` 1. So `admin.routes.js` is deleted by PR 4, one PR
|
||||
earlier than the sequencing list implies, and PR 5 touches only `public/*`, `player/*` and `auth/*`.
|
||||
- **A shared module was unavoidable here, and it is the first one in the split.** The multer config
|
||||
(upload dir, mimetype→extension allowlist, 8 MB cap) was defined inline in `admin.routes.js` and used
|
||||
by *two* routes that this PR puts in different files: `POST /posts/upload` (→ `{image_url}`) and
|
||||
`POST /uploads` (→ `{url}`). It moved to `admin/imageUpload.js` rather than being duplicated —
|
||||
duplicating a security allowlist is how the two copies drift. It stays in `admin/` deliberately:
|
||||
`UPLOAD_DIR` is resolved `__dirname`-relative, so relocating the file would silently repoint the
|
||||
upload directory. Guard freshness is unaffected — multer's middleware is named `multerMiddleware`
|
||||
wherever it is constructed, so `routes.guards.json` did not move.
|
||||
- **`POST /uploads` keeps its `Admin · Posts` swagger tag**, which now disagrees with its filename. The
|
||||
acceptance criterion is a byte-identical spec, so retagging is a real OpenAPI diff and does not
|
||||
belong in a route-move PR. Same call as PR 1's `/shard/*` tag mismatch: fix tags in a PR that is
|
||||
*about* tags.
|
||||
- **The wiki router is the first one with load-bearing intra-file route order.** `/categories` and
|
||||
`/tags` are literal paths that must stay ahead of `/:slug`, or `GET /admin/wiki/categories` gets
|
||||
dispatched as a page whose slug is "categories". **The manifest cannot catch this — it sorts its
|
||||
entries, so a reordering is invisible in all three gates.** It was verified separately by
|
||||
introspecting the built router stack and asserting the last literal layer precedes the first `/:slug`
|
||||
layer. Any future PR moving `/:slug`-style routes needs the same explicit check.
|
||||
- **`/admin/pages` (CMS page builder) and `/admin/shard/pages` (in-game help-page queue) are unrelated
|
||||
capabilities that read alike** — the latter stays with `shard` in PR 4. Same trap as PR 2's
|
||||
`activity` / `dashboard` / `bot-activity` trio.
|
||||
|
||||
### PR 4 — as landed
|
||||
|
||||
`shard`, `uo-link`, `email`, `discord-bot`, `settings` and `dashboard`/`site-mode` — the whole residual
|
||||
33. **`admin.routes.js` is deleted**, so the admin group is fully split and every one of its 110 routes
|
||||
is declared in a capability router.
|
||||
|
||||
| Router | Routes | Prefix | Extra gate |
|
||||
|---|---|---|---|
|
||||
| `shard.router.js` | 16 | `/admin/shard` | none on the 7 self-service routes; `modAccess` per route on the 9 staff ops |
|
||||
| `uoLink.router.js` | 5 | `/admin/uo-link` | `adminOnly` per route |
|
||||
| `email.router.js` | 6 | `/admin/email` | `adminOnly` per route |
|
||||
| `discordBot.router.js` | 2 | `/admin/discord-bot` | `adminOnly` per route |
|
||||
| `settings.router.js` | 2 | `/admin/settings` | `adminOnly` per route |
|
||||
| `dashboard.router.js` | 2 | **group root** (`/dashboard`, `/site-mode`) | `adminOnly` per route on `/site-mode` only |
|
||||
| ~~`admin.routes.js`~~ | — | deleted | — |
|
||||
|
||||
16 + 5 + 6 + 2 + 2 + 2 = 33. All four gates zero-diff: `routes.manifest.json` (200 public + 2
|
||||
internal), `routes.guards.json`, `swagger-output.json` (198 operations), and
|
||||
`docs/website/api-route-inventory.json` was already in sync. 434 server tests green.
|
||||
|
||||
Notes:
|
||||
|
||||
- **`dashboard.router.js` is mounted at the group root, not a prefix — the one relaxation of the
|
||||
"always mount at a prefix" rule, and it is deliberate.** `GET /dashboard` and `PUT /site-mode` own no
|
||||
common path segment, so a prefix mount would mean two one-route files instead of the single file the
|
||||
target tree calls for. It is safe **only** because the file declares no router-level middleware: a
|
||||
bare `use(gate)` in a root-mounted router runs for every request passing through toward another
|
||||
mount and would 403 an editor on an unrelated route (the PR 1 finding). The file says so in a
|
||||
comment, because the next person to add a gate there is the one who needs to know.
|
||||
- **`/shard` is the first prefix where two tiers share one router**, and it is why prefix ownership
|
||||
beats swagger-tag grouping. The 7 self-service routes (`link`, `accounts`, `roster/:account`,
|
||||
`vendors/:account`, `char/:serial`, `sales`, `POST account`) are tagged `Admin · Account`, run with
|
||||
no gate beyond the shared `staffOnly`, and are served by the very same `player/shard.controller`
|
||||
handlers as `/player/shard` — staff are a superset of players, and the controller keys off
|
||||
`req.user.id`. The 9 in-game ops are tagged `Admin · Shard` and carry `modAccess`. Splitting them by
|
||||
tag would put two routers under one prefix for no gain; instead one router owns `/shard` and gates
|
||||
per route. The tag mismatch stays, on the PR 1 and PR 3 precedent: retagging is a real spec diff and
|
||||
belongs in a PR that is about tags.
|
||||
- **No gate moved to router level anywhere in this PR.** Every `adminOnly` in the residual file was
|
||||
per-route, and `modAccess` on `/shard` must stay per-route because half that router must *not* have
|
||||
it. This keeps the per-route handler count intact — the one number `routes.guards.json` can actually
|
||||
check, since `requireRole(...)` returns an anonymous arrow.
|
||||
- **The `/:param` shadowing check was run again and is clean**, since the manifest sorts and therefore
|
||||
cannot see declaration order. Introspecting the built stack, all 110 admin routes and all 59 literal
|
||||
admin paths dispatch to their own layer — nothing is captured first by a `:param` sibling. The
|
||||
near-misses worth naming: `GET /shard/pages` (help-page queue) sits alongside
|
||||
`POST /shard/pages/:id/respond|close`, and `POST /shard/towncrier` alongside
|
||||
`DELETE /uo-link/towncrier/:id` — different depths and methods, so neither collides.
|
||||
- **Deleting the file left dangling `see admin.routes.js` pointers**, which were repointed in the same
|
||||
PR: `botActivity.controller.js` → `botActivity.router.js`, `moderation.controller.js` →
|
||||
`moderation.router.js`, `announceJobs.logic.js`'s town-crier cap mirror → `admin/uoLink.router.js`,
|
||||
and the "route paths sit on the line *after* `adminRouter.get(`" rationale in
|
||||
`scripts/routeManifest.js`, `README.md` and `pr-checks.yml` was generalized (it was never about that
|
||||
one file).
|
||||
- **`/admin/shard/pages` vs `/admin/pages` stayed separate**, as PR 3 flagged: the former is the
|
||||
in-game help-page (support) queue and belongs to `shard`; the latter is the CMS page builder.
|
||||
|
||||
### PR 5 — as landed
|
||||
|
||||
`public`, `player` and the residual `auth` — 54 routes across three groups, the last split PR.
|
||||
`public.routes.js`, `player.routes.js` and `auth.routes.js` are all **deleted**, so every one of the
|
||||
200 routes in the manifest is now declared in a capability router and no monolithic route file
|
||||
remains anywhere in `router/v1/`.
|
||||
|
||||
| Group | Router | Routes | Prefix | Extra gate |
|
||||
|---|---|---|---|---|
|
||||
| `public` | `posts.router.js` | 2 | `/public/posts` | none — `siteMode` per route |
|
||||
| | `wiki.router.js` | 4 | `/public/wiki` | none — `siteMode` per route |
|
||||
| | `pages.router.js` | 2 | `/public/pages` | none — `siteMode` per route except the preview |
|
||||
| | `shard.router.js` | 12 | `/public/shard` | none — never `siteMode` gated |
|
||||
| | `site.router.js` | 4 | **group root** (`/settings`, `/status`, `/version`, `/contact`) | none |
|
||||
| `player` | `account.router.js` | 8 | `/player/account` | none beyond the group gate |
|
||||
| | `shard.router.js` | 8 | `/player/shard` | none beyond the group gate |
|
||||
| | `appeals.router.js` | 4 | `/player/appeals` | none beyond the group gate |
|
||||
| `auth` | `login.router.js` | 2 | `/auth/login` | `loginGuards` per route |
|
||||
| | `register.router.js` | 1 | `/auth/register` | `loginGuards` + `registerLimiter` |
|
||||
| | `invite.router.js` | 2 | `/auth/invite` | `loginGuards` + `registerLimiter` on accept |
|
||||
| | `password.router.js` | 3 | `/auth/password` | per-route reset limiters |
|
||||
| | `session.router.js` | 2 | **group root** (`/logout`, `/me`) | per route |
|
||||
|
||||
24 + 20 + 10 = 54. `auth/`'s other 32 routes (`me` 23, `mobile` 5, `sso` 4) were already in their own
|
||||
files and did not move. All four gates zero-diff: `routes.manifest.json` (200 public + 2 internal),
|
||||
`routes.guards.json`, `swagger-output.json` (198 operations), and `docs/website/api-route-inventory.json`
|
||||
was already in sync. 434 server tests green.
|
||||
|
||||
Notes:
|
||||
|
||||
- **Each group is now a directory with an `index.js`**, matching `admin/`: `public/index.js`,
|
||||
`player/index.js`, `auth/index.js` own the group gate (where there is one) and the mount table and
|
||||
declare no routes. `v1.router.js` requires the directories. The four tests that imported the
|
||||
deleted entry files were repointed.
|
||||
- **Two of the three groups have no group gate, and that is the security-relevant fact about them.**
|
||||
`player/index.js` carries `noindex, requireAuth` — authenticated, *any* role, because staff are a
|
||||
superset of players. `public/index.js` and `auth/index.js` carry **nothing**, deliberately: the
|
||||
public surface is anonymous by contract (logged-out SPA, Discord bot, and the Android
|
||||
`ShardStreamClient` on `/public/shard/stream`, none of which send credentials), and `/auth` is where
|
||||
an anonymous caller *becomes* authenticated. Both index files say so, because the obvious "hardening"
|
||||
edit to either one is an outage.
|
||||
- **`GET /auth/me` has a mount-order dependency, and it is the one genuinely non-obvious thing in this
|
||||
PR.** `authRouter.use('/me', meRouter)` matches the bare path `/me`, not just `/me/*` — so a request
|
||||
to `GET /auth/me` runs `meRouter`'s (and `notifRouter`'s) `noindex, requireAuth`, matches no route
|
||||
inside either, and falls through to its own handler. `session.router.js` must therefore stay mounted
|
||||
**last**. Verified by the counterfactual rather than by reading the mount table: moving the mount to
|
||||
the top of `auth/index.js` still answers `401`, but the response loses its `X-Robots-Tag` header. No
|
||||
gate file and neither manifest can see that — only a header assertion can.
|
||||
- **Two root-mounted routers, on the PR 4 `dashboard.router.js` precedent.** `public/site.router.js`
|
||||
(`/settings`, `/status`, `/version`, `/contact`) and `auth/session.router.js` (`/logout`, `/me`) hold
|
||||
the routes that own no path segment. Both are safe at the root **only** because they declare no
|
||||
router-level middleware — a bare `use(gate)` there runs for every request passing through toward
|
||||
another mount. Both files say so.
|
||||
- **`loginGuards` is the PR's one shared module**, the counterpart to PR 3's `imageUpload.js`. The
|
||||
`[backoffGuard, slowLogin, loginLimiter]` array was defined inline in `auth.routes.js` and spread by
|
||||
four routes that this PR puts in three different files — plus a fifth, already-duplicated copy in
|
||||
`sso.routes.js`. It moved to `auth/loginGuards.js` and `sso.routes.js` now imports it too, so there
|
||||
is one definition rather than five: duplicating a throttling stack is how the copies drift, and the
|
||||
copy that drifts is the one that stops throttling. It is exported `Object.freeze`d — it is
|
||||
module-level shared state, and a router that pushed onto it would silently add middleware to every
|
||||
other login surface. Guard freshness is unaffected: the same three named functions, so
|
||||
`routes.guards.json` did not move.
|
||||
- **The `/:param` shadowing check was run again and is clean.** All 86 public/player/auth routes and
|
||||
all 64 literal paths among them dispatch to their own layer. This was checked in *dispatch* order
|
||||
against the built stack, since the manifest sorts and therefore cannot see declaration order. The
|
||||
only ordering-sensitive pair is `GET /public/wiki/{categories,tags}` ahead of `/public/wiki/:slug` —
|
||||
the public twin of the `admin/wiki.router.js` trap PR 3 found, and `wiki.router.js` says so. The
|
||||
`/public/pages` preview route also stays ahead of `/:slug`, though at a different depth.
|
||||
- **Filename deviations from the target tree, both for prefix agreement.** The tree named the public
|
||||
posts router `news.router.js`; it is `posts.router.js`, matching the `/posts` prefix it owns and its
|
||||
`admin/posts.router.js` sibling. The tree also implied `sso.router.js` / `mobile.router.js`; those
|
||||
files already exist as `sso.routes.js` / `mobile.routes.js` and were not renamed — they did not move
|
||||
in this PR, and churning their names would add diff noise to a PR whose value is being reviewable.
|
||||
- **`auth/session.router.js` is a deviation the target tree did not anticipate**, the same shape as
|
||||
PR 2's `activity.router.js`. The tree listed `login.router.js` but had nowhere to put `/logout` and
|
||||
`GET /me`, which own no prefix. Folding them into `login.router.js` would have forced *that* router
|
||||
to the group root and given up prefix ownership for the four login routes; a separate root-mounted
|
||||
singleton file keeps `/login` a real prefix mount.
|
||||
- **Tag mismatches were left alone again**, on the PR 1 / PR 3 / PR 4 precedent: the acceptance
|
||||
criterion is a byte-identical spec, so retagging belongs in a PR that is about tags.
|
||||
- **`public.controller.js` was not split.** Unlike the admin controllers, it is still one file serving
|
||||
settings/status/version/contact *and* posts/wiki/pages. The plan's rule is that these PRs re-wire
|
||||
routes, not logic — splitting a controller is a separate change with a separate risk profile, and
|
||||
bundling it would have cost this PR its "pure mechanical refactor" acceptance criteria.
|
||||
|
||||
### The swagger path-normalization prerequisite (landed before PR 1)
|
||||
|
||||
swagger-autogen builds a path by string-concatenating the mount prefix with the route argument, so a
|
||||
@@ -449,11 +672,14 @@ deliberate `+1` in the manifest — which is exactly the mechanism working as de
|
||||
flipping. Also decide there whether `/api/csp-report` is retired with the report-only twin or kept
|
||||
as a `report-to` group on the enforced policy.
|
||||
4. **PR 1 — admin:** `users`, `account`, `invites`, `auth` (providers). ✅ landed
|
||||
5. **PR 2 — admin:** `moderation`, `bot-activity`, `activity`.
|
||||
6. **PR 3 — admin (content):** `posts`, `pages`, `wiki`, `uploads`.
|
||||
5. **PR 2 — admin:** `moderation`, `bot-activity`, `activity`. ✅ landed
|
||||
6. **PR 3 — admin (content):** `posts`, `pages`, `wiki`, `uploads`. ✅ landed
|
||||
7. **PR 4 — admin (ops/config):** `shard`, `uo-link`, `email`, `discord-bot`, `settings`, `site-mode`,
|
||||
`dashboard`.
|
||||
8. **PR 5 — `public/*` + `player/*`** (and the residual `auth/*` grouping).
|
||||
`dashboard`. (`activity` went with PR 2 — see § PR 2 — as landed.) **This is the whole residual
|
||||
file** — `admin.routes.js` is deleted here, not by PR 5. ✅ landed
|
||||
8. **PR 5 — `public/*` + `player/*`** (and the residual `auth/*` grouping). ✅ landed — **the domain
|
||||
split is complete.** The only remaining item in this plan is the CSP enforce PR (3), which is
|
||||
blocked on soak data, not on code.
|
||||
|
||||
Each PR: **zero-line diff in `routes.manifest.json`**, server tests green
|
||||
(`cd website/server && npm test`), Swagger regenerated, matching `docs/` edit, Conventional Commit,
|
||||
|
||||
@@ -100,7 +100,13 @@ flowchart TB
|
||||
talks to it. Every backend→sidecar call carries `Authorization: Bearer <token>` and an
|
||||
`X-UOLink-Version` header (a protocol mismatch fails fast with `409`). The REST client
|
||||
(`uoLinkClient.js`) never throws — every call returns `{ ok, data, status }` — so the site degrades
|
||||
gracefully when the shard is down.
|
||||
gracefully when the shard is down. That guarantee covers **reading the config too**: resolving the
|
||||
admin-managed config decrypts the stored auth token, which throws when the ciphertext can't be
|
||||
authenticated (`SECRET_ENC_KEY` rotated, or a DB dump restored under a different key). This is
|
||||
handled inside the client and reported as `{ ok: false, status: 0, error: 'uo-link config
|
||||
unreadable' }` plus a distinct `ERROR`-level log, so a wrong key degrades the shard surface to
|
||||
"unavailable" instead of 500ing it — and `GET /admin/uo-link/config` keeps working, which is the
|
||||
screen an admin needs to re-enter the token and recover.
|
||||
- **Two ways in from the sidecar.** Live game events arrive over an outbound **WebSocket** and are
|
||||
routed by the `shardIngest.js` dispatcher (state-changing kinds update `shard_*` tables, notable
|
||||
kinds append to `shard_events`, high-frequency kinds only update state). Point-in-time reads and
|
||||
|
||||
@@ -29,14 +29,17 @@ Public contact email: **UOMysticmoon@gmail.com**
|
||||
|
||||
Skeleton from the spec, with a small number of justified additions marked **(+)**.
|
||||
|
||||
> **In progress:** the monolithic route files below (`admin.routes.js` especially, originally 1552
|
||||
> lines / 110 routes) are being split into one router file per business capability — **in place, with
|
||||
> every URL unchanged**. This section and §4 get updated as each split PR lands. See
|
||||
> [API_V2_PLAN.md](./API_V2_PLAN.md) § Phase 2.
|
||||
> **Complete.** The monolithic route files (`admin.routes.js` especially, originally 1552 lines /
|
||||
> 110 routes) have been split into one router file per business capability — **in place, with every
|
||||
> URL unchanged**. See [API_V2_PLAN.md](./API_V2_PLAN.md) § Phase 2.
|
||||
>
|
||||
> **Landed so far:** admin `users`, `account`, `invites` and `auth/providers` (28 routes) now live in
|
||||
> their own routers under `admin/`, behind a new `admin/index.js`. The remaining 82 admin routes are
|
||||
> still in `admin.routes.js`, and `public/` and `player/` are untouched.
|
||||
> `users`, `account`, `invites`, `auth/providers` (PR 1, 28 routes), `moderation`, `bot-activity`,
|
||||
> `activity` (PR 2, 18 routes), `posts`, `uploads`, `wiki`, `pages` (PR 3, 31 routes) and `shard`,
|
||||
> `uo-link`, `email`, `discord-bot`, `settings`, `dashboard`/`site-mode` (PR 4, 33 routes) each live
|
||||
> in their own router under `admin/`, behind `admin/index.js`. PR 5 did the same for `public/` (24),
|
||||
> `player/` (20) and the residual `auth/` (10). **`admin.routes.js`, `public.routes.js`,
|
||||
> `player.routes.js` and `auth.routes.js` are all deleted**; each group is now a directory whose
|
||||
> `index.js` owns the group gate and the mount table and declares no routes of its own.
|
||||
>
|
||||
> "Every URL unchanged" is enforced mechanically, not by review: `server/scripts/routeManifest.js`
|
||||
> (`npm run routes:manifest`) walks the live Express stack and writes the sorted
|
||||
@@ -57,9 +60,61 @@ server/
|
||||
router/
|
||||
api.router.js mounts /v1
|
||||
v1/
|
||||
v1.router.js mounts /auth /public /admin
|
||||
auth/ auth.routes.js + auth.controller.js
|
||||
public/ public.routes.js + public.controller.js
|
||||
v1.router.js mounts /auth /public /admin /player
|
||||
auth/ index.js mounts the routers below; no group gate — /auth
|
||||
is where an anonymous caller becomes
|
||||
authenticated, so the authenticated parts gate
|
||||
themselves. Mount order is load-bearing (see
|
||||
session.router.js)
|
||||
login.router.js (2) /auth/login + /login/totp — shared
|
||||
loginGuards stack
|
||||
register.router.js (1) /auth/register — honours the
|
||||
player_registration setting
|
||||
invite.router.js (2) /auth/invite/:token[/accept] — the
|
||||
token is its own authority, so it
|
||||
bypasses player_registration
|
||||
password.router.js (3) /auth/password/forgot + reset/:token
|
||||
session.router.js (2) POST /logout and GET /me — the two
|
||||
singletons owning no path segment, so
|
||||
mounted at the group root, LAST: the
|
||||
/me sub-routers below also match the
|
||||
bare /me and supply its noindex header
|
||||
me.routes.js (23) /auth/me/account*, sessions, trusted
|
||||
devices — router-level requireAuth
|
||||
notifications.routes.js (3) /auth/me/devices*, notifications/*
|
||||
mobile.routes.js + /auth/mobile/* — native bearer login
|
||||
mobileSso.routes.js (5)
|
||||
sso.routes.js (4) mounted PATHLESS: owns two prefixes,
|
||||
/auth/providers and /auth/sso/*
|
||||
loginGuards.js shared backoff/slow/limiter stack for
|
||||
every credential-guessing surface
|
||||
(not a router)
|
||||
auth.controller.js + invite/passwordReset/sso/mobile controllers
|
||||
public/ index.js mounts the routers below; **no group gate** —
|
||||
this surface is anonymous by design (SPA
|
||||
logged-out, Discord bot, Android ShardStream)
|
||||
posts.router.js (2) /public/posts/:category[/:idOrSlug]
|
||||
wiki.router.js (4) /public/wiki — /categories and /tags
|
||||
MUST precede /:slug
|
||||
pages.router.js (2) /public/pages — the draft-preview
|
||||
route precedes /:slug and is
|
||||
deliberately not site-mode gated
|
||||
shard.router.js (14) /public/shard/* incl. the anonymous
|
||||
SSE stream; never site-mode gated
|
||||
site.router.js (4) /settings /status /version /contact —
|
||||
the group-root singletons; declares no
|
||||
router-level middleware
|
||||
public.controller.js + shard.controller.js
|
||||
player/ index.js owns the shared `noindex, requireAuth` gate
|
||||
(authenticated, ANY role — staff are a superset
|
||||
of players) and the mount table
|
||||
account.router.js (8) /player/account — credentials, TOTP,
|
||||
linked identities; handlers shared
|
||||
with /admin/account and /auth/me
|
||||
shard.router.js (8) /player/shard — linking + own roster,
|
||||
vendors, chars, sales, houses
|
||||
appeals.router.js (4) /player/appeals
|
||||
shard.controller.js + appeals.controller.js
|
||||
admin/ index.js mounts the capability routers below at their
|
||||
own prefixes; owns the shared
|
||||
`noindex, isLoggedIn, staffOnly` gate and
|
||||
@@ -68,9 +123,38 @@ server/
|
||||
users.router.js (15) /admin/users — adminOnly
|
||||
invites.router.js (3) /admin/invites — adminOnly
|
||||
authProviders.router.js (4) /admin/auth — adminOnly
|
||||
admin.routes.js (82) everything not yet split, mounted
|
||||
last at the group root; goes away
|
||||
when the final split PR lands
|
||||
moderation.router.js (15) /admin/moderation — modAccess
|
||||
(admin+moderator) at router level
|
||||
botActivity.router.js (2) /admin/bot-activity — adminOnly
|
||||
activity.router.js (1) /admin/activity — staff-wide
|
||||
audit log, no extra gate
|
||||
posts.router.js (9) /admin/posts — editor tier, no
|
||||
gate beyond staffOnly
|
||||
uploads.router.js (1) /admin/uploads — rich-text editor
|
||||
image upload
|
||||
wiki.router.js (14) /admin/wiki — pages, revisions,
|
||||
categories, tags
|
||||
pages.router.js (7) /admin/pages — CMS page builder
|
||||
imageUpload.js shared multer config for the two
|
||||
upload routes above (not a router)
|
||||
shard.router.js (16) /admin/shard — 7 self-service
|
||||
account-linking routes (no extra
|
||||
gate, handlers shared with
|
||||
/player/shard) + 9 in-game staff
|
||||
ops on modAccess, per route
|
||||
uoLink.router.js (5) /admin/uo-link — sidecar config,
|
||||
town crier, admin SSE — adminOnly
|
||||
email.router.js (6) /admin/email — Gmail OAuth2
|
||||
delivery — adminOnly
|
||||
discordBot.router.js (2) /admin/discord-bot — adminOnly
|
||||
settings.router.js (2) /admin/settings — adminOnly
|
||||
dashboard.router.js (2) GET /dashboard (staff-wide) and
|
||||
PUT /site-mode (adminOnly) — the
|
||||
two singletons owning no path
|
||||
segment, so mounted at the group
|
||||
root; declares no router-level
|
||||
middleware, which is what makes a
|
||||
root mount safe
|
||||
admin.controller.js + the per-capability controllers
|
||||
(already domain-split; the split PRs re-wire
|
||||
routes, not logic)
|
||||
@@ -227,6 +311,7 @@ construction, and the authorization code is stored as a **sha256 hash only** (sa
|
||||
| state | VARCHAR(255) NOT NULL | app-generated opaque CSRF value, echoed on the callback for the app to verify |
|
||||
| status | ENUM('pending','completed','consumed') DEFAULT 'pending' | `pending`→`completed` when the code is minted; `consumed` after a successful exchange |
|
||||
| user_id | INT NULL FK→users(id) ON DELETE CASCADE | set once SSO resolves the account |
|
||||
| trust_device | TINYINT(1) NOT NULL DEFAULT 0 | user ticked "trust this device" on the Custom Tab TOTP form. A **boolean only** — it tells `/exchange` to mint the app's own trust token; the token never rests here (only its sha256 reaches `trusted_devices`) |
|
||||
| expires_at | DATETIME NOT NULL | short (~10 min — one redirect round-trip incl. TOTP) |
|
||||
| created_at / used_at | DATETIME | `used_at` stamped at exchange |
|
||||
|
||||
@@ -273,6 +358,97 @@ analogue to a password — and there is no hash-lookup constraint (verification
|
||||
unused rows and `bcrypt.compare`s each, like password verification). `used_at` is the single-use
|
||||
marker. Cleared wholesale on TOTP disable / password change / password reset.
|
||||
|
||||
### shard_ruleset — the shard's published ruleset (Protocol 3.0)
|
||||
|
||||
Singleton row (`id = 1`, CHECK-constrained) holding the latest `world.ruleset` frame: `rev`,
|
||||
`expansion`, `payload` JSON (the whole frame), `t`, `updated_at`. The shard re-emits the complete
|
||||
ruleset on every sidecar connect, so this is an **overwrite, not an append** — and the kind is
|
||||
deliberately **not** in `LOGGED_KINDS`, since logging it would put a duplicate row in `shard_events`
|
||||
on every reconnect while `server.hello` already marks each of those.
|
||||
|
||||
The frame is stored whole rather than normalized into columns: it is a flat description of server
|
||||
config that is read as one page, so splitting it up would mean a schema change every time the shard
|
||||
grows a new block. `rev` (the shard's FNV-1a of the body) and `expansion` are hoisted only because
|
||||
they are cheap to display — the same payload-plus-hoisted-columns shape `shard_champs` uses.
|
||||
|
||||
**No row means the shard has never published one** (an older plugin, or `Bridge.RulesetEnabled=false`),
|
||||
served as `null` rather than `{}`: "not published yet" and "published, everything off" are different
|
||||
answers and the page renders them differently.
|
||||
|
||||
### shard_feature_visibility — per-feature audience config (Protocol 3.0)
|
||||
|
||||
One row per shard feature: `feature` (PK), `enabled`, `audience` (a rung on the ladder in §6.5),
|
||||
`stream` (whether the feature's kinds fan out over SSE at all), `field_rules` JSON (`{field: rung}`
|
||||
for the sensitive fields only), `updated_by`, `updated_at`.
|
||||
|
||||
**An absent row means "use the compiled default", and the compiled defaults reproduce pre-3.0
|
||||
behavior — so an empty table is a no-op and there is nothing to seed.** Stored rows are merged over
|
||||
the defaults on read, which is also where the invariants are re-applied: a row naming an unknown
|
||||
feature is ignored (a stale row must not resurrect a removed feature), an invalid rung falls back to
|
||||
the default rather than failing open, and a rule touching a locked field (`acct` / `webId`) is
|
||||
discarded. See §6.5.
|
||||
|
||||
### shard_spawn_* / shard_regions / shard_landmarks / shard_champion_spawns / shard_atlas_meta — the spawn atlas (Protocol 3.0)
|
||||
|
||||
Static shard **content**, not live shard state. Nothing here comes from the sidecar: the atlas is
|
||||
derived from the shard's own ServUO tree, re-read on **every server boot** and hash-gated so an
|
||||
unchanged tree costs one read pass and no write. Nothing is precomputed and committed — a shard's
|
||||
maps change over its life, and a snapshot in the repo would silently drift from the world players
|
||||
actually see. These tables stay populated whether the shard is up or not. Full operator detail in
|
||||
[`SPAWN_ATLAS.md`](SPAWN_ATLAS.md); the design is `docs/link/v3.md` §6.
|
||||
|
||||
**No facet name appears anywhere in the code.** A shard may add facets, replace them, or rename them
|
||||
when its maps are updated; the facet set is discovered from the tree, and the loose spellings in
|
||||
`Data/Locations` are matched against it rather than looked up in a table.
|
||||
|
||||
| Table | Key columns |
|
||||
|---|---|
|
||||
| `shard_spawn_creatures` | `slug` PK, `name`, `total`, `points`, `facets` JSON, `art` NULL |
|
||||
| `shard_spawn_points` | `id` PK, `facet`, `name`, `x`, `y`, `width`, `height`, `spawn_range`, `max_count`, `min_delay`, `max_delay`, `tod_start/end/mode`, `region`, `landmark`, `label` |
|
||||
| `shard_spawn_point_types` | `(point_id, slug)` PK, `max_count` |
|
||||
| `shard_regions` | `facet`, `name`, `type`, `priority`, `parent`, `rects` JSON |
|
||||
| `shard_landmarks` | `facet`, `name`, `grp`, `x`, `y`, `z` |
|
||||
| `shard_champion_spawns` | `slug` PK, `name`, `grp`, `type`, `random_type`, `facet`, `x`, `y`, `z`, `radius`, `label` |
|
||||
| `shard_atlas_meta` | Singleton (`id = 1`), `payload` JSON (counts + a sha256 per source file), `imported_at` |
|
||||
| `shard_atlas_pending` | Singleton (`id = 1`), `status` (`pending`/`rejected`), `payload` JSON, `detected_at` |
|
||||
|
||||
The first seven are **import-owned**: a refresh empties and reloads every one inside a single
|
||||
transaction, so a failed reload leaves the previous atlas intact rather than a half-loaded world.
|
||||
Nothing else writes to them, and nothing holds a foreign key to them — no FKs at all, consistent with
|
||||
every other `shard_*` table.
|
||||
|
||||
**`shard_atlas_pending` is the security-relevant one.** A refresh that would REMOVE a facet is never
|
||||
applied automatically: facet loss is indistinguishable at boot from a half-copied or mid-update tree,
|
||||
so it is staged here for an admin to approve or reject, and **startup is never blocked by it**. Only
|
||||
the decision is stored — source hashes plus the facet diff, a few KB — and approving re-parses the
|
||||
tree, so a multi-megabyte blob never lands in the database and what gets applied matches the tree at
|
||||
approval time. A rejection is remembered against those exact hashes so a declined refresh does not
|
||||
re-prompt on every restart. Everything else (new facets, renamed regions, changed spawns) applies
|
||||
immediately, since none of it can destroy data an operator would miss.
|
||||
|
||||
The boot refresh is **best-effort by contract**: no configured path, an unreadable mount, a malformed
|
||||
file or a database error is caught and logged, and the site comes up serving whatever atlas it had.
|
||||
The tree path comes from the `spawn_atlas_servuo_path` setting, falling back to `SERVUO_PATH`.
|
||||
|
||||
Four column choices worth stating, because each one is a trap:
|
||||
|
||||
- **`spawn_range`, not `range`**, and **`grp`, not `group`** — both are reserved words.
|
||||
- **`DELETE`, not `TRUNCATE`.** `TRUNCATE` is DDL in MariaDB and implicitly commits, which would
|
||||
defeat the all-or-nothing reload. At ~7k rows the difference does not matter.
|
||||
- **Point ids are assigned explicitly**, not left to `AUTO_INCREMENT`: the `shard_spawn_point_types`
|
||||
rows need to know them, and `conn.batch()` reports no usable `insertId` for a multi-row insert.
|
||||
- **Plain `INDEX` on `name`, deliberately not `FULLTEXT`.** ~800 creature rows makes a `LIKE` scan
|
||||
free, and FULLTEXT's minimum token length would break searches for names like "orc".
|
||||
|
||||
`shard_champion_spawns` is the *configured* altar roster ("there is an Unholy Terror altar in
|
||||
Deceit"). The live `champ.update` feed in `shard_champs` is the separate answer to "it is on level 3
|
||||
right now". Both exist; they are not the same data.
|
||||
|
||||
**`shard_spawn_creatures.art` is always NULL on a fresh import.** The project ships no creature
|
||||
artwork: sprites live in the operator's own client `.mul`/`.uop` files and are theirs, not ours to
|
||||
redistribute. An operator supplies art via a gitignored map plus images under the (already
|
||||
gitignored) `server/uploads/atlas/`. Text-only is the normal, supported state.
|
||||
|
||||
---
|
||||
|
||||
## 4. API contract
|
||||
@@ -313,7 +489,12 @@ authenticated endpoints silently. Names are a hint only — `requireRole(...)` r
|
||||
arrow and cannot be observed — but a *missing* `requireAuth` is unambiguous, and the server test suite
|
||||
asserts every `/admin/**` and `/player/**` route still carries it.
|
||||
|
||||
### /auth (auth.routes.js → auth.controller.js)
|
||||
### /auth (auth/index.js → the capability routers in §2)
|
||||
|
||||
No group gate — `/auth` is where an anonymous caller becomes authenticated. The authenticated parts
|
||||
gate themselves: `me.routes.js` and `notifications.routes.js` each apply `noindex, requireAuth` at
|
||||
their own router level, and `/sso/:provider/link` carries `requireAuth` per route.
|
||||
|
||||
| Method | Path | Auth | Body | Purpose |
|
||||
|---|---|---|---|---|
|
||||
| POST | `/login` | — (rate-limited) | `{username,password}` | verify, set cookie, log `auth.login`, update `last_login_at`. If the account has TOTP **and this browser is a trusted device** (a valid `rg_trust` cookie bound to the user), the TOTP step is **skipped** and a session is issued directly (logs `auth.login.trusted_device`). Otherwise a 2FA account returns `{totpRequired, challenge}`. |
|
||||
@@ -346,9 +527,9 @@ lets a client (the Android app) manage its own account through one surface witho
|
||||
for web back-compat.
|
||||
|
||||
**The `/player/*` group is self-service, not player-only.** Staff are a **superset** of players — every
|
||||
player ability plus their staff tools on top — so the whole `/player/*` router (game-account linking,
|
||||
character/vendor/house reads, credential changes, appeals) sits behind `requireAuth` **only**, never
|
||||
`requireRole('player')`. Every handler is self-scoped to the caller by `req.user.id`, so an admin/editor/
|
||||
player ability plus their staff tools on top — so the whole group (`account.router.js`,
|
||||
`shard.router.js`, `appeals.router.js`, mounted by `player/index.js`) sits behind the shared
|
||||
`noindex, requireAuth` gate **only**, never `requireRole('player')`. Every handler is self-scoped to the caller by `req.user.id`, so an admin/editor/
|
||||
moderator using it sees only their **own** linked accounts and characters (with the pre-existing
|
||||
`isAdmin` bypass still letting a genuine admin read *any* character). Staff also reach the identical
|
||||
self-scoped handlers under `/admin/shard/*` (same controller) for the web admin surface; the two are
|
||||
@@ -391,12 +572,18 @@ consumer of the existing SSO + mobile-bearer machinery**, not a parallel auth pa
|
||||
`/auth/sso/:provider/*` redirect flow, the link-only + opt-in-provisioning policy, the TOTP gate, and
|
||||
issues the **same** token pair as `/auth/mobile/login`.
|
||||
|
||||
The TOTP gate it reuses includes the **trusted-device skip** (see
|
||||
`TRUSTED_DEVICES_MFA.md` §6). Because the app opens this flow in a Custom Tab, which shares the
|
||||
system browser's cookie jar, the `rg_trust` cookie set on the TOTP form is presented back on the next
|
||||
app sign-in — so "don't ask me again" works for native SSO without the app injecting a header into a
|
||||
tab it does not control, and without a trust token ever appearing in a start URL.
|
||||
|
||||
| Method | Path | Auth | Body / Query | Purpose |
|
||||
|---|---|---|---|---|
|
||||
| GET | `/auth/providers` | — | — | **reused** discovery; the app renders provider buttons from this (never exposes secrets) |
|
||||
| GET | `/auth/mobile/sso/start` | — (rate-limited per-IP + per-provider) | `?provider&code_challenge&state&redirect_uri` | validate provider enabled + `redirect_uri` **exact-match** allowlist; insert a `mobile_auth_sessions` row; create the existing `sso_tx` tagged `mode:'mobile'` carrying `session_id`; **302 to the IdP** (existing authorize URL) |
|
||||
| GET | `/auth/sso/:provider/callback` | — (signed `sso_tx`) | `?code&state` | **existing** endpoint; a new branch when `tx.mode==='mobile'`: resolve the account (same policy as web login incl. TOTP), mint a single-use hashed authorization code into `mobile_auth_codes`, mark the session `completed`, and **302 to `redirect_uri?code=…&state=…`** (the app's original `state`) — **no cookie is set** |
|
||||
| POST | `/auth/mobile/sso/exchange` | — (rate-limited per-IP) | `{code, code_verifier}` | validate the code exists / unexpired / unused (mark used) and `sha256(code_verifier)` matches the stored challenge → issue the existing mobile access + refresh pair (`createMobileSession`) → `{accessToken, refreshToken, expiresIn, user}` |
|
||||
| POST | `/auth/mobile/sso/exchange` | — (rate-limited per-IP) | `{code, code_verifier}` | validate the code exists / unexpired / unused (mark used) and `sha256(code_verifier)` matches the stored challenge → issue the existing mobile access + refresh pair (`createMobileSession`) → `{accessToken, refreshToken, expiresIn, user}`. When the session carries `trust_device`, also mint a `platform:'mobile'` trusted device and add `trustToken` — minted here, on an authenticated app→server call, so it never travels in the deep link. Best-effort: at the trusted-device cap the response simply omits it rather than failing the sign-in |
|
||||
| POST | `/auth/mobile/refresh` | — | `{refreshToken}` | **reused** unchanged — rotate the pair |
|
||||
| POST | `/auth/mobile/logout` | bearer | `{refreshToken?, all?}` | **reused** unchanged — revoke this (or all) refresh token(s) |
|
||||
| GET | `/auth/me/sessions` · DELETE `…/:id` | cookie / bearer | — | list / revoke own **mobile sessions** (device_name, last_used_at, created_at) — the "Active Devices" surface (distinct from `/auth/me/devices`, which is push endpoints) |
|
||||
@@ -443,7 +630,13 @@ web sessions already use.
|
||||
**Authorization code.** Cryptographically random, ≥128 bits, stored **hash-only**, single-use, short
|
||||
expiry (~5 min); `/exchange` is rate-limited per-IP. The bridge tables self-prune (§3).
|
||||
|
||||
### /public (public.routes.js → public.controller.js) — all GET, no auth
|
||||
### /public (public/index.js → the capability routers in §2) — all GET except `/contact`, no auth
|
||||
|
||||
**No group gate, deliberately.** This surface is anonymous by design: the SPA renders it logged-out,
|
||||
the Discord bot reads it with no credentials, and the Android `ShardStreamClient` consumes
|
||||
`/public/shard/stream` without an `Authorization` header. Content visibility during maintenance comes
|
||||
from the per-route **siteMode** middleware (§5), never from an auth gate.
|
||||
|
||||
| Method | Path | Notes |
|
||||
|---|---|---|
|
||||
| GET | `/settings` | whitelisted public keys, derived `registration`/`gameAccountSignup` flags, the per-shard **`brand`** block (name, `accent` color, logo/hero/favicon) a client themes itself from — one image runs as any shard, asset fields may be site-relative paths (resolve against the base URL) — and a **`push`** block `{ ntfyUrl }` (M7): the client-facing ntfy relay URL the app's embedded distributor registers its device topic against, from `NTFY_PUBLIC_URL` / first `NTFY_ALLOWED_ORIGINS` (never the internal `NTFY_BASE_URL`); `null` when push isn't configured for the shard. |
|
||||
@@ -454,15 +647,32 @@ expiry (~5 min); `/exchange` is rate-limited per-IP. The bridge tables self-prun
|
||||
| GET | `/wiki` | list of pages (slug + title) |
|
||||
| GET | `/wiki/:slug` | single page |
|
||||
| POST | `/contact` | (rate-limited) send mail via SMTP; if unconfigured, respond `{fallback:"mailto", email}` |
|
||||
| GET | `/shard/ruleset` | the shard's own published ruleset (Protocol 3.0 `world.ruleset`): expansion, which optional systems are on, skill/stat caps, account and house limits, champion scroll rules, the save/restart schedule. Served from `shard_ruleset`, so it renders while the shard is down; live via `world.ruleset` on `/shard/stream`. Behind `requireFeature('ruleset')`. **`null`** means the shard has never published one — a real answer, distinct from a published ruleset. `caps.skill` / `caps.totalSkill` are in **tenths** (1000 = 100.0). |
|
||||
| GET | `/shard/features` | the shard features **this caller** may reach plus the audience rung they resolved to (§6.5), so a client hides nav it can't follow. Reports only what the caller can see — the list itself never discloses a gated feature. Consumed by the SPA header and (pending) the Android nav. |
|
||||
|
||||
Public content GETs pass through the **siteMode** gate (§5).
|
||||
|
||||
### /admin (admin/index.js → the capability routers in §2) — all behind `isLoggedIn` + `noindex` + `staffOnly`
|
||||
|
||||
`admin/index.js` applies the shared gate and mounts each capability router at the prefix it owns;
|
||||
`users`, `invites` and `auth/providers` add `adminOnly` on top. Routes not yet extracted still live
|
||||
in `admin.routes.js`, mounted last at the group root. The URLs below are unaffected by which file a
|
||||
route currently sits in — that is the property the route manifest freezes.
|
||||
`users`, `invites`, `auth/providers` and `bot-activity` add `adminOnly` on top, and `moderation` adds
|
||||
`modAccess` (admin + moderator, so editors are excluded). The content capabilities — `posts`,
|
||||
`uploads`, `wiki`, `pages` — add nothing: managing content is the editor tier's job, so `staffOnly` is
|
||||
the whole gate. The ops/config capabilities — `uo-link`, `email`, `discord-bot`, `settings`, and
|
||||
`PUT /site-mode` — are `adminOnly`; `shard` is the one mixed prefix, where self-service account
|
||||
linking carries no extra gate and the in-game staff operations carry `modAccess`. There is no residual
|
||||
file: every admin route is declared in a capability router.
|
||||
|
||||
`GET`/`PUT /admin/shard/visibility` are the third tier on that mixed prefix: **`adminOnly`**, because
|
||||
they decide what *anonymous* visitors can see (§6.5). They sit above `modAccess` deliberately — a
|
||||
moderator can ban a player but cannot decide what the public internet reads.
|
||||
|
||||
`GET /dashboard` and `PUT /site-mode` are the one place where a **single screen spans two tiers**: the
|
||||
dashboard is staff-wide, but the site-mode toggle on it is `adminOnly`. The client must therefore gate
|
||||
that control on its own (`Dashboard.jsx` renders it only for `role === 'admin'`) rather than relying on
|
||||
the route gate that admitted them to the page — the same rule the sidebar follows, so a non-admin is
|
||||
never shown a control that would 403. The URLs below are unaffected by which
|
||||
file a route sits in — that is the property the route manifest freezes.
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| GET | `/dashboard` | current mode, last change time + who, content counts, recent activity |
|
||||
@@ -559,6 +769,77 @@ who"; `activity_log` provides the history feed.
|
||||
- **`app.set('trust proxy', 1)`** so secure cookies, `req.ip`, and rate-limiting work behind Pangolin.
|
||||
- **CORS**: same-origin in prod (SPA served by Express). Dev only: allow `CLIENT_ORIGIN` (Vite, `http://localhost:5173`) with `credentials:true`.
|
||||
|
||||
### 6.5 Shard visibility — the audience boundary (Protocol 3.0)
|
||||
|
||||
Every shard-derived surface is gated by an **admin-configurable, per-feature and per-field** audience
|
||||
setting. This **replaces** the static `PUBLIC_KINDS` allowlist that used to be the whole boundary.
|
||||
Policy lives in `utils/shardVisibility.js`; rows live in `shard_feature_visibility`; the admin surface
|
||||
is `GET`/`PUT /admin/shard/visibility` (`adminOnly`). Admin-facing guide:
|
||||
[`SHARD_VISIBILITY.md`](SHARD_VISIBILITY.md). Design: [`../link/v3.md`](../link/v3.md) §3.
|
||||
|
||||
**The ladder.** `anonymous < logged_in < player < staff < admin`, each rung implying the ones below.
|
||||
`viewerLevel(req)` resolves it: no session ⇒ `anonymous`; authenticated ⇒ `logged_in`; authenticated
|
||||
with a linked game account ⇒ `player`; moderator ⇒ `staff`; admin ⇒ `admin`. **Staff satisfy the
|
||||
`player` rung without a linked account** (consistent with `/player/*` being role-agnostic).
|
||||
**`editor` gets no shard privilege** — it is a content role, and mapping it to `staff` would silently
|
||||
widen what editors see.
|
||||
|
||||
**Two invariants that are code, not configuration.** Both are enforced server-side and both reject
|
||||
rather than silently ignore:
|
||||
|
||||
1. **`acct` and `webId` are admin-only, always.** They are not exposed as configurable fields, and a
|
||||
stored row attempting to loosen them is discarded on read as well as rejected on write. A character
|
||||
name is visible in game; the account behind it and the website user it links to are not.
|
||||
The lock is on the field's **meaning, not one spelling**: `isLockedField(key)` matches a key that
|
||||
*is* or *ends in* `acct`/`webId`, case-insensitively, so the flattened forms the read models emit
|
||||
(`shapeHouse` → `ownerAcct`, `shapeGuild` → `leaderWebId`) are covered too. An exact-key check was
|
||||
the original implementation and it let `GET /public/shard/idoc` serve `ownerAcct` anonymously.
|
||||
2. **A kind absent from `KIND_FEATURE` is never broadcast below `admin`.** Fail closed. This is what
|
||||
keeps the kind map a security boundary rather than a convenience filter, and it means a shard that
|
||||
starts emitting an unknown event degrades to staff-only, never to public.
|
||||
|
||||
**Fail-closed everywhere else too.** An unreadable visibility config withholds every public frame; a
|
||||
DB failure falls back to the compiled defaults (pre-3.0 behavior), not to open; an unresolvable viewer
|
||||
subscribes as `anonymous`. The ladder comparison uses **asymmetric** fallbacks by design — an unknown
|
||||
*viewer* level floors to the bottom rung and an unknown *requirement* ceils to admin, so an
|
||||
unrecognised value loses on both sides. (A single shared fallback cannot do that: whichever direction
|
||||
it picks, it fails open on one side.)
|
||||
|
||||
**Three enforcement points, one config:**
|
||||
|
||||
| Where | Mechanism |
|
||||
|---|---|
|
||||
| Routes | `requireFeature(name)` — **404** when the feature is disabled (don't leak that it exists), **403** when the caller is below its audience. `projectFeature` then strips out-of-rung fields from the body. |
|
||||
| SSE (`utils/shardBroadcast.js`) | Per-connection filtering. A subscriber's rung is resolved **once at subscribe time and frozen** for that connection, so a long-lived stream can't gain privilege; each frame is then mapped kind→feature, gated, and field-projected per viewer. Two subscribers can legitimately receive different versions of one event, or one of them nothing. |
|
||||
| Nav | `GET /public/shard/features` returns only what the caller may reach, so the SPA never renders a link that would 403. Presentation only. |
|
||||
|
||||
Config reads are cached ~5s, so admin changes take effect within seconds **including on already-open
|
||||
streams**. `PUBLIC_KINDS` still exists and is still exported (`notificationStreams.js`) but is now
|
||||
**derived** from the kind map rather than hand-maintained, so the two cannot drift.
|
||||
|
||||
**`PUBLIC_KINDS` is a module-load constant and must not be used to answer "may this caller read this
|
||||
kind?"** — it is computed from the compiled *defaults*, so it cannot see an admin's changes. Use
|
||||
`visibleKinds(level, config)`, which resolves against the live config. `/feed` uses it; it originally
|
||||
used `PUBLIC_KINDS` and consequently kept serving `guild.join` to anonymous callers after an admin had
|
||||
moved `guilds` to `staff`. `visibleKinds` deliberately ignores the `stream` flag: that governs SSE
|
||||
fan-out only, so a feature whose live firehose ships off (market) stays readable from stored history.
|
||||
|
||||
**Every read path that returns shard data must call `projectFeature`.** The stored-history endpoints
|
||||
are not exempt — `/feed` returns the same events the stream does, and returning them unprojected
|
||||
reopens on the REST side exactly what the stream closes. Relatedly, `shardEvents.db.list` treats an
|
||||
**empty** `kinds` array as "serve nothing", never "no filter"; the fall-through it used to take would
|
||||
have turned a fully-gated config into a dump of the entire event log.
|
||||
|
||||
`projectFeature` walks **arrays and plain objects only**. A `Date`, `Buffer` or other class instance
|
||||
is passed through as a value — rebuilding one key-by-key yields `{}`, which is the difference between
|
||||
the pure-JSON wire frames and the DB-backed read models whose rows carry real `Date` columns.
|
||||
|
||||
**Defaults reproduce pre-3.0 behavior exactly**, so installing the framework is a no-op until an admin
|
||||
changes something — with deliberate exceptions, which are the leaks it was written to close.
|
||||
`/public/shard/guilds`, `/public/shard/governors` and `/public/shard/feed` previously returned the raw
|
||||
stored payload, whose actors carry `acct` and `webId`; `/public/shard/idoc` returned the flattened
|
||||
`ownerAcct`. All are now stripped for every caller below admin.
|
||||
|
||||
---
|
||||
|
||||
## 7. Email
|
||||
|
||||
@@ -259,6 +259,8 @@ website/
|
||||
│ ├── db/
|
||||
│ │ ├── schema.sql
|
||||
│ │ └── seed.js
|
||||
│ ├── scripts/
|
||||
│ │ └── routeManifest.js
|
||||
│ ├── src/
|
||||
│ │ ├── auth/
|
||||
│ │ │ ├── providers/
|
||||
@@ -290,6 +292,7 @@ website/
|
||||
│ │ │ └── validateBlocks.js
|
||||
│ │ ├── config/
|
||||
│ │ │ ├── brand.js
|
||||
│ │ │ ├── csp.js
|
||||
│ │ │ ├── notificationStreams.js
|
||||
│ │ │ └── version.js
|
||||
│ │ ├── middleware/
|
||||
@@ -392,18 +395,31 @@ website/
|
||||
│ │ │ ├── v1/
|
||||
│ │ │ │ ├── admin/
|
||||
│ │ │ │ │ ├── account.controller.js
|
||||
│ │ │ │ │ ├── account.router.js
|
||||
│ │ │ │ │ ├── activity.router.js
|
||||
│ │ │ │ │ ├── admin.controller.js
|
||||
│ │ │ │ │ ├── admin.routes.js
|
||||
│ │ │ │ │ ├── authProviders.controller.js
|
||||
│ │ │ │ │ ├── authProviders.router.js
|
||||
│ │ │ │ │ ├── botActivity.controller.js
|
||||
│ │ │ │ │ ├── botActivity.router.js
|
||||
│ │ │ │ │ ├── discordBot.controller.js
|
||||
│ │ │ │ │ ├── emailConfig.controller.js
|
||||
│ │ │ │ │ ├── imageUpload.js
|
||||
│ │ │ │ │ ├── index.js
|
||||
│ │ │ │ │ ├── invites.controller.js
|
||||
│ │ │ │ │ ├── invites.router.js
|
||||
│ │ │ │ │ ├── moderation.controller.js
|
||||
│ │ │ │ │ ├── moderation.router.js
|
||||
│ │ │ │ │ ├── pages.controller.js
|
||||
│ │ │ │ │ ├── pages.router.js
|
||||
│ │ │ │ │ ├── posts.router.js
|
||||
│ │ │ │ │ ├── shardOps.controller.js
|
||||
│ │ │ │ │ ├── uoLink.controller.js
|
||||
│ │ │ │ │ └── usersShard.controller.js
|
||||
│ │ │ │ │ ├── uploads.router.js
|
||||
│ │ │ │ │ ├── users.router.js
|
||||
│ │ │ │ │ ├── usersShard.controller.js
|
||||
│ │ │ │ │ └── wiki.router.js
|
||||
│ │ │ │ ├── auth/
|
||||
│ │ │ │ │ ├── auth.controller.js
|
||||
│ │ │ │ │ ├── auth.routes.js
|
||||
@@ -432,6 +448,7 @@ website/
|
||||
│ │ │ │ │ └── shard.controller.js
|
||||
│ │ │ │ └── v1.router.js
|
||||
│ │ │ ├── api.router.js
|
||||
│ │ │ ├── cspReport.controller.js
|
||||
│ │ │ └── wellKnown.controller.js
|
||||
│ │ ├── utils/
|
||||
│ │ │ ├── announceWorker.js
|
||||
@@ -471,6 +488,7 @@ website/
|
||||
│ │ ├── authTrustedDevice.test.js
|
||||
│ │ ├── botInternalKey.test.js
|
||||
│ │ ├── botScore.test.js
|
||||
│ │ ├── csp.test.js
|
||||
│ │ ├── emailConfig.model.test.js
|
||||
│ │ ├── honeypot.test.js
|
||||
│ │ ├── inviteController.test.js
|
||||
@@ -499,6 +517,7 @@ website/
|
||||
│ │ ├── recoveryCodes.test.js
|
||||
│ │ ├── registry.test.js
|
||||
│ │ ├── requireInternalKey.test.js
|
||||
│ │ ├── routeManifest.test.js
|
||||
│ │ ├── secretBox.test.js
|
||||
│ │ ├── selfTrustedDevices.test.js
|
||||
│ │ ├── session.test.js
|
||||
@@ -515,7 +534,9 @@ website/
|
||||
│ │ └── usernamePolicy.test.js
|
||||
│ ├── .env.example
|
||||
│ ├── package-lock.json
|
||||
│ └── package.json
|
||||
│ ├── package.json
|
||||
│ ├── routes.guards.json
|
||||
│ └── routes.manifest.json
|
||||
├── .dockerignore
|
||||
├── .env.example
|
||||
├── .env.uomysticmoon.example
|
||||
|
||||
138
website/SHARD_VISIBILITY.md
Normal file
138
website/SHARD_VISIBILITY.md
Normal file
@@ -0,0 +1,138 @@
|
||||
# Shard visibility — who sees which shard data
|
||||
|
||||
**Status:** Built (Protocol 3.0 Part A). Admin → Shard Visibility.
|
||||
**Audience:** shard owners and admins.
|
||||
**Companion to** [`../link/v3.md`](../link/v3.md) §3 (the design) and
|
||||
[`BACKEND_DESIGN.md`](BACKEND_DESIGN.md) §6 (the security contract).
|
||||
|
||||
The website surfaces a lot of live shard data. What your players, your staff and the anonymous
|
||||
internet may each see is **yours to decide**, per feature, from Admin → Shard Visibility.
|
||||
|
||||
Nothing changes until you change it: every setting ships at the value that reproduces how the site
|
||||
behaved before this panel existed.
|
||||
|
||||
---
|
||||
|
||||
## 1. The audience ladder
|
||||
|
||||
Five rungs. Each one includes everyone below it.
|
||||
|
||||
| Rung | In the UI | Who that is |
|
||||
|---|---|---|
|
||||
| `anonymous` | **Everyone** | Anyone at all, signed in or not. |
|
||||
| `logged_in` | **Signed in** | Any registered account, whether or not they've linked a game account. |
|
||||
| `player` | **Linked players** | Accounts with a linked in-game account. **Staff always qualify**, linked or not. |
|
||||
| `staff` | **Staff** | Admins and moderators. |
|
||||
| `admin` | **Admins only** | Admins. |
|
||||
|
||||
Two notes that surprise people:
|
||||
|
||||
- **`editor` is a content role, not a shard role.** Editors write news and wiki pages; they get no
|
||||
shard privilege from that. An editor is treated by link status like any other member. This matches
|
||||
the rest of the site, where shard staff powers are admin-or-moderator.
|
||||
- **Staff satisfy `player` without linking.** Otherwise an admin would be locked out of surfaces
|
||||
they'd gated to players, which is how the `/player/*` routes already behave.
|
||||
|
||||
## 2. What you can set per feature
|
||||
|
||||
**Enabled.** Off means gone. The feature's pages return “not found”, not “forbidden” — a disabled
|
||||
feature doesn't advertise that it exists.
|
||||
|
||||
**Who can see it.** The minimum rung, from the ladder above.
|
||||
|
||||
**Live updates.** Whether this feature pushes changes to open pages in real time. Turning it off
|
||||
doesn't break the page; it just refreshes on load instead of updating in place.
|
||||
|
||||
**Sensitive fields.** Some features expose a field that deserves its own rung — you can publish the
|
||||
board while holding back one column. See the table in §3.
|
||||
|
||||
## 3. The features, and their defaults
|
||||
|
||||
| Feature | What it exposes | Default | Sensitive fields |
|
||||
|---|---|---|---|
|
||||
| **Shard status** | Connection state, online count, gold-supply series | Everyone | — |
|
||||
| **Activity feed** | Deaths, kills, skill gains, quests, logins | Everyone | — |
|
||||
| **Champion spawns** | The live champion / mini-champ / sea-boss board | Everyone | — |
|
||||
| **Guilds** | Guild rosters, alliances, leaders | Everyone | — |
|
||||
| **Town governors** | City Loyalty governors, elections, term history | Everyone | — |
|
||||
| **Houses / IDOC** | Houses in danger | Everyone | House owner → Staff · House price → Staff |
|
||||
| **Players online** | Population aggregate, staff-online widget | Everyone | In-game location → Staff |
|
||||
| **Shard rules** | Skill/stat caps, house limits, vet rewards, the ruleset | Everyone | Connect address → Everyone |
|
||||
| **Spawn atlas** | Bestiary and spawn locations (static content) | Everyone | — |
|
||||
| **Leaderboards** | Point and loyalty standings | Everyone | Character names → Everyone |
|
||||
| **Marketplace** | The shard-wide player-vendor index | Everyone, **live updates off** | Vendor owner name → Everyone · Location → Everyone |
|
||||
|
||||
**Why the marketplace ships with live updates off.** A live feed of every vendor's full inventory
|
||||
would be the single largest thing the site sends. No page needs it — the marketplace is a search over
|
||||
stored data with a “prices last refreshed N minutes ago” stamp. Turn it on only if you want it.
|
||||
|
||||
**Why house owner/price default to Staff.** The public Houses page has always been a "where are the
|
||||
falling houses" board — location only. Owner and price are the staff view. That split is preserved.
|
||||
|
||||
## 4. What you cannot change
|
||||
|
||||
Two rules are enforced in code and are not settings. Attempting to set them returns an error rather
|
||||
than silently ignoring you.
|
||||
|
||||
**1. Game account names and website user ids are admin-only, always.**
|
||||
`acct` and `webId` never appear below the admin rung on any surface. A character *name* is visible in
|
||||
game to anyone standing next to them; the **account** behind it is not, and neither is the website
|
||||
user it's linked to. Publishing those would disclose something the shard itself doesn't, and would
|
||||
tie a player's in-game identity to their forum identity without their consent.
|
||||
|
||||
This rule matches the *meaning* of a field, not one spelling of it. Some responses nest the player
|
||||
who owns a record (`leader.acct`); others flatten it into the row (`ownerAcct`, `leaderWebId`,
|
||||
`governorAcct`). Every one of those is locked, and the admin API refuses to configure any of them —
|
||||
so a new response shape can't quietly reopen the hole by naming the field differently.
|
||||
|
||||
**2. Unknown event kinds are never broadcast below admin.**
|
||||
The live stream maps each event kind to a feature. A kind with no mapping — a new event from a shard
|
||||
plugin the site doesn't know yet, say — goes to admins only. It fails closed. This is what keeps the
|
||||
stream safe by default when the shard starts sending something new: the worst case is that staff see
|
||||
it and players don't, never the reverse.
|
||||
|
||||
## 5. How it's enforced
|
||||
|
||||
Three places, one config:
|
||||
|
||||
- **Page and API requests** are checked before the handler runs, and the response is then stripped of
|
||||
any field above the caller's rung.
|
||||
- **The live stream** resolves a viewer's rung once, when they connect, and freezes it for that
|
||||
connection — a long-open page can't gain privilege because something changed underneath it. Each
|
||||
event is then gated and stripped per viewer, so two people watching the same page can legitimately
|
||||
receive different versions of the same event, or one of them nothing.
|
||||
- **Navigation** hides links a viewer can't follow, so they don't hit a wall. This is presentation
|
||||
only — the gate is server-side either way.
|
||||
|
||||
**Stored history answers the same way the live stream does.** The activity feed reads from the event
|
||||
log rather than the live stream, but it resolves the *same* question against the *same* config: which
|
||||
kinds you may read, and which fields survive. So moving a feature up a rung hides it from the history
|
||||
as well as the stream — there is no back door where yesterday's copy of an event is more revealing
|
||||
than today's.
|
||||
|
||||
One deliberate asymmetry: turning **live updates** off for a feature stops the push, not the reading.
|
||||
The marketplace ships this way — its history and its pages are public, only the firehose is off.
|
||||
|
||||
Changes take effect within about five seconds, **including on streams that are already open**. You
|
||||
don't need to restart anything.
|
||||
|
||||
If the database is briefly unreachable, the site falls back to the built-in defaults — the pre-v3
|
||||
behavior — rather than to "everything is public".
|
||||
|
||||
## 6. Worked examples
|
||||
|
||||
**"I want a private shard — nothing public until people register."**
|
||||
Set every feature to **Signed in**. Anonymous visitors still get the site itself; the shard data
|
||||
disappears from the nav.
|
||||
|
||||
**"Publish the market, but don't tie vendors to players."**
|
||||
Marketplace → Everyone, with **Vendor owner name** → Staff. Prices, items and locations stay public;
|
||||
who owns each vendor doesn't.
|
||||
|
||||
**"Leaderboards for members only."**
|
||||
Leaderboards → **Linked players**. Anyone who's linked a game account sees the standings; drive-by
|
||||
visitors don't.
|
||||
|
||||
**"Let players see house owners."**
|
||||
Houses → Everyone, **House owner** → Linked players. Note this is a real disclosure: house ownership
|
||||
is visible in game, but the website makes it searchable in a way the game doesn't.
|
||||
261
website/SPAWN_ATLAS.md
Normal file
261
website/SPAWN_ATLAS.md
Normal file
@@ -0,0 +1,261 @@
|
||||
# Spawn atlas
|
||||
|
||||
**Status:** Data pipeline landed on `edge` (website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112)); API and client pages follow in a second PR.
|
||||
**Design:** [`docs/link/v3.md` §6](../link/v3.md) — Protocol 3.0 Part C.
|
||||
|
||||
The spawn atlas is a browsable catalogue of what the shard *contains*: which
|
||||
creatures spawn, where, how many, and which champion altars are configured. It
|
||||
answers "where do I find a lizardman?" with **"Shrines, Yew, Isamu-Jima"** rather
|
||||
than with a list of raw coordinates.
|
||||
|
||||
## Two things that shape the whole design
|
||||
|
||||
**The shard's ServUO tree is the single source of truth.** Nothing is
|
||||
precomputed and committed to the repository. A shard's maps change over its
|
||||
lifetime — facets get added, replaced, or renamed — and a snapshot in the repo
|
||||
would silently drift from the world players actually see. The atlas is therefore
|
||||
re-derived from the tree **on every server boot**.
|
||||
|
||||
**Facets are not a fixed list.** Nothing in the codebase names Felucca, Trammel,
|
||||
or any other stock facet. The facet set is whatever the shard's own files
|
||||
declare, discovered at parse time. A shard running entirely custom maps gets
|
||||
exactly the same treatment as a stock one, with no code change.
|
||||
|
||||
## What it is not
|
||||
|
||||
The atlas is **static shard content, not live shard state.**
|
||||
|
||||
- It does **not** come from the sidecar. Nothing here touches the bridge, and
|
||||
there is no event kind, no wire change and no `PROTOCOL_VERSION` bump for it.
|
||||
Part C is website-only.
|
||||
- It stays fully populated while the shard is down.
|
||||
- Its champion table (`shard_champion_spawns`) is the *configured roster* —
|
||||
"there is an Unholy Terror altar in Deceit". The live `champ.update` feed in
|
||||
`shard_champs` is the separate, sidecar-fed answer to "it is on level 3 right
|
||||
now". Both exist; do not conflate them.
|
||||
|
||||
Routes live at `/api/v1/public/atlas`, deliberately **not** under `/shard`,
|
||||
because `/shard/*` means sidecar-dependent.
|
||||
|
||||
## Configuring the tree
|
||||
|
||||
The website needs to be able to *read* the ServUO tree — same host, a bind mount,
|
||||
or a shared volume. Two ways to point at it, the setting winning over the
|
||||
environment:
|
||||
|
||||
| Source | Notes |
|
||||
|---|---|
|
||||
| `spawn_atlas_servuo_path` setting | Admin-editable; changes take effect on the next refresh without a redeploy |
|
||||
| `SERVUO_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |
|
||||
|
||||
With neither set the atlas is simply skipped — the site runs normally without
|
||||
one.
|
||||
|
||||
## The boot path
|
||||
|
||||
On every start the server hashes the source files and compares them against what
|
||||
is loaded. Unchanged (the normal case on a restart) costs one read pass, ~120 ms,
|
||||
and no database write. A real change costs a ~400 ms parse and a reload.
|
||||
|
||||
Two contracts govern it:
|
||||
|
||||
**1. It never blocks startup.** No configured path, an unreadable mount, a
|
||||
malformed file, a database error — every one is caught and logged, and the site
|
||||
comes up serving whatever atlas it already had.
|
||||
|
||||
**2. A facet disappearing is never applied automatically.** Losing a facet looks
|
||||
exactly like a half-copied or mid-update tree, and boot cannot tell that apart
|
||||
from a real map change. That refresh is *staged* for a human instead. Everything
|
||||
else — new facets, renamed regions, changed spawns — applies immediately, since
|
||||
none of it can destroy something an operator would miss.
|
||||
|
||||
```
|
||||
boot
|
||||
└─ path configured? no ──▶ skip
|
||||
└─ tree readable? no ──▶ warn, carry on
|
||||
└─ hashes changed? no ──▶ done (nothing parsed)
|
||||
└─ parse
|
||||
└─ a facet would be removed?
|
||||
no ──▶ import
|
||||
yes ──▶ stage for admin review; atlas unchanged
|
||||
```
|
||||
|
||||
### Approving or rejecting a staged refresh
|
||||
|
||||
Only the *decision* is stored, never the parsed world — a few KB of source hashes
|
||||
plus the facet diff. Approving **re-parses** the tree, so what lands matches the
|
||||
tree at approval time rather than at boot, and a multi-megabyte blob never sits
|
||||
in the database.
|
||||
|
||||
A rejection is remembered against those exact source hashes, so a declined
|
||||
refresh does not re-prompt on every restart. Change the tree and the hashes
|
||||
differ, which asks again.
|
||||
|
||||
From the admin panel (second PR), or from the CLI:
|
||||
|
||||
```bash
|
||||
cd website/server
|
||||
npm run atlas:import -- --status # what is loaded, and what is pending
|
||||
npm run atlas:import -- --approve # apply the staged refresh
|
||||
npm run atlas:import -- --reject # keep the current atlas, dismiss it
|
||||
```
|
||||
|
||||
## The CLI
|
||||
|
||||
The server refreshes itself on boot, so this is for applying a map change
|
||||
*without* a restart, and for the approve/reject flow above.
|
||||
|
||||
```bash
|
||||
npm run atlas:import # import if the tree differs
|
||||
npm run atlas:import -- --servuo <path> # override the path for this run
|
||||
npm run atlas:import -- --force # reimport even if unchanged
|
||||
```
|
||||
|
||||
`--servuo` is a per-run override and deliberately does **not** persist — changing
|
||||
where the atlas permanently reads from is an admin action, not a side effect of a
|
||||
one-off import.
|
||||
|
||||
## Sources
|
||||
|
||||
| File | Count (stock ServUO 57.4) | Used for |
|
||||
|---|---|---|
|
||||
| `Spawns/*.xml` | 13 files, ~10.5 MB | Every spawner: location, size, delays, time-of-day, creature types |
|
||||
| `Data/Regions.xml` | 129 KB, nested | Named regions and their rectangles |
|
||||
| `Data/Locations/*.xml` | 6 files | Landmarks (dungeon levels, town markers) |
|
||||
| `Config/ChampionSpawns.xml` | 4.8 KB | Configured champion altars |
|
||||
|
||||
**A stock tree has 13 spawn files but only 6 facets.** `Eodon.xml`,
|
||||
`GravewaterLake.xml`, `TreasuresOfKotl.xml` and the other named-area files hold
|
||||
TerMur/Trammel points. The facet always comes from each record's own `<Map>`,
|
||||
never from the file name.
|
||||
|
||||
## How a coordinate becomes a place name
|
||||
|
||||
This is the transform the atlas exists for, in `resolveRegion()`:
|
||||
|
||||
1. The highest-`priority` named region whose rectangle contains the point. Ties
|
||||
break toward the **smallest** rect, so a specific room wins over the
|
||||
dungeon-wide rect enclosing it.
|
||||
2. Otherwise the nearest landmark within the landmark radius (200 tiles by
|
||||
default), labelled by its **group** ("Covetous"), not its individual marker
|
||||
("Level 1").
|
||||
3. Otherwise `"Wilderness"`.
|
||||
|
||||
The radius cap in step 2 is what keeps step 3 reachable. Without it the nearest
|
||||
landmark is always *some* landmark however far away, and open countryside gets
|
||||
labelled with a dungeon on the far side of the map.
|
||||
|
||||
Against stock ServUO this resolves **83.2%** of points (5,369 of 6,455): 3,681 by
|
||||
region, 1,688 by landmark, 1,086 Wilderness.
|
||||
|
||||
## Three quirks in the source data
|
||||
|
||||
Each of these is silent if unhandled — the atlas still builds, it is just wrong.
|
||||
|
||||
**Facet names disagree between sources.** `Data/Locations/*.xml` spells them
|
||||
`Ter Mur` and `Tokuno Islands`, while `<Map>` and `<Facet name>` say `TerMur` and
|
||||
`Tokuno`. Unreconciled, the landmark bucket is keyed differently from the points
|
||||
looking it up, so the fallback never fires and every unregioned spawn on those
|
||||
facets reads "Wilderness".
|
||||
|
||||
This is reconciled **by matching, not by a lookup table** — there is no list of
|
||||
facet names anywhere. `facetKey()` collapses spelling differences (lowercase,
|
||||
alphanumerics only), and `resolveFacetName()` matches a loose spelling against
|
||||
the canonical set discovered from the shard's own spawn and region data, by exact
|
||||
key then by prefix in either direction. A name matching nothing keeps its own
|
||||
name: forcing a wrong match would file a real custom facet's landmarks under the
|
||||
wrong facet, which is worse than leaving it alone.
|
||||
|
||||
**Spawn type tokens carry XmlSpawner directives.** The `<Objects2>` type is not
|
||||
always a bare class name:
|
||||
|
||||
```
|
||||
Fairy,{RND,4,8} alchemist/z/-50 Agralem/Name/Agralem
|
||||
GargishRouser,1 greatape,true GargishRefugee/hue/34532
|
||||
```
|
||||
|
||||
Taken literally these invent creatures that do not exist *and* split real ones in
|
||||
two, because `Fairy` and `Fairy,{RND,4,8}` slug apart into separate entries. 71 of
|
||||
845 were affected. Everything from the first `/` or `,` is stripped, leaving 800
|
||||
real creatures.
|
||||
|
||||
**Case is inconsistent across files.** The same creature is `Lizardman` in one
|
||||
file and `lizardman` in another. Slugging collapses them correctly, but the
|
||||
display name is chosen deterministically — most common spelling wins, ties break
|
||||
to the more capitalised form, then alphabetically — because otherwise it would
|
||||
depend on file read order and change on an unrelated restart.
|
||||
|
||||
## Tables
|
||||
|
||||
All are **import-owned**: a refresh empties and reloads them in one transaction,
|
||||
so a failed reload leaves the previous atlas intact rather than a half-loaded
|
||||
world. Nothing else writes to them and nothing holds a foreign key to them — no
|
||||
FKs at all, consistent with every other `shard_*` table. Full column listings in
|
||||
[`BACKEND_DESIGN.md`](BACKEND_DESIGN.md).
|
||||
|
||||
| Table | Rows (stock) | Notes |
|
||||
|---|---|---|
|
||||
| `shard_spawn_creatures` | 800 | `slug` PK; `total` = sum of each type's own max; nullable `art` |
|
||||
| `shard_spawn_points` | 6,455 | `spawn_range`, since `range` is reserved in MariaDB |
|
||||
| `shard_spawn_point_types` | 23,927 | The many-to-many; one spawner commonly carries six types |
|
||||
| `shard_regions` | 387 | Flattened out of the nesting; `rects` JSON |
|
||||
| `shard_landmarks` | 558 | `grp`, since `group` is reserved in SQL |
|
||||
| `shard_champion_spawns` | 25 | Configured altars, not the live feed |
|
||||
| `shard_atlas_meta` | 1 | Singleton; source hashes, for the change check |
|
||||
| `shard_atlas_pending` | 0–1 | Singleton; a staged refresh awaiting admin review |
|
||||
|
||||
`shard_spawn_creatures.name` carries a plain `INDEX`, deliberately **not
|
||||
`FULLTEXT`**: ~800 rows makes a `LIKE` scan free, and FULLTEXT's minimum token
|
||||
length would break searches for names like "orc".
|
||||
|
||||
The reload uses `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and would
|
||||
implicitly commit, defeating the all-or-nothing guarantee. Point ids are assigned
|
||||
explicitly rather than left to `AUTO_INCREMENT`, because the join rows need them
|
||||
and `conn.batch()` reports no usable `insertId`.
|
||||
|
||||
## Artwork — operator-supplied, never shipped
|
||||
|
||||
**This project ships no creature art and no extraction tooling, and never will.**
|
||||
UO sprites live in the operator's own client `.mul`/`.uop` files. They are the
|
||||
operator's, not ours to redistribute.
|
||||
|
||||
The atlas is fully functional as text. `shard_spawn_creatures.art` is nullable
|
||||
and is NULL on every fresh import; pages render without images, which is the
|
||||
normal and supported state, not a degraded one.
|
||||
|
||||
An operator who wants art:
|
||||
|
||||
1. Extracts it from **their own** client files (UOFiddler, ClassicUO tooling, or
|
||||
any art extractor).
|
||||
2. Drops the images under `server/uploads/atlas/`.
|
||||
3. Copies `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json`
|
||||
and maps creature slugs to file names.
|
||||
4. Restarts, or runs `npm run atlas:import -- --force`.
|
||||
|
||||
Both `spawnAtlas.art.json` and `server/uploads/` are gitignored, so neither the
|
||||
map nor the images can be committed by accident.
|
||||
|
||||
## Code layout
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `src/utils/spawnAtlasParse.js` | **Pure and fs-free** parsers, so CI covers them with no ServUO tree. Zero dependencies. |
|
||||
| `src/utils/spawnAtlasSource.js` | The only thing that reads a ServUO tree; shared by the boot path and the CLI |
|
||||
| `src/model/shardAtlas/shardAtlas.db.js` | The one-transaction replace |
|
||||
| `src/model/shardAtlas/shardAtlas.model.js` | The refresh decision, staging, approve/reject |
|
||||
| `scripts/importSpawnAtlas.js` | Thin CLI over the model |
|
||||
|
||||
Parsing notes:
|
||||
|
||||
- `Regions.xml`, `Locations/*.xml` and `ChampionSpawns.xml` genuinely nest, and
|
||||
get a small hand-rolled **subset** tokenizer — elements, attributes,
|
||||
self-closing tags, comments, the XML declaration, CDATA, and the five
|
||||
predefined entities plus numeric refs. It is not a general-purpose XML parser
|
||||
and must not be reused as one.
|
||||
- The ~10.5 MB of `Spawns/*.xml` never touches that tokenizer. Those records are
|
||||
flat, so they get a streaming regex sweep instead; a DOM would allocate a node
|
||||
per element across ~40 fields on every record to keep 14 of them. **Do not put
|
||||
the Points files through a DOM parser.**
|
||||
- `<Objects2>` is `Type:MX=n:SB=…` segments joined by `:OBJ=`. Split on `:OBJ=`
|
||||
*first* — a naive `split(':')` shreds it. A single Trammel point carries six
|
||||
types.
|
||||
@@ -90,6 +90,19 @@ Pattern-identical to `mobile_refresh_tokens`; stores only the token hash.
|
||||
|
||||
Indices: `idx_td_user (user_id)`, `idx_td_expires (expires_at)`.
|
||||
|
||||
### `mobile_auth_sessions.trust_device` (SSO bridge)
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| trust_device | TINYINT(1) NOT NULL DEFAULT 0 | user ticked "trust this device" on the Custom Tab TOTP form |
|
||||
|
||||
A **boolean only**. It records the user's choice so `POST /auth/mobile/sso/exchange`
|
||||
knows to mint the app's own trust token over that authenticated app→server call; the
|
||||
token itself is never written here (only its sha256 reaches `trusted_devices`). Set
|
||||
only while the session is still `pending` and unexpired, for the same reason
|
||||
`completeSession` is guarded — a replayed TOTP post must not re-arm a consumed
|
||||
session.
|
||||
|
||||
### `recovery_codes`
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
@@ -134,6 +147,42 @@ succeeds — only the trust marker is withheld. The web client then renders a mo
|
||||
`trustToken` the app stores in EncryptedSharedPreferences and replays on a later
|
||||
login to skip TOTP. Same cap behavior.
|
||||
|
||||
#### SSO login paths
|
||||
|
||||
SSO is **not** exempt: an account with TOTP on is asked for a code after a
|
||||
Google/Discord sign-in exactly as it is after a password one, and a trusted device
|
||||
skips that code exactly the same way. (Originally SSO consulted trust nowhere, so a
|
||||
user who signed in with an external identity was asked for a code on *every* sign-in
|
||||
no matter how many times they had ticked "trust this device".)
|
||||
|
||||
- `GET /auth/sso/:provider/callback` — once the account is resolved and before a
|
||||
TOTP challenge is staged, resolve the presented trust (cookie, or `X-Trust-Token`)
|
||||
and, if it belongs to **this** user, skip the code, stamp `last_used_at`, and log
|
||||
`auth.login.trusted_device`. The first factor is the IdP authentication that just
|
||||
succeeded, so this is the same posture as the password path. A store error falls
|
||||
through to the challenge — fail **closed** to asking for the code.
|
||||
- `POST /auth/sso/totp` — gains optional `trustDevice` + `deviceName`, mints the
|
||||
trust and sets the `rg_trust` cookie on success. At the cap the sign-in still
|
||||
completes and the response carries `{ trustLimitReached, devices }`, matching
|
||||
`POST /auth/login/totp`. Recovery codes remain password-login only: this step
|
||||
verifies an authenticator code against the staged challenge.
|
||||
|
||||
**How this reaches the Android app.** The app's SSO runs in a Custom Tab, which
|
||||
shares the system browser's cookie jar, so both halves land in the same place: the
|
||||
`rg_trust` cookie set on the Custom Tab TOTP form is presented back on the *next*
|
||||
app SSO sign-in and skips the code — no app change, and no trust token smuggled
|
||||
through a start URL where it would leak into query strings and logs.
|
||||
|
||||
To cover the app's **native** password login on the same device as well, ticking
|
||||
the box also sets `mobile_auth_sessions.trust_device` (a boolean — never the
|
||||
token), and `POST /auth/mobile/sso/exchange` then mints a `platform: 'mobile'`
|
||||
trust and returns `{ trustToken }` in its JSON body. Minting at exchange time is
|
||||
deliberate: it is an authenticated app→server call, so the raw token never travels
|
||||
in the deep link and never rests in the bridge row. One tick therefore produces two
|
||||
independently-revocable rows (the browser and the app) — which is honest, since they
|
||||
are two distinct credentials on one device. If the user is at the cap, the exchange
|
||||
simply returns no token; it never turns a successful sign-in into an error.
|
||||
|
||||
### Self-service (`/auth/me/*`, `requireAuth`, any role)
|
||||
- `GET /auth/me/trusted-devices` — list active trusted devices (never tokens).
|
||||
- `POST /auth/me/trusted-devices` — trust the current browser/device (cap-checked).
|
||||
|
||||
@@ -54,7 +54,7 @@ auth model (admin/editor — no new roles, no public contributions).
|
||||
| Schema | flat `wiki_pages(slug,title,body,updated_by,timestamps)` | [server/db/schema.sql:31](server/db/schema.sql) |
|
||||
| Model | thin CRUD by slug | [server/src/model/wiki/wiki.db.js](server/src/model/wiki/wiki.db.js), [wiki.model.js](server/src/model/wiki/wiki.model.js) |
|
||||
| Public API | `GET /public/wiki`, `GET /public/wiki/:slug` | [public.controller.js:53](server/src/router/v1/public/public.controller.js) |
|
||||
| Admin API | `GET/POST/PUT/DELETE /admin/wiki[...]` | [admin.controller.js:163](server/src/router/v1/admin/admin.controller.js), [admin.routes.js:68](server/src/router/v1/admin/admin.routes.js) |
|
||||
| Admin API | `GET/POST/PUT/DELETE /admin/wiki[...]` | [admin.controller.js:163](server/src/router/v1/admin/admin.controller.js), [wiki.router.js](server/src/router/v1/admin/wiki.router.js) |
|
||||
| Public UI | card grid (hardcoded blurbs + Roman numerals), article w/ auto-TOC | [Wiki.jsx](client/src/routes/wiki/Wiki.jsx), [WikiArticle.jsx](client/src/routes/wiki/WikiArticle.jsx) |
|
||||
| Admin UI | raw-HTML `<textarea>` modal | [WikiAdmin.jsx](client/src/routes/admin/views/WikiAdmin.jsx), [WikiEditor.jsx](client/src/routes/admin/views/WikiEditor.jsx) |
|
||||
| API client | `api.wiki`, `api.admin.*Wiki` | [client/src/api/client.js:52](client/src/api/client.js) |
|
||||
@@ -196,7 +196,7 @@ centralized error handler unchanged. **Every write logs to `activity_log`**
|
||||
|
||||
### 4.4 Image uploads
|
||||
|
||||
Generalize the existing screenshot upload (multer config in [admin.routes.js:17](server/src/router/v1/admin/admin.routes.js))
|
||||
Generalize the existing screenshot upload (multer config now in [admin/imageUpload.js](server/src/router/v1/admin/imageUpload.js))
|
||||
into a shared `POST /admin/uploads` returning `{ url: "/uploads/<file>" }`, reused by both
|
||||
the post editor and the wiki editor. Same size/mime limits. No new storage —
|
||||
served from the existing `uploads/` volume.
|
||||
|
||||
Reference in New Issue
Block a user