Compare commits
1 Commits
cdea1aa7cd
...
docs/andro
| Author | SHA1 | Date | |
|---|---|---|---|
| cbaa18ea0b |
19
README.md
19
README.md
@@ -9,8 +9,6 @@ so they live in one place, independent of either codebase.
|
||||
```
|
||||
website/ docs from the shard website (Node/Express + MariaDB + React/Vite)
|
||||
link/ docs from the ServUO bridge (C# plugin + Rust sidecar + Node WS)
|
||||
android/ docs from the native Android client (Kotlin + Jetpack Compose)
|
||||
ci/ cross-cutting CI/quality notes
|
||||
```
|
||||
|
||||
### `website/`
|
||||
@@ -19,35 +17,18 @@ ci/ cross-cutting CI/quality notes
|
||||
| [BACKEND_DESIGN.md](website/BACKEND_DESIGN.md) | API contract, DB schema, security model |
|
||||
| [HERO_EDITOR.md](website/HERO_EDITOR.md) | Hero canvas editor feature spec |
|
||||
| [WIKI_UPGRADE.md](website/WIKI_UPGRADE.md) | Wiki subsystem upgrade notes |
|
||||
| [SHARD_VISIBILITY.md](website/SHARD_VISIBILITY.md) | Who sees which shard data — the admin-configurable audience framework |
|
||||
| [SPAWN_ATLAS.md](website/SPAWN_ATLAS.md) | The bestiary / spawn atlas: what the shard contains, parsed from its own ServUO tree |
|
||||
| [CLILOCS.md](website/CLILOCS.md) | UO's id → name table: converting one from your client so items have names |
|
||||
| [MARKETPLACE.md](website/MARKETPLACE.md) | The player-vendor index: how it is gathered, what it costs, how to tune it |
|
||||
| [website-README.md](website/website-README.md) | Snapshot of the website repo's README (setup/run reference) |
|
||||
| [PROJECT_TREE.md](website/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout |
|
||||
|
||||
### `link/`
|
||||
| Doc | What it covers |
|
||||
|---|---|
|
||||
| [INTEGRATION.md](link/INTEGRATION.md) | How the website integrates with the uo-link sidecar |
|
||||
| [PROTOCOL_2.md](link/PROTOCOL_2.md) | Protocol 2.0 / 2.1 design |
|
||||
| [v3.md](link/v3.md) | Protocol 3.0 design — shard content/standings streams + the visibility framework |
|
||||
| [ADMIN_CONTROLS.md](link/ADMIN_CONTROLS.md) | Staff write-plane (kick/ban/broadcast, page queue) |
|
||||
| [SHARD_PREREQS.md](link/SHARD_PREREQS.md) | Shard-side prerequisites for the bridge |
|
||||
| [PLAN.md](link/PLAN.md) | uo-link build plan |
|
||||
| [RESEARCH.md](link/RESEARCH.md) | Research notes |
|
||||
| [link-README.md](link/link-README.md) | Snapshot of the link repo's README |
|
||||
| [PROJECT_TREE.md](link/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout |
|
||||
|
||||
### `android/`
|
||||
| Doc | What it covers |
|
||||
|---|---|
|
||||
| [PLAN.md](android/PLAN.md) | Android client build plan / milestones |
|
||||
| [COVERAGE_PLAN.md](android/COVERAGE_PLAN.md) | Test-coverage rollout plan |
|
||||
| [APP_LINKS.md](android/APP_LINKS.md) | Android App Links / deep-link setup |
|
||||
| [theme-plan.md](android/theme-plan.md) | Theming plan |
|
||||
| [TRUSTED_DEVICES_APP_HANDOFF.md](android/TRUSTED_DEVICES_APP_HANDOFF.md) | Trusted-devices app handoff notes |
|
||||
| [PROJECT_TREE.md](android/PROJECT_TREE.md) | Auto-generated snapshot of the repo's tracked file layout |
|
||||
|
||||
## Provenance
|
||||
|
||||
|
||||
@@ -1,196 +0,0 @@
|
||||
# Android App Links — implementation spec
|
||||
|
||||
Status: **implementation spec (M9 follow-up).** Stacks on the native SSO bridge (M9 Part 2):
|
||||
the app already handles the **custom-scheme** callback `runicgateway://auth/callback`, and that stays
|
||||
the permanent default and universal fallback. App Links are an **opt-in hardening** layered on top —
|
||||
a verified `https://` callback that only the domain's real owner can claim.
|
||||
|
||||
Read alongside: the "Mobile SSO Authorization Bridge" section of
|
||||
[`../website/BACKEND_DESIGN.md`](../website/BACKEND_DESIGN.md) (endpoints/tables/allowlist), and
|
||||
[`PLAN.md`](./PLAN.md) §4.2 / §9 (the app milestone). This spec matches what ships on the
|
||||
`feat/mobile-app-links` (website) and `feat/app-links` (android) branches.
|
||||
|
||||
---
|
||||
|
||||
## 1. The problem it solves
|
||||
|
||||
The mobile SSO bridge redirects the browser back to the app with a one-time code:
|
||||
|
||||
```
|
||||
runicgateway://auth/callback?code=…&state=…
|
||||
```
|
||||
|
||||
A **custom URI scheme** is fine for a self-hosted internal client, but it is not *owned* by anyone:
|
||||
any other Android app can register an intent-filter for `runicgateway://auth/callback` and, if chosen
|
||||
by the user, intercept the callback. The code is single-use, PKCE-bound (Layer B), and short-lived —
|
||||
so an interceptor still cannot complete `/exchange` without the app's `code_verifier` — but a hijacked
|
||||
callback is still a denial-of-service and a phishing surface we would rather close.
|
||||
|
||||
**Android App Links** (verified `https://` deep links) close it: the OS only routes an `https://`
|
||||
link to an app that has proven, via a file served from *that domain*, that it owns the app. An
|
||||
attacker cannot serve that file on a domain they do not control.
|
||||
|
||||
## 2. Why this is harder here than in a normal app
|
||||
|
||||
RunicGateway is **self-hosted per shard**. There is no single canonical domain — every shard owner
|
||||
runs the website on **their own** domain (`play.exampleshard.com`, `uo.anothershard.net`, …). App
|
||||
Links verification is **per-domain**: the domain must serve
|
||||
|
||||
```
|
||||
https://<shard-domain>/.well-known/assetlinks.json
|
||||
```
|
||||
|
||||
asserting the Android app's **package name** + **signing-certificate SHA-256 fingerprint**.
|
||||
|
||||
That is only half the problem. The other half is an Android platform constraint that decides the whole
|
||||
shape of the app side:
|
||||
|
||||
> **`android:autoVerify` needs a *literal* host at build time.** An intent-filter's `<data android:host>`
|
||||
> is a static string in the merged manifest; there is no "any host" or runtime host. A **single
|
||||
> published multi-tenant APK therefore cannot autoVerify an open-ended set of shard domains** — the set
|
||||
> is not known when the APK is built.
|
||||
|
||||
So App Links here are **not** a drop-in replacement for the custom scheme. They split into two pieces
|
||||
that ship independently:
|
||||
|
||||
1. **Server (`assetlinks.json`) — shippable now, benefits any App-Links-capable build.** Every shard
|
||||
can auto-serve its Digital Asset Links statement behind an admin toggle. This is a pure add and is
|
||||
implemented on `feat/mobile-app-links`.
|
||||
2. **App (`autoVerify` intent-filter) — a *build-time* opt-in.** Because the host must be baked in,
|
||||
App Links are available to:
|
||||
- a **white-label / first-party build** that bakes one shard's host (`-PappLinkHost=play.myshard.com`);
|
||||
- a future **canonical relay domain** (`runicgateway.app`, PLAN §14 — *not yet secured*) that all
|
||||
shards could bounce their final callback through, autoVerified by the generic build.
|
||||
|
||||
The **generic multi-tenant build bakes no host and stays custom-scheme-only** — correct and safe.
|
||||
|
||||
The custom scheme is never removed. It is the fallback on every build, for every shard, always.
|
||||
|
||||
## 3. Server design — `feat/mobile-app-links`
|
||||
|
||||
### 3.1 Auto-served `assetlinks.json`
|
||||
|
||||
- **Route:** `GET /.well-known/assetlinks.json`, served at the **web root** (outside `/api/v1`, before
|
||||
the SPA catch-all) in `server/src/app.js`.
|
||||
- **Gate:** the admin setting `mobile_app_links_enabled` (default **off**). Off ⇒ the route **404s** and
|
||||
the app stays on the custom scheme for that shard. On ⇒ the shard opts into App Links.
|
||||
- **Body:** the Digital Asset Links statement for the fixed package `com.runicgateway.app` and the
|
||||
release signing cert SHA-256 fingerprint(s):
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"relation": ["delegate_permission/common.handle_all_urls"],
|
||||
"target": {
|
||||
"namespace": "android_app",
|
||||
"package_name": "com.runicgateway.app",
|
||||
"sha256_cert_fingerprints": ["AB:CD:…"]
|
||||
}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
- **Fingerprint source:** env `MOBILE_APP_CERT_SHA256` — comma-separated (supports **cert rotation** and
|
||||
a debug + release cert during testing). It is a **constant of the published app**, identical for every
|
||||
shard, so it is a shipped/env default, not something each owner types. The package name is likewise
|
||||
fixed (`MOBILE_APP_PACKAGE`, default `com.runicgateway.app`).
|
||||
- **Enabled but no fingerprint configured ⇒ 404** (+ a one-time warn): serving a statement with no
|
||||
fingerprint asserts nothing and would only mislead the verifier.
|
||||
- Response is `application/json`, `Cache-Control: public, max-age=3600` (the Play verifier and the OS
|
||||
re-fetch it; it changes only on a cert rotation).
|
||||
|
||||
### 3.2 Redirect-URI allowlist extension
|
||||
|
||||
`mobileSso.controller` validates the app's `redirect_uri` by **exact match** against
|
||||
`MOBILE_AUTH_REDIRECT_URIS` (default `runicgateway://auth/callback`). App Links add exactly one more
|
||||
acceptable value, and **only when the toggle is on**:
|
||||
|
||||
- When `mobile_app_links_enabled`, `/start` additionally accepts the **self-origin** HTTPS callback
|
||||
`https://<request-host>/mobile/callback` (derived from the request/`APP_BASE_URL`, never from
|
||||
attacker-controlled input). Still **exact match** — never a prefix match.
|
||||
- The static custom-scheme allowlist is never narrowed; the HTTPS entry is *additive*.
|
||||
- No new table or schema: the check reads the one boolean setting.
|
||||
|
||||
### 3.3 Public settings advertise the capability
|
||||
|
||||
`settings.getPublic()` gains `mobileAppLinks: <bool>` (mirrors the toggle) so a client can tell whether
|
||||
a shard opted in before requesting an HTTPS `redirect_uri` (a white-label build uses it to avoid asking
|
||||
for a callback the server would reject).
|
||||
|
||||
## 4. App design — `feat/app-links`
|
||||
|
||||
### 4.1 Build-time host (`appLinkHost`)
|
||||
|
||||
- Gradle property `appLinkHost` (default empty). Wired in `app/build.gradle.kts` into **both**:
|
||||
- `BuildConfig.APP_LINK_HOST` — read by `SsoAuthManager` to decide the redirect;
|
||||
- `manifestPlaceholders["appLinkHost"]` — substituted into the App Link intent-filter's host.
|
||||
- **Default (generic build):** empty ⇒ `BuildConfig.APP_LINK_HOST = ""` and the placeholder falls back
|
||||
to the reserved sentinel `runic-gateway.invalid` (RFC 6761 — never resolves), so the `autoVerify`
|
||||
filter is **inert**: it matches no real link and verification simply never succeeds. No custom-scheme
|
||||
behaviour changes.
|
||||
- **White-label build:** `./gradlew assembleRelease -PappLinkHost=play.myshard.com` bakes that one host
|
||||
into the filter and enables the HTTPS redirect for that host.
|
||||
|
||||
### 4.2 Manifest
|
||||
|
||||
A second intent-filter on `MainActivity`, alongside the unchanged custom-scheme one:
|
||||
|
||||
```xml
|
||||
<intent-filter android:autoVerify="true">
|
||||
<action android:name="android.intent.action.VIEW" />
|
||||
<category android:name="android.intent.category.DEFAULT" />
|
||||
<category android:name="android.intent.category.BROWSABLE" />
|
||||
<data android:scheme="https"
|
||||
android:host="${appLinkHost}"
|
||||
android:path="/mobile/callback" />
|
||||
</intent-filter>
|
||||
```
|
||||
|
||||
### 4.3 `SsoAuthManager` (pure Kotlin, unit-tested on the JVM)
|
||||
|
||||
- **Redirect selection in `buildStartUrl`:** request the HTTPS `redirect_uri`
|
||||
`https://<pairedHost>/mobile/callback` **iff** `BuildConfig.APP_LINK_HOST` is non-blank *and* equals
|
||||
the paired base-URL host (case-insensitive); otherwise the fixed custom-scheme `REDIRECT_URI`. A
|
||||
white-label build that bakes the host is responsible for enabling the server toggle too (§3.2).
|
||||
- **Verified-callback matcher + host-trust check:** a new `matchesAppLinkCallback(scheme, host, path)`
|
||||
accepts only `scheme == https`, `path == /mobile/callback`, and **`host == the paired base-URL host`**.
|
||||
The paired-host equality is defense-in-depth: even though `autoVerify` already means only a real,
|
||||
opted-in shard domain can route here, the app still refuses any HTTPS callback whose host isn't the
|
||||
shard it is currently paired to.
|
||||
- The rest is unchanged: both matchers feed the *same* `complete(state, code, error)` → `/exchange` →
|
||||
`SessionManager.onSignedIn`. There is no second auth path.
|
||||
|
||||
### 4.4 `MainActivity`
|
||||
|
||||
`handleSsoCallback` routes a VIEW intent through **`matchesCallback(...) || matchesAppLinkCallback(...)`**;
|
||||
everything downstream (state check, exchange, sign-in) is shared. Custom-scheme and App Link callbacks
|
||||
are indistinguishable past the edge.
|
||||
|
||||
## 5. Turning it on for a shard
|
||||
|
||||
1. Publish/point the app build at the shard host (`-PappLinkHost=<host>`) — or use the generic build and
|
||||
leave App Links off.
|
||||
2. Set `MOBILE_APP_CERT_SHA256` (release cert fingerprint) in the website env.
|
||||
3. Admin → Shard/Settings: enable **App Links** (`mobile_app_links_enabled`).
|
||||
4. Verify `https://<host>/.well-known/assetlinks.json` returns the statement; confirm Android verifies
|
||||
(`adb shell pm get-app-links com.runicgateway.app`).
|
||||
|
||||
If any step is skipped the app transparently keeps using the custom scheme — nothing breaks.
|
||||
|
||||
## 6. Testing
|
||||
|
||||
- **Server (`node --test`):** route 404s when the toggle is off; 404s when on but no fingerprint;
|
||||
returns the correct statement + content-type when on and configured; the redirect allowlist accepts
|
||||
`https://<host>/mobile/callback` only when enabled and rejects it otherwise (custom scheme always
|
||||
accepted).
|
||||
- **App (JVM unit tests):** `matchesAppLinkCallback` accepts only https + `/mobile/callback` + the paired
|
||||
host and rejects a foreign host / http / wrong path; `buildStartUrl` requests the HTTPS redirect only
|
||||
when the baked host matches the paired host, else the custom scheme.
|
||||
|
||||
## 7. What does *not* change
|
||||
|
||||
- The bridge's server design (PKCE Layer A/B, single-use codes, `/start` + `/exchange`) is untouched;
|
||||
App Links are *one more allowlist entry* + *one static file route*. That is the whole point of keeping
|
||||
the allowlist exact-match and configurable from day one.
|
||||
- The custom scheme remains on every build and is the permanent fallback.
|
||||
- No change to `servuo-plugins/` — App Links are entirely a website ↔ app concern.
|
||||
@@ -1,204 +0,0 @@
|
||||
# Android App — Test Coverage Plan
|
||||
|
||||
**Goal:** clear the SonarQube coverage quality gate (`new_coverage ≥ 50%`) for
|
||||
`Runic-Gateway-Android-app`, and leave a durable unit-test culture behind it. Companion to
|
||||
[`PLAN.md`](./PLAN.md) §12.1 (the JaCoCo wiring that made coverage measurable).
|
||||
|
||||
## 1. Current state (2026-07-22, post `Android-app#26`)
|
||||
|
||||
The JaCoCo→Sonar wiring is live on `main`, so coverage is now real — and the gate is **failing**:
|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| Quality gate | **ERROR** (one condition) |
|
||||
| `new_coverage` | **16.4%** (threshold ≥ 50%) |
|
||||
| overall `coverage` | 16.3% |
|
||||
| lines to cover | 3,188 |
|
||||
| covered | 542 |
|
||||
|
||||
Everything else on the gate is green (reliability/security/maintainability **A**, duplication 0%).
|
||||
Because this is the *first* measured version, Sonar's "new code" window is essentially the whole
|
||||
codebase, so `new_coverage ≈ overall coverage` — to pass we need to roughly **triple** covered
|
||||
lines, from 542 to ~1,600.
|
||||
|
||||
### Where the uncovered lines are
|
||||
|
||||
Bucketed from Sonar's per-file `uncovered_lines` (exclusions from #26 already applied, so `*Screen.kt`
|
||||
is absent):
|
||||
|
||||
| Bucket | Files | Lines to cover | Covered % | Verdict |
|
||||
|---|--:|--:|--:|---|
|
||||
| **ViewModels** | 28 | 1,153 | **0.1%** | **Test** — the dominant lever; no ViewModel has any test |
|
||||
| **DTOs** | 12 | 648 | 26.7% | **Test** — trivial (serialization); a pattern already exists |
|
||||
| **Repositories** | 11 | 274 | 11.3% | **Test** — fake the API interface |
|
||||
| **core/\* (logic)** | 21 | 365 | 56.2% | **Test** — top up the partially-covered ones |
|
||||
| UI composables (`*Components.kt`, `BlockRenderer`, …) | 7 | 255 | 3.9% | **Exclude** — not JVM-unit-testable, and `*Screen.kt`'s exclusion missed them |
|
||||
| core/push (Android services) | 4 | ~191 | ~0% | **Exclude** (or Robolectric later) — foreground service / notifications |
|
||||
| core/auth `Encrypted*` stores | 3 | 76 | 0% | **Exclude** — Android Keystore / EncryptedSharedPreferences |
|
||||
| framework glue (`di/`, `RunicGatewayApp`, `LocalAssetResolver`) | 3 | ~17 | 0% | **Exclude** |
|
||||
|
||||
**Two levers, applied together:** (a) stop *counting* code a JVM unit test physically cannot execute,
|
||||
and (b) actually *test* the logic — ViewModels, DTOs, repositories, core utilities.
|
||||
|
||||
## 2. Strategy & projected math
|
||||
|
||||
Numbers below are line-coverage projections against Sonar's `lines_to_cover`. They are estimates, but
|
||||
grounded in the current per-bucket totals.
|
||||
|
||||
### Phase 0 — Broaden coverage exclusions (no tests; ~½ day)
|
||||
|
||||
Move non-unit-testable code out of the **coverage** denominator (it stays in *analysis* — bugs and
|
||||
smells are still reported). Extend `sonar.coverage.exclusions` in `sonar-project.properties`:
|
||||
|
||||
```properties
|
||||
sonar.coverage.exclusions=\
|
||||
app/src/main/java/**/ui/**/*Screen.kt,\
|
||||
app/src/main/java/**/ui/**/*Screen*.kt,\
|
||||
app/src/main/java/**/ui/**/*Components.kt,\
|
||||
app/src/main/java/**/ui/page/BlockRenderer.kt,\
|
||||
app/src/main/java/**/ui/components/**,\
|
||||
app/src/main/java/**/ui/shard/FrameFields.kt,\
|
||||
app/src/main/java/**/ui/theme/**,\
|
||||
app/src/main/java/**/ui/LocalAssetResolver.kt,\
|
||||
app/src/main/java/**/RunicApp.kt,\
|
||||
app/src/main/java/**/MainActivity.kt,\
|
||||
app/src/main/java/**/*Application.kt,\
|
||||
app/src/main/java/**/RunicGatewayApp.kt,\
|
||||
app/src/main/java/**/di/**,\
|
||||
app/src/main/java/**/core/push/PushService.kt,\
|
||||
app/src/main/java/**/core/push/PushManager.kt,\
|
||||
app/src/main/java/**/core/push/PushNotifier.kt,\
|
||||
app/src/main/java/**/core/push/NtfyStreamClient.kt,\
|
||||
app/src/main/java/**/core/auth/Encrypted*.kt
|
||||
```
|
||||
|
||||
> Verify each glob targets composable-only / framework-only files before committing (e.g. confirm
|
||||
> `FrameFields.kt` holds no testable logic). Keep the *pure-logic* push files in coverage
|
||||
> (`PushPreferences`, `PushTickle`, `NtfyTopic`, `PushStreams`) — they already have tests.
|
||||
|
||||
Effect: denominator ~3,188 → ~2,650; covered ~542 → ~532. **Coverage ≈ 20%.** (Removing ~540 lines
|
||||
that were ~2% covered.)
|
||||
|
||||
### Phase 1 — DTO serialization tests (highest ROI; ~1 day) → ~34%
|
||||
|
||||
DTOs are `@Serializable` data classes; test them with kotlinx-serialization round-trips against
|
||||
representative backend JSON. The pattern already exists (`AccountDtoTest`, `AuthDtoTest`,
|
||||
`NotificationsDtoTest`, `PlayerShardDtoTest`, `ShardDtoTest`). Add/extend:
|
||||
|
||||
- **New:** `AdminDto` (111 uncov — biggest single file), `WikiDto` (51), `PublicDto` (32),
|
||||
`PageDto` (16), `PostDto` (12), `ContactDto` (11), `SsoDto`, `PageDto`.
|
||||
- **Extend to ~85%:** `PlayerShardDto` (30.8%), `ShardDto` (35.6%), `AccountDto` (50.7%),
|
||||
`AuthDto` (47.4%).
|
||||
|
||||
Target DTOs to ~85%: **+~380 covered lines** → covered ~912 / ~2,650 ≈ **34%**.
|
||||
|
||||
### Phase 2 — ViewModel tests (the big one; ~3–4 days) → clears the gate
|
||||
|
||||
28 ViewModels, ~1,153 lines, currently 0%. This is where the gate is won. Requires a small test
|
||||
harness (§3). Each test drives the VM with fake collaborators and asserts `UiState` transitions
|
||||
(loading → success/error, form validation, actions).
|
||||
|
||||
Priority by uncovered lines:
|
||||
|
||||
1. `LoginViewModel` (116), `AccountViewModel` (104), `CharactersViewModel` (77),
|
||||
`AdminContentViewModel` (74), `NotificationsViewModel` (66), `TrustedDevicesViewModel` (63),
|
||||
`ShardViewModel` (59)
|
||||
2. `HousesViewModel` (51), `GovernorsViewModel` (47), `AdminDashboardViewModel` (43),
|
||||
`AdminSupportViewModel` (41), `ChampsViewModel` (40), `AdminModerationViewModel` (40),
|
||||
`GuildsViewModel` (39), `RecoveryCodesViewModel` (38), `ConnectViewModel` (37),
|
||||
`ContactViewModel` (36), `VendorsViewModel` (32), `AppViewModel` (29)
|
||||
3. The small ones (`PostViewModel`, `NewsViewModel`, `WikiViewModel`, `CharacterViewModel`,
|
||||
`MyHousesViewModel`, `WikiPageViewModel`, `PageViewModel`, `HomeViewModel`, `SessionViewModel`)
|
||||
|
||||
Target ViewModels to ~70%: **+~800 covered lines** → covered ~1,712 / ~2,650 ≈ **65%. ✅ Gate passes.**
|
||||
|
||||
> Phases 0 + 2 alone (skipping DTOs) already reach ~50.5% — but DTOs are cheap insurance and Phase 1
|
||||
> lands first because it de-risks the harness work.
|
||||
|
||||
### Phase 3 — Repository tests (~1–2 days) → margin
|
||||
|
||||
Repositories map API `Response`/exceptions to `ApiResult`; test with a fake `*Api` interface (or
|
||||
OkHttp `MockWebServer`). Priority: `AuthRepository` (89), `ConnectionRepository` (43),
|
||||
`ShardRepository` (27), `AdminRepository` (21), then the small content/wiki/player repos. Target ~70%:
|
||||
**+~160 lines** → buffer well above 50% and resilience as the new-code window narrows.
|
||||
|
||||
### Phase 4 — core/net + core/auth top-up (~½–1 day) → durability
|
||||
|
||||
Fill the partially-covered utilities: `TokenAuthenticator` (31), `ShardStreamClient` (36),
|
||||
`HostSelectionInterceptor` (9), `ApiResult` (6), `WebHandoff`, `WebsiteUrls`, `DeviceNameProvider`,
|
||||
`ServerPreferences`, `AppConfig`.
|
||||
|
||||
### Trajectory
|
||||
|
||||
| After | Denominator | Covered | Coverage |
|
||||
|---|--:|--:|--:|
|
||||
| Today | 3,188 | 542 | 16.3% |
|
||||
| Phase 0 (exclusions) | ~2,650 | ~532 | ~20% |
|
||||
| Phase 1 (DTOs) | ~2,650 | ~912 | ~34% |
|
||||
| **Phase 2 (ViewModels)** | ~2,650 | ~1,712 | **~65% ✅** |
|
||||
| Phase 3 (repos) | ~2,650 | ~1,872 | ~71% |
|
||||
| Phase 4 (core) | ~2,650 | ~2,000+ | ~75%+ |
|
||||
|
||||
## 3. Test infrastructure to add
|
||||
|
||||
The existing suite tests pure-logic classes only; ViewModel/coroutine testing needs a little scaffold.
|
||||
`kotlinx-coroutines-test` is already a `testImplementation` dependency.
|
||||
|
||||
**`MainDispatcherRule`** (JUnit4) — swaps `Dispatchers.Main` (used by `viewModelScope`) for a test
|
||||
dispatcher:
|
||||
|
||||
```kotlin
|
||||
// app/src/test/java/com/runicgateway/app/util/MainDispatcherRule.kt
|
||||
@OptIn(ExperimentalCoroutinesApi::class)
|
||||
class MainDispatcherRule(
|
||||
private val dispatcher: TestDispatcher = StandardTestDispatcher(),
|
||||
) : TestWatcher() {
|
||||
override fun starting(d: Description) = Dispatchers.setMain(dispatcher)
|
||||
override fun finished(d: Description) = Dispatchers.resetMain()
|
||||
}
|
||||
```
|
||||
|
||||
**ViewModel test pattern** — hand-written fakes (matches the repo's existing no-mock convention; no new
|
||||
dependency):
|
||||
|
||||
```kotlin
|
||||
class LoginViewModelTest {
|
||||
@get:Rule val mainDispatcher = MainDispatcherRule()
|
||||
|
||||
private class FakeAuthRepository(var result: LoginResult) : AuthRepository { /* stub the seam */ }
|
||||
|
||||
@Test fun `blank credentials surface INVALID_CREDENTIALS without a network call`() = runTest {
|
||||
val vm = LoginViewModel(FakeAuthRepository(LoginResult.Success), /* … */)
|
||||
vm.submit()
|
||||
assertEquals(LoginError.INVALID_CREDENTIALS, vm.state.value.error)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- Assert on `viewModel.state.value` after `advanceUntilIdle()`; or collect the `StateFlow` in a
|
||||
background `launch` when you need to see intermediate (loading) states.
|
||||
- **Optional deps (decide once):** `mockk` would cut fake-writing for wide interfaces, and `turbine`
|
||||
simplifies Flow assertions. Recommendation: **stay with hand fakes** to match convention; revisit
|
||||
only if VM tests get boilerplate-heavy.
|
||||
|
||||
## 4. Execution notes
|
||||
|
||||
- CI already runs `./gradlew testDebugUnitTest jacocoTestReport` before the scan (`sonarqube.yml`),
|
||||
so new tests count automatically on merge to `main`. Locally on this machine: JDK 21 needs
|
||||
`-Pksp.incremental=false`.
|
||||
- Sonar recomputes the gate on the post-merge scan; there's no way to fully confirm the number
|
||||
pre-merge. Land phases as separate PRs (0, 1, 2, …) so coverage climbs visibly and reviews stay
|
||||
small.
|
||||
- `sonar.coverage.exclusions` removes files from **coverage only** — analysis still flags bugs/smells
|
||||
in them, so excluding UI/framework code is safe.
|
||||
- **Android-framework code deferred, not abandoned:** push services and `Encrypted*` stores are
|
||||
excluded now; if we want them covered later, add Robolectric (`testImplementation`) and a
|
||||
`RobolectricTestRunner` suite rather than instrumented tests, to keep it in the fast JVM `test`
|
||||
source set the scan already consumes.
|
||||
|
||||
## 5. Definition of done
|
||||
|
||||
- `new_coverage ≥ 50%` and the SonarQube quality gate is **green**.
|
||||
- `MainDispatcherRule` + a documented ViewModel test pattern exist and are reused.
|
||||
- Coverage exclusions list only genuinely non-unit-testable files (UI composables, Android-framework
|
||||
glue) — no ViewModel, repository, DTO, or pure core-logic file is excluded.
|
||||
535
android/PLAN.md
535
android/PLAN.md
@@ -1,6 +1,6 @@
|
||||
# Android App — Plan
|
||||
|
||||
Status: **M0–M7 landed; M7 (push notifications) both parts done — Part 1 backend (website#78) and Part 2 app (Android-app#15) plus a small `push.ntfyUrl` settings addition (website#79). Remaining: set the shard's `NTFY_*` deploy config so push lights up, and cut the v1 tag. M9 (native SSO login) is now underway backend-first — the Mobile SSO Authorization Bridge is being built in `website/` + `docs/` ahead of the app-side client (§4.2, §9 M9); custom-scheme callback only for now, App Links deferred (see [`APP_LINKS.md`](./APP_LINKS.md)).** This document is the
|
||||
Status: **M0–M6 landed; the functional build, design pass, and release mechanics are complete (the auto-release engine cuts a tagged, signed APK on merge to `main`; M7 push notifications next).** This document is the
|
||||
design contract for the `RunicGateway/Android-app` repo. It was written before implementation so the
|
||||
API changes it depends on could be landed in `website/` and `docs/` first. The authoritative API
|
||||
reference is the committed OpenAPI spec at `website/server/swagger/swagger-output.json` (regenerated
|
||||
@@ -128,266 +128,22 @@ probe refuses a backend whose API version this build can't speak (a future `v2`)
|
||||
signed release) with keep-rules for the kotlinx.serialization serializers, the wire DTOs, and the
|
||||
Retrofit interfaces; a release `signingConfig` that reads keystore material from a **gitignored**
|
||||
`keystore.properties` or env vars (absent → unsigned; the keystore is never committed); and
|
||||
`versionName`/`versionCode` overridable via `-P` so a release tag + CI run number drive them (§10).
|
||||
**CI `release.yml`** — on a `v*` tag, builds a **signed** APK (keystore decoded from a base64 Gitea
|
||||
secret) and attaches it + `SHA256SUMS` to a Gitea release; `workflow_dispatch` is a signing dry run.
|
||||
`versionName` is the committed source of truth (bumped by the release engine); `versionCode` is derived
|
||||
from it (`major*10000+minor*100+patch`, monotonic); both stay `-P`-overridable for local builds (§10).
|
||||
**CI `release.yml`** — mirrors `link/`'s language-agnostic release engine, adapted for Android: on every
|
||||
push to `main` it derives the next version from conventional-commit subjects since the last `v*` tag
|
||||
(`feat!`/BREAKING → major, `feat` → minor, `fix`/`perf` → patch; nothing releasable → no release),
|
||||
generates a grouped changelog, bumps `build.gradle.kts`, builds the **signed** APK (keystore decoded from
|
||||
a base64 Gitea secret), then commits the bump `[skip ci]`, tags `vX.Y.Z`, and creates the Gitea release
|
||||
with notes + APK + `SHA256SUMS`. Uses `REGISTRY_USER`/`REGISTRY_TOKEN` (as `link/` does) to push the bump
|
||||
and create the release, so `main` must allow that account to push.
|
||||
HTTPS-only in release (M1), no token logging (logging is debug-gated, M3), and the Settings → Server
|
||||
hard reset (M3) were already in place. Biometric app-lock is **descoped from v1** (see the note below).
|
||||
|
||||
**The functional build (M0–M4), design pass (M5), and release mechanics (M6) are complete. Cutting
|
||||
the first `v*` release tag (once the signing secrets are set + the on-device QA pass is done) and M7
|
||||
push notifications are what remain.**
|
||||
|
||||
### M7 plan — push notifications (in progress)
|
||||
|
||||
M7 spans three repos, so it ships in **two parts**; the backend contract lands first because the app
|
||||
is a pure consumer of it (§8/§11).
|
||||
|
||||
**Part 1 — `website/` backend + `docs/` — ✅ LANDED** (2026-07-20, `RunicGateway/website#78` merged
|
||||
+ docs#20). Additive, v1-only (new tables/routes/compose
|
||||
service; no existing response shape changes). Decision: **no ntfy publish token** — publishes go over
|
||||
the internal compose network to unguessable per-device topics carrying **content-free tickles**
|
||||
(`{ stream, ref }`); the publisher honors an optional `NTFY_PUBLISH_TOKEN` if ever set but requires
|
||||
none (keeps §11's zero-interaction promise).
|
||||
- **Two event sources, one publisher.** The fan-out is a small transport-agnostic
|
||||
`utils/pushDispatch.js` that both producers call: `utils/shardIngest.js` (`ingest()`, beside the
|
||||
existing `broadcast(event)`) for shard-derived streams, and the admin create-post path for the
|
||||
`news.post` stream (§11 lists news posts as a public stream, but they originate in the website, not
|
||||
the shard feed).
|
||||
- **Stream catalog** (`config/notificationStreams.js`): public/opt-in — `news.post`,
|
||||
`server.status`, `idoc.warning`, `champ.start`, `governor.election`; personal/owner-keyed
|
||||
(require a linked game account) — `vendor.sale`, `house.idoc`, `account.login`. `mapShardEvent()`
|
||||
maps event kinds → streams, drawing public streams **only** from the SSE `PUBLIC_KINDS` allowlist;
|
||||
sensitive kinds are never fanned out publicly. Personal events are delivered only to the owning
|
||||
user's devices, resolved via `shardLinks.getByAccount` (same ownership source as `/player/shard/*`).
|
||||
- **Tables:** `push_devices` (per-device endpoint) and `notification_subscriptions` (per-user opted-in
|
||||
streams), FK → `users` ON DELETE CASCADE, mirroring `mobile_refresh_tokens`.
|
||||
- **Routes** under the role-agnostic self surface (never `/admin`): `POST|GET /auth/me/devices`,
|
||||
`DELETE /auth/me/devices/:id`, `GET /auth/me/notifications/streams` (catalog),
|
||||
`GET|PUT /auth/me/notifications/subscriptions`. All bearer/cookie auth; Swagger regenerated.
|
||||
- **SSRF guard (important):** a device `endpoint` is a client-supplied URL the backend POSTs to, so
|
||||
registration and every publish validate it is HTTPS and its origin is in the shard's ntfy
|
||||
allow-set (`NTFY_BASE_URL` / `NTFY_ALLOWED_ORIGINS`), rejecting loopback/private hosts.
|
||||
- **ntfy** added to `website/docker-compose.yml` as a pinned upstream image with a committed
|
||||
declarative `./ntfy/server.yml` and named volume, **published on a host port** (`NTFY_HOST_PORT`,
|
||||
default `2586` → container `:80`) so the public reverse proxy — which runs *outside* the compose
|
||||
network — can forward the notification subdomain to it, anonymous read-write to unguessable topics
|
||||
(no per-user accounts — safe because tickles are content-free). Both the app (SSE subscribe) and the
|
||||
backend (POSTing tickles to registered device endpoints) reach ntfy on that public origin, so all
|
||||
ntfy traffic transits the proxy — there is no separate internal publish port.
|
||||
|
||||
**Part 2 — the Android app — ✅ LANDED** (2026-07-20, `RunicGateway/Android-app#15` + a small
|
||||
`RunicGateway/website#79` settings addition + this docs PR). Built exactly to the plan below, with
|
||||
two recorded implementation decisions:
|
||||
- **Direct-ntfy embedded distributor, no UnifiedPush library (deviation from §2's "UnifiedPush
|
||||
connector" wording — the plan's stated likely path, work item 1).** The app talks straight to ntfy
|
||||
over its own topic rather than pulling in `org.unifiedpush.android:connector` + an external
|
||||
distributor: a foreground `PushService` holds an OkHttp-SSE connection to `<ntfy>/<topic>/sse`
|
||||
(reusing the M2 `ShardStreamClient` reconnect pattern) on a **bare** client, `PushManager`
|
||||
orchestrates topic mint / device register / start-stop keyed to the session, and `PushNotifier`
|
||||
posts a per-stream notification whose tap deep-links via `MainActivity` intent extras. No new Gradle
|
||||
dependency; a `PushResult`/transport seam keeps the future FCM Play flavor cheap. Reasons: the
|
||||
UnifiedPush distributor model assumes a *separate* app (exactly what the user vetoed), we already own
|
||||
the SSE machinery, and this keeps the APK Google-free and dependency-light. New code lives in
|
||||
`core/push/` + `ui/notifications/` + a `NotificationsApi`/`NotificationsRepository`; no existing
|
||||
screen's data flow changed.
|
||||
- **One small additive backend field was required after all (`push.ntfyUrl`).** The embedded
|
||||
distributor must know the shard's client-facing ntfy URL to build its topic endpoint, and Part 1
|
||||
never surfaced it (the `NTFY_*` vars are server-only). So `/public/settings` now carries
|
||||
`push: { ntfyUrl }` (from `NTFY_PUBLIC_URL` / first `NTFY_ALLOWED_ORIGINS`; never the internal
|
||||
`NTFY_BASE_URL`), null when unconfigured → the app shows push as unavailable for that shard. This is
|
||||
the "no backend work in Part 2" caveat corrected: it is additive, non-sensitive, and forward-compatible
|
||||
(an older backend omitting it just decodes to null). **Deploy dependency stands:** push only delivers
|
||||
once the shard sets `NTFY_PUBLIC_URL`/`NTFY_ALLOWED_ORIGINS` (§13).
|
||||
|
||||
Verified green: `:app:testDebugUnitTest` (18 new JVM tests — notifications DTO decode, ntfy tickle
|
||||
parse incl. malformed, topic/URL building, stream→route map + personal gating) + `:app:lintDebug` +
|
||||
`:app:assembleDebug`; backend 250 tests (+3 for `push.ntfyUrl`) and `npm run swagger` clean. The
|
||||
foreground-service tradeoff (§11) and the POST_NOTIFICATIONS runtime permission are implemented as
|
||||
planned; an on-device delivery pass against a live ntfy is the one open QA item.
|
||||
|
||||
**Part 2 (original plan) — the Android app.** UnifiedPush receiver + device registration
|
||||
against the merged Part-1 contract, a Notifications settings screen, and notification-tap deep-links.
|
||||
The app is architected for push from M0 (§11), so this is **additive** — a new feature slice
|
||||
(`core/push` + `ui/notifications` + a `DevicesApi`/`NotificationsApi` pair) that touches no existing
|
||||
screen's data flow. Everything the app calls already exists and is merged; there is **no backend
|
||||
work** in Part 2.
|
||||
|
||||
The Part-1 contract the app codes against (verified against the merged `website` source):
|
||||
- `POST /auth/me/devices` `{ transport?: 'unifiedpush'|'fcm', endpoint, platform? }` → `201 PushDevice`
|
||||
`{ id, transport, endpoint, platform, createdAt, lastSeenAt }`. Idempotent per `(user, endpoint)`
|
||||
(upsert). `endpoint` **must** be HTTPS on the shard's ntfy allow-set — a private/loopback or
|
||||
off-allowlist origin is rejected `400` (the SSRF guard). Bearer-auth, so registration only happens
|
||||
while signed in.
|
||||
- `GET /auth/me/devices` → `PushDevice[]`; `DELETE /auth/me/devices/:id` → `{ ok: true }` (`404` if not
|
||||
the caller's).
|
||||
- `GET /auth/me/notifications/streams` → `{ streams: [{ id, label, description, personal,
|
||||
requiresLinkedAccount }] }` — the eight-stream catalog (`news.post`, `server.status`,
|
||||
`idoc.warning`, `champ.start`, `governor.election`; personal `vendor.sale`, `house.idoc`,
|
||||
`account.login`). Render from this, don't hardcode.
|
||||
- `GET /auth/me/notifications/subscriptions` → `{ streams: [id…] }`; `PUT` the same shape (full
|
||||
replace; unknown ids dropped server-side; the stored set is echoed back).
|
||||
- **The wire tickle** the device receives is the content-free `{ "stream": "<id>", "ref": "<opaque>" }`
|
||||
JSON body (`utils/pushDispatch.js`). `ref` is a serial / city / timestamp hint — **never** content.
|
||||
|
||||
Work items:
|
||||
|
||||
1. **Transport — the app is its own distributor; no second app (DECIDED).** The Runic Gateway app
|
||||
**embeds its own UnifiedPush distributor**. The self-hosted **ntfy is only the relay server**, never
|
||||
a user-installed app — the user installs *one* APK and it receives its own notifications, with no
|
||||
external distributor (no ntfy app, no NextPush) and no Google Play Services. Concretely, the embedded
|
||||
distributor holds a **persistent connection to the shard's ntfy** in a **foreground service**,
|
||||
reusing the OkHttp reconnect/backoff pattern already built for `core/net/ShardStreamClient` (M2): it
|
||||
subscribes to the app's own random, unguessable ntfy **topic** (over `wss://<ntfy-host>/<topic>/ws`
|
||||
or the `/json` stream) and forwards each received `{stream,ref}` tickle to the app's receiver. The
|
||||
**endpoint the app registers** with the backend (work item 5) is that topic's public URL
|
||||
(`https://<ntfy-host>/<topic>`) — exactly the client-supplied `endpoint` the merged `POST
|
||||
/auth/me/devices` contract expects and the URL the backend POSTs tickles to. Keep the transport
|
||||
behind a small `PushTransport` seam so the **future Play/FCM build flavor** (§11, §M8) can swap the
|
||||
embedded-ntfy distributor for FCM without touching registration, subscriptions, or notification code.
|
||||
(Implementation detail to confirm: whether a maintained Google-free embedded UnifiedPush-distributor
|
||||
library fits, or — more likely — a thin in-app distributor written directly over ntfy's subscribe API
|
||||
reusing `ShardStreamClient`. Either way the distributor lives **inside this app**; the UnifiedPush
|
||||
*receiver* abstraction is retained only to keep the FCM-flavor seam clean.)
|
||||
- **Tradeoff, accepted:** instant background delivery requires a persistent foreground service with
|
||||
an ongoing (low-importance) notification and its battery cost — this is exactly how ntfy's own app
|
||||
does instant delivery, and it is the price of Google-free self-delivery. A future "battery saver"
|
||||
option could fall back to periodic polling, but v1 ships the always-connected foreground service.
|
||||
2. **Deps + manifest.** Add the UnifiedPush connector + the embedded-distributor transport (per #1) to
|
||||
the version catalog; declare `POST_NOTIFICATIONS` (API 33+ runtime permission) **and
|
||||
`FOREGROUND_SERVICE` + `FOREGROUND_SERVICE_DATA_SYNC`** (API 34+, for the persistent ntfy
|
||||
connection); register the receiver and the foreground service in `AndroidManifest.xml`; define the
|
||||
notification channels (id/name externalized, §2) — one for real notifications plus a low-importance
|
||||
channel for the ongoing foreground-service notification — and reuse the "RG" notification icon
|
||||
**already staged in M6**.
|
||||
3. **`core/push` — embedded distributor + receiver.** The **distributor** component is a foreground
|
||||
service that owns the ntfy connection (per #1): it (re)creates the app's topic, subscribes over
|
||||
OkHttp with reconnect/backoff cloned from `ShardStreamClient`, and forwards each frame to the
|
||||
receiver. The **receiver** parses the `{ stream, ref }` tickle (`kotlinx.serialization`; an
|
||||
unknown/garbled body is dropped, not crashed — §7 discipline) and posts a notification (work item 7).
|
||||
Endpoint (re)registration against the backend fires on first subscribe / topic (re)creation
|
||||
(work item 5); a transient ntfy drop is just a reconnect, not a re-register.
|
||||
4. **`DevicesApi` + `NotificationsApi` (Retrofit) + DTOs.** Hand-authored, spec-aligned (as recorded
|
||||
for M1): `RegisterDeviceRequestDto`, `PushDeviceDto`, `NotificationStreamDto`,
|
||||
`NotificationStreamsDto`, `NotificationSubscriptionsDto`. Both go through the existing bearer/refresh
|
||||
stack (`AuthInterceptor` + `TokenAuthenticator`) and return the typed `ApiResult` (§7). A
|
||||
`NotificationsRepository` owns register/list/delete-device and get/put streams+subscriptions.
|
||||
5. **Endpoint ↔ backend lifecycle (mirror the M3 token teardown).** Persist the app's ntfy topic, its
|
||||
endpoint URL, and the returned device `id` in prefs (DataStore; the topic/endpoint isn't a secret —
|
||||
its security rests on being unguessable + the content-free tickle, §11). Start the embedded
|
||||
distributor and `POST /auth/me/devices` **only when the user has ≥1 subscription and is signed in**.
|
||||
On **logout / dead-refresh sign-out / Settings→Server switch**, `DELETE /auth/me/devices/:id`, **stop
|
||||
the foreground service**, and drop the topic — wire this into `SessionManager` beside the existing
|
||||
token-clear so a signed-out device stops receiving (§4.3, §11 "unregister on logout / token
|
||||
revocation"). On a **server (base-URL) switch**, mint a fresh topic against the new shard's ntfy (the
|
||||
old endpoint's origin won't be on the new host's allow-set). Re-assert the endpoint + restart the
|
||||
service on app start when signed-in + subscribed. A `400` on register (endpoint origin off the
|
||||
shard's `NTFY_ALLOWED_ORIGINS`) surfaces a clear "your shard's push relay isn't reachable" state, not
|
||||
a crash.
|
||||
6. **Notifications settings screen (`ui/notifications`).** Lists the catalog from
|
||||
`GET …/streams` with a per-stream toggle bound to `GET/PUT …/subscriptions`; a **personal** stream
|
||||
(`requiresLinkedAccount`) is greyed with a "link a game account" hint until the user has a linked
|
||||
account — reuse the linked-accounts signal already fetched for M4's player surface
|
||||
(`PlayerShardRepository`), not a fresh source of truth. Toggling to a non-empty set triggers the
|
||||
register flow (#5) and requests `POST_NOTIFICATIONS`; emptying the set unregisters. Each mutation
|
||||
folds its `ApiResult` into a section-scoped, localized banner (§7 parity with M4).
|
||||
7. **Deep-links (resolves the §13 open item).** Tapping a notification opens the app to the stream's
|
||||
home: `news.post`→News, `server.status`/`champ.start`/`idoc.warning`/`governor.election`→Shard,
|
||||
`vendor.sale`→Vendors, `house.idoc`→My Houses, `account.login`→My Account. Routed through the
|
||||
existing `ui/navigation/Routes.kt`; a signed-out/deep-link-to-player tap lands on the `PlayerGate`
|
||||
(M4) rather than erroring. **v1 shows a generic per-stream notification** (localized catalog
|
||||
`label`) and deep-links — it does **not** pull `ref` content first; the content-free design means
|
||||
nothing needs decrypting to render the tap, and the target screen fetches fresh over the
|
||||
authenticated API on open. (Pulling `ref` for a richer inline notification is a possible later
|
||||
enhancement, not v1.)
|
||||
8. **Menu.** Add a **Notifications** entry to the signed-in group in `ui/navigation/Menu.kt` (near My
|
||||
Account), visible once signed in.
|
||||
9. **Permission UX.** Request `POST_NOTIFICATIONS` at the moment the user first enables a stream (API
|
||||
33+); on denial, keep the toggle off and show how to enable it in system settings — never nag on
|
||||
launch.
|
||||
10. **Tests (JVM, `testDebugUnitTest`).** DTO decode (device/stream/subscription), `{ stream, ref }`
|
||||
tickle parse (incl. a malformed body → dropped), the stream→deep-link map, the "personal greyed
|
||||
until linked" gate, and the register/unregister lifecycle over a fake `SessionManager` + repository
|
||||
(parity with M3's session tests).
|
||||
|
||||
**Cross-repo dependency to confirm before/at implementation** (a Part-1 §13 open item): the shard's
|
||||
finalized **ntfy reverse-proxy hostname** must be in `NTFY_ALLOWED_ORIGINS`, because the distributor
|
||||
hands the app an endpoint on *that* origin and the backend rejects a register whose origin isn't
|
||||
allow-listed. This is deployment config, not code, but Part 2 can't be end-to-end tested until it's
|
||||
pinned. No `website`/`link`/`servuo-plugins` code change is expected in Part 2.
|
||||
|
||||
Ships as `RunicGateway/Android-app#15`; bumps `versionCode`/`versionName` for a post-v1 release
|
||||
(§10). Like M1–M4 it records itself in the §9 build-progress block on landing.
|
||||
|
||||
### M9 plan — native SSO login (in progress)
|
||||
|
||||
M9 spans `website/` + `android-app/` + `docs/`, so — like M7 — it ships in **two parts**, backend
|
||||
first (the app is a pure consumer of the bridge contract; §4.2, §9 item 10).
|
||||
|
||||
**Part 1 — `website/` backend + `docs/` — ✅ LANDED** (the Mobile SSO Authorization Bridge:
|
||||
`mobile_auth_sessions`/`mobile_auth_codes` tables, `GET /auth/mobile/sso/start`, the `mode:'mobile'`
|
||||
branch in the reused SSO callback + TOTP completion, `POST /auth/mobile/sso/exchange`, the exact-match
|
||||
`MOBILE_AUTH_REDIRECT_URIS` allowlist, and bridge-table cleanup). Canonical ref:
|
||||
`../website/BACKEND_DESIGN.md` → "Mobile SSO Authorization Bridge".
|
||||
|
||||
**Part 2 — the Android app (this milestone).** The native in-app "Sign in with Google / Discord"
|
||||
client. **Additive** — a new auth slice (`core/auth/sso` + a `SsoApi`/`SsoAuthManager` + a login-screen
|
||||
provider list) that feeds the *existing* M3 session machinery; it adds **no** new token-storage or
|
||||
refresh code, and touches no other screen. **No backend work** — every endpoint it calls is merged.
|
||||
|
||||
The Part-1 contract the app codes against (verified against the merged `website` source):
|
||||
- `GET /auth/providers` → `[{ id, name, icon, loginUrl, priority }]` (public discovery, no secrets).
|
||||
`icon` ∈ `google|discord|oidc|oauth2`. Render the provider buttons from this — don't hardcode.
|
||||
- `GET /auth/mobile/sso/start?provider&code_challenge&state&redirect_uri` — **opened in a Custom Tab**
|
||||
(not an XHR): it 302s through the IdP and finally deep-links back to `redirect_uri`. `redirect_uri`
|
||||
must be an **exact** allowlist entry — the app always sends the one fixed callback
|
||||
`runicgateway://auth/callback`.
|
||||
- The callback deep link carries **either** `?code=<one-time>&state=<echoed>` (success) **or**
|
||||
`?error=<reason>&state=<echoed>` (`invalid_provider`/`provider_unavailable`/`server_error`, or an
|
||||
IdP/link refusal) — **never a token**.
|
||||
- `POST /auth/mobile/sso/exchange` `{ code, code_verifier }` → the **same** `{ accessToken,
|
||||
refreshToken, expiresIn, user }` pair as `/auth/mobile/login`; `401` on an unknown/expired/used code
|
||||
or a PKCE-verifier mismatch.
|
||||
|
||||
Work items:
|
||||
|
||||
1. **PKCE + state (Layer B, app↔website).** A pure-JVM `Pkce` helper (unit-testable, no Android
|
||||
framework types): `code_verifier` = 32 random bytes base64url (RFC 7636 S256), `code_challenge` =
|
||||
base64url(SHA-256(verifier)), plus a random `state`. `java.util.Base64` URL encoder without padding
|
||||
+ `MessageDigest` — matches the backend's `crypto.createHash('sha256')…base64url` exactly.
|
||||
2. **`SsoAuthManager` (Singleton) — the flow orchestrator.** Holds the **pending** `{state, verifier}`
|
||||
in memory (lost on process death → the exchange fails closed and the user retries; acceptable and
|
||||
safe, documented). `buildStartUrl(provider)` mints PKCE+state, stashes pending, and builds the
|
||||
absolute `/start` URL off `BaseUrlHolder` for the Custom Tab. `isCallback(uri)` matches our scheme;
|
||||
`complete(uri)` verifies `state` (CSRF), maps an `error`, exchanges the `code` with the stashed
|
||||
`verifier`, and on success drives `SessionManager.onSignedIn` — the *same* entry the password login
|
||||
uses, so push registration (`PushManager` observes the session) and the menu react identically. It
|
||||
exposes an `outcome: StateFlow` (Idle/Success/Failed(reason)) the login screen consumes, robust to a
|
||||
ViewModel/activity recreation while the Custom Tab is foreground.
|
||||
3. **`SsoApi` + DTOs.** `GET api/v1/auth/providers` → `List<SsoProviderDto>`; `POST
|
||||
api/v1/auth/mobile/sso/exchange` tagged `Http.NO_SESSION_HEADER` (no bearer; a credential-style
|
||||
`401` must not be read as an expired session or trip the refresh `Authenticator`) → the reused
|
||||
`MobileTokenResponse`. Lenient Json (additive fields safe, §8).
|
||||
4. **Deep link.** Register the `runicgateway://auth/callback` intent-filter on `MainActivity`
|
||||
(`VIEW` + `DEFAULT` + `BROWSABLE`, `scheme/host/path` from one shared constant) and set
|
||||
`launchMode="singleTop"` so the returning Custom Tab reuses the running task; `onCreate`/`onNewIntent`
|
||||
route a matching `ACTION_VIEW` intent to `SsoAuthManager.complete` on `lifecycleScope`. Custom scheme
|
||||
only for now — App Links deferred (`APP_LINKS.md`).
|
||||
5. **Login screen.** Replace the single "SSO on the website" hand-off with a native provider list from
|
||||
`GET /auth/providers`: one button per provider (Google/Discord/OIDC glyph from `icon`), each opening
|
||||
its `/start` URL in a Custom Tab via the existing `WebHandoff`. The `LoginViewModel` collects
|
||||
`SsoAuthManager.outcome` → a success pops back like a password sign-in; a failure surfaces a friendly
|
||||
inline error (reusing the existing `LoginError` channel + a new SSO string). Falls back to the
|
||||
website login hand-off when discovery returns no providers or the base URL is unset.
|
||||
6. **Tests (JVM, `testDebugUnitTest`).** `Pkce` (verifier charset/length, challenge = base64url-SHA-256
|
||||
of a known vector, no padding), start-URL building (encoded params, fixed `redirect_uri`), and
|
||||
`SsoAuthManager.complete` over a fake `SsoApi` + `SessionManager`: success signs in; a mismatched or
|
||||
missing `state` fails without exchanging; an `error=` callback maps to the right reason; a `401`
|
||||
exchange maps to expired-code; a missing pending (process death) fails closed.
|
||||
|
||||
Ships as a `RunicGateway/Android-app` PR; bumps `versionCode`/`versionName` for a post-v1 release
|
||||
(§10) and records itself in the §9 build-progress block on landing. No `website`/`link`/`servuo-plugins`
|
||||
code change is expected in Part 2.
|
||||
**The functional build (M0–M4), design pass (M5), and release mechanics (M6) are complete. The first
|
||||
signed release now cuts automatically on the next release-worthy merge to `main` — once the signing +
|
||||
`REGISTRY_*` secrets are set, `main` allows the CI account to push, and the on-device QA pass is done.
|
||||
M7 push notifications are what remain.**
|
||||
|
||||
**Prerequisite progress (§8):** all v1 prerequisites are **done** (2026-07-19) — ✅ password reset
|
||||
(item 2; website#75 + docs#8), ✅ role-agnostic `/auth/me/*` self surface (item 1; website#76 + docs#10),
|
||||
@@ -422,29 +178,21 @@ console. It is a **read + self-service** app, not an operator tool.
|
||||
- **Access-level menu**: one shared navigation that reveals items based on the signed-in user's role.
|
||||
- **Opt-in push notifications** (post-v1; architected for from the start): per-stream subscriptions the
|
||||
user chooses — nothing is pushed unless subscribed. See §11.
|
||||
- **Staff operations subset** (post-v1, **M10** — decided 2026-07-21): a *defined* slice of the admin
|
||||
surface, native, for `admin`/`moderator` staff. In scope: **moderation actions** (kick / ban / unban /
|
||||
broadcast), the **support (help-page) queue** (reply / close / resolve), the **dashboard summary +
|
||||
site-mode** toggle, and **content management** (news posts — list / create / edit / publish / delete —
|
||||
plus wiki category & tag management). These consume the existing `/api/v1/admin/**` routes (which
|
||||
already accept bearer auth and re-check role every request); no backend routes are added. See §6.4 + §10.
|
||||
|
||||
### Explicitly OUT of scope (never in the app, for any role)
|
||||
The exclusions are now a *narrow* list (M10 brought the operational admin subset in-scope, above). The
|
||||
app still never ships:
|
||||
- The **hero editor** and CMS **block/page authoring** (the visual page builder). *(News-post and
|
||||
wiki category/tag management ARE in scope; the excluded piece is the CMS block editor / hero builder.)*
|
||||
- **Discord bot** configuration (and anything under the unpublished `/internal/**` port — it returns the
|
||||
decrypted bot token and must never be reachable from a client).
|
||||
- **uo-link / sidecar administration** — the shard sidecar base-URL/token config (`uoLinkConfig`). (The
|
||||
app performs *shard operations* like kick/ban/broadcast against the live shard, but never configures
|
||||
the sidecar connection itself.)
|
||||
- **OAuth / SSO provider setup** — creating/editing identity providers and their client secrets.
|
||||
- The **hero editor** and any CMS authoring/block editing.
|
||||
- The **admin / auth-management console** — user management, invites issuance, SSO provider config,
|
||||
moderation console, email config, bot-activity/ban console. (Players still *log in*; what's
|
||||
excluded is the management surface, not authentication itself.)
|
||||
- The **Discord bot** management (and anything under the unpublished `/internal/**` port — it
|
||||
returns the decrypted bot token and must never be reachable from a client).
|
||||
- **Shard / uo-link administration** — sidecar base-URL/token config (`uoLinkConfig`), shard ops,
|
||||
the staff shard-user console. (The app shows *public* shard widgets and a player's *own* game
|
||||
data; it does not manage the sidecar.)
|
||||
|
||||
> The app calls `/api/v1/public/**`, `/api/v1/auth/**` (incl. the role-agnostic self surface
|
||||
> `/auth/me/*`, §6.4), `/api/v1/player/**`, and — for staff, M10 — the **defined `/api/v1/admin/**`
|
||||
> operations listed above. It never touches `/api/v1/internal/**`, nor the four excluded admin surfaces
|
||||
> (hero/CMS block editor, Discord-bot config, uo-link config, OAuth-provider setup).
|
||||
> The excluded surfaces all live under `/api/v1/admin/**` and `/api/v1/internal/**`. The app only
|
||||
> ever calls `/api/v1/public/**`, `/api/v1/auth/**` (incl. the new role-agnostic self surface
|
||||
> `/auth/me/*`, §6.4), and `/api/v1/player/**` — it never references `/admin`.
|
||||
|
||||
---
|
||||
|
||||
@@ -502,13 +250,7 @@ compiled in.
|
||||
`/public/settings` for branding: name/colors/logo). Only on a successful, well-formed response is
|
||||
the URL persisted to DataStore and the app allowed to initialize its main UI.
|
||||
- Accept `https://host[/base]`; normalize/trim; require HTTPS in release builds (allow HTTP only in
|
||||
debug for local dev against `127.0.0.1:3000`). This app-layer rule (`ServerUrl`,
|
||||
`allowInsecureHttp = BuildConfig.DEBUG`) is backed at the platform socket layer by an explicit
|
||||
**network security config** (`res/xml/network_security_config.xml`, wired via
|
||||
`application android:networkSecurityConfig`): the main/release config permits **no** cleartext,
|
||||
and a debug-only override (`app/src/debug/res/xml/`) re-permits cleartext to loopback
|
||||
(`127.0.0.1`/`localhost`) only. Being explicit also stops a merged library manifest from
|
||||
re-enabling cleartext and clears the `usesCleartextTraffic`-implicitly-enabled scanner finding.
|
||||
debug for local dev against `127.0.0.1:3000`).
|
||||
- Failure states: unreachable, non-2xx, not-a-Runic-Gateway-site (missing expected `/public/status`
|
||||
shape), TLS error — each gets a clear retry message. Nothing else in the app runs until this
|
||||
succeeds.
|
||||
@@ -546,39 +288,6 @@ Uses the existing **mobile bearer** surface, no backend changes:
|
||||
`401` from a bearer call, with a mutex so concurrent 401s trigger only one refresh.
|
||||
- `POST /auth/mobile/logout` `{ refreshToken?, all? }` (requires bearer) — revoke this session or all
|
||||
sessions. Called on user logout and on "sign out everywhere."
|
||||
|
||||
#### 4.1.1 Trusted devices & recovery codes — **implemented** (app PR `feature/trusted-devices-mfa`)
|
||||
The backend trusted-device + recovery-code feature (canonical ref: `../website/TRUSTED_DEVICES_MFA.md`)
|
||||
is additive on the mobile surface. As built:
|
||||
- **Login extras:** `POST /auth/mobile/login` accepts `recoveryCode` (a single-use alternative to
|
||||
`code`), `trustDevice: true`, `device_name`, and an `X-Trust-Token` header. On the `401 { totpRequired }`
|
||||
screen the app offers a "use a recovery code instead" toggle and a **"Trust this device"** checkbox.
|
||||
When `trustDevice` is accepted, the response carries `trustToken` → stored in a **dedicated
|
||||
EncryptedSharedPreferences file** (`runic_trust`, AES-256-GCM; never plain prefs/logs) **scoped to the
|
||||
username** it was minted for, and replayed as `X-Trust-Token` on that account's future logins to skip
|
||||
the TOTP prompt.
|
||||
- **Login-time cap:** a `{ trustLimitReached, devices }` response means login **succeeded** but the
|
||||
device was not remembered (session is already issued on native, unlike the web cookie step). Rather
|
||||
than a blocking login-time modal, this is surfaced + resolved on the **Trusted Devices** account
|
||||
screen, whose `POST /auth/me/trusted-devices` gives the exact revoke-one-then-retry flow. (Deliberate
|
||||
deviation from the earlier "prompt at login" sketch — the mobile login is past the point a modal would
|
||||
gate.)
|
||||
- **Self-service (account screens):** a **Security** section on the account screen links to two
|
||||
dedicated screens. **Trusted Devices** — `GET /auth/me/trusted-devices` (list), `DELETE …/:id`
|
||||
(revoke one), `DELETE …/trusted-devices` (untrust all), and `POST /auth/me/trusted-devices` to trust
|
||||
the current device (stores the returned `{ trustToken }`). **Recovery Codes** — enabling TOTP returns
|
||||
the one-time `recoveryCodes` (shown once on the account screen with copy/share); `GET
|
||||
…/recovery-codes/status` for the remaining count; `POST …/recovery-codes/generate` (password step-up)
|
||||
to regenerate, shown once.
|
||||
- **Invalidation — trust token deliberately survives logout.** The trust token is the native analogue
|
||||
of the web `rg_trust` cookie, which per the canonical design *survives logout so the next login skips
|
||||
2FA*. On mobile the token is only ever consulted at a **fresh** login — i.e. exactly after a logout or
|
||||
a dead-refresh sign-out — so clearing it there would make the feature a no-op. It is therefore kept in
|
||||
its own store, **untouched by session teardown** (`SessionManager.onSignedOut`), and cleared only on a
|
||||
**Settings → Server switch** (bound to the old host), an **untrust-all**, or server-side revocation
|
||||
(password change/reset, TOTP disable — which makes any surviving token inert; the next login just
|
||||
prompts for the code). This supersedes the earlier handoff note that said to clear it on logout.
|
||||
|
||||
### 4.2 Website-handled flows: registration, invite, forgot-password, SSO
|
||||
These are **not** rebuilt in the app. The app links out to the website's own pages/API and the user
|
||||
completes them in a Custom Tab, then returns and signs in natively (§4.1):
|
||||
@@ -587,19 +296,14 @@ completes them in a Custom Tab, then returns and signs in natively (§4.1):
|
||||
new username + password. (No mobile register/invite endpoints needed.)
|
||||
- **Forgot / reset password** — the app links to the website's reset page (the flow being built in §8
|
||||
before app work). The user resets there, then signs into the app. (No mobile reset endpoint needed.)
|
||||
- **SSO (Google / Discord / OIDC)** — **v1** shipped this as a website browser hand-off: an SSO user
|
||||
links their identity and sets a password on the website, then uses password login in the app.
|
||||
`GET /auth/providers` is shown so the login screen can direct users to "sign in with … on the website."
|
||||
- **Native in-app SSO — now being built (M9), post-v1 additive.** The "possible later enhancement"
|
||||
noted here is now the **Mobile SSO Authorization Bridge**: a Custom-Tab flow that hands a one-time
|
||||
code back to the app's fixed callback (`runicgateway://auth/callback`), exchanged for the *existing*
|
||||
mobile bearer tokens. It **extends** the existing `/auth/sso/*` redirect flow rather than adding a
|
||||
parallel auth path — same PKCE-vs-IdP, same link-only + opt-in-provisioning policy, same TOTP gate,
|
||||
same token shape as `/auth/mobile/login`. The bridge adds a **second** PKCE layer (app ↔ website)
|
||||
and an app-generated `state` (CSRF, verified by the app before exchange). Backend + docs land first
|
||||
(this document's canonical API ref is `../website/BACKEND_DESIGN.md` → "Mobile SSO Authorization
|
||||
Bridge"); the native app client is M9. Custom-scheme callback only for now — App Links are deferred
|
||||
(see [`APP_LINKS.md`](./APP_LINKS.md)).
|
||||
- **SSO (Google / Discord / OIDC)** — SSO stays the website's browser redirect flow (`/auth/sso/*`),
|
||||
**link-only** (no auto-provisioning). For v1 the app does **not** do one-tap in-app SSO; instead an
|
||||
SSO user links their identity and sets a password on the website (the existing "set initial password"
|
||||
path for SSO-provisioned accounts), then uses password login in the app. `GET /auth/sso/providers`
|
||||
can still be shown so the login screen can direct users to "sign in with … on the website."
|
||||
- *Possible later enhancement (out of v1):* true in-app SSO via a Custom-Tab flow that hands a
|
||||
one-time code back to an app link, exchanged for mobile tokens — a small new backend endpoint. Only
|
||||
build it if password-for-SSO-users proves too clunky.
|
||||
|
||||
### 4.3 Session model (all paths)
|
||||
- **Refresh:** `POST /auth/mobile/refresh` `{ refreshToken }` → new pair. **Single-use / rotated:** store
|
||||
@@ -648,10 +352,6 @@ Guidelines:
|
||||
|
||||
## 6. Screen ↔ endpoint map
|
||||
|
||||
> **Path note:** endpoints below are `/api/v1`-relative and stay that way. The website's router
|
||||
> refactor preserves every URL, and the `/api/mobile` facade that would have renamed them is deferred —
|
||||
> see [`../website/API_V2_PLAN.md`](../website/API_V2_PLAN.md) § Deferred: the `/api/mobile` facade.
|
||||
|
||||
### 6.1 Public content
|
||||
- **Home/Status** — `GET /public/status`, `GET /public/settings` (branding + maintenance banner).
|
||||
- **News hub** — `GET /public/posts/:category` (`news | five-on-friday | newsletter | screenshots`),
|
||||
@@ -683,18 +383,11 @@ Guidelines:
|
||||
Player self-service is under `/player/account/*` (gated to `role='player'`) and staff use the *same*
|
||||
handlers under `/admin/account/*`. Rather than have the app branch by role (and touch `/admin`), we
|
||||
**add a role-agnostic self surface under `/auth/**`** — the canonical "me" endpoints for every role.
|
||||
The app calls these regardless of role. This is an **additive v1**
|
||||
The app calls these regardless of role, and never references `/admin`. This is an **additive v1**
|
||||
change (see §8): the existing `/player/account/*` and `/admin/account/*` routes stay for web
|
||||
back-compat; `/auth/me/*` reuses the same `account.controller` handlers behind `requireAuth` (any
|
||||
authenticated role), so there's no logic duplication.
|
||||
|
||||
> **M10 update (2026-07-21):** *self-service* stays role-agnostic under `/auth/me/*` as above. Separately,
|
||||
> the **operational** admin subset (§1, §10 — moderation, support queue, dashboard/site-mode, content)
|
||||
> *does* call `/api/v1/admin/**` directly, gated to staff by a new `STAFF`/`ADMIN` menu access level.
|
||||
> The earlier "the app never references `/admin`" rule is superseded for these defined staff operations
|
||||
> only; `validateSession` accepts the mobile bearer on those routes and `requireRole` re-checks every
|
||||
> request, so a demoted user loses the surface immediately (the app treats any `403` as authoritative).
|
||||
|
||||
---
|
||||
|
||||
## 7. Degradation & offline
|
||||
@@ -764,12 +457,6 @@ maintenance cost. Reserve v2 for a real breaking re-shape if one ever arises.
|
||||
3. **Push notifications** — see §11. Additive v1 endpoints under `/auth/me/devices*` and
|
||||
`/auth/me/notifications*`, plus a **self-hosted `ntfy` service added to `website/docker-compose.yml`**
|
||||
with fully declarative, zero-interaction config. Not required for the first release (M7, not M1–M6).
|
||||
✅ **Backend + docs LANDED (2026-07-20, RunicGateway/website#78 merged (+ docs#20)).** The
|
||||
contract Part 1 is built: the two tables, the stream catalog + `PUBLIC_KINDS`-gated event mapping,
|
||||
the content-free-tickle fan-out (`utils/pushDispatch`, SSRF-guarded endpoints, owner-keyed personal
|
||||
streams), the six `/auth/me/*` routes (Swagger regenerated), and the declarative `ntfy` compose
|
||||
service (247 server tests green). The app (Part 2, §9 M7) consumes this next — see the "M7 plan"
|
||||
block for the detailed Part 2 plan.
|
||||
4. **Version/health surfacing.**
|
||||
✅ **DONE (2026-07-19, RunicGateway/website#77 (+ this docs PR)).** A dependency-free
|
||||
`config/version.js` (`{ service:'runic-gateway', api:'v1', server:<pkg> }`) is surfaced on
|
||||
@@ -826,72 +513,15 @@ push, and Play (M6–M8) follow the designed app.
|
||||
7. **M6 — Polish & release mechanics**: settings (server switch = hard reset, done M3),
|
||||
version-mismatch guard, release build hardening (HTTPS-only, no token logging, R8 minify + resource
|
||||
shrink, release signing). No offline cache in v1 (§7). **Ships v1 as a signed APK attached to a Gitea
|
||||
release** via `release.yml` on a `v*` tag (see §10). Biometric app-lock **descoped** (below).
|
||||
release** via `release.yml`'s conventional-commit engine on merge to `main` (see §10, §12).
|
||||
Biometric app-lock **descoped** (below).
|
||||
**Landed** 2026-07-20 (`RunicGateway/Android-app#11`).
|
||||
8. **M7 — Push notifications** (post-v1): add the self-hosted `ntfy` service to
|
||||
`website/docker-compose.yml` (declarative, zero-interaction config), UnifiedPush integration in the
|
||||
app, device registration, the subscriptions UI, and the content-free-tickle backend fan-out (see
|
||||
§11). The app is built with room for this from M0 but it does not gate the first release.
|
||||
✅ **Both parts landed** 2026-07-20 — Part 1 backend (`RunicGateway/website#78` merged + docs#20),
|
||||
Part 2 app (`RunicGateway/Android-app#15`) + a small `push.ntfyUrl` settings addition
|
||||
(`RunicGateway/website#79`). The app embeds its own ntfy distributor (a foreground-service SSE
|
||||
connection, no second app, no Google Play Services, no UnifiedPush library); see the "M7 plan"
|
||||
Part 2 block for the recorded transport + backend-field decisions. Push delivers once the shard sets
|
||||
its `NTFY_*` deploy config (§13).
|
||||
9. **M8 — Google Play**: Play Console listing, signing/upload key, and (optionally) an FCM build flavor
|
||||
— after the direct-APK release is stable.
|
||||
10. **M9 — Native SSO login** (post-v1, additive; independent of M8): in-app "Sign in with Google /
|
||||
Discord" via the **Mobile SSO Authorization Bridge** (§4.2). **Backend-first**, mirroring M7's
|
||||
split:
|
||||
- **Part 1 — backend + docs (in progress):** `mobile_auth_sessions` + `mobile_auth_codes` bridge
|
||||
tables; `GET /auth/mobile/sso/start` (seeds a bridge session, reuses the existing SSO redirect
|
||||
tagged `mode:'mobile'`); a mobile branch in the SSO callback + TOTP-completion that mints a
|
||||
single-use, hashed, PKCE-bound authorization code and redirects to the fixed app callback instead
|
||||
of setting a cookie; `POST /auth/mobile/sso/exchange` (code + PKCE verifier → the existing mobile
|
||||
bearer token pair); an exact-match redirect-URI allowlist; boot-time + opportunistic cleanup of
|
||||
the bridge tables. Reuses `GET /auth/providers` for discovery and `POST /auth/mobile/{refresh,
|
||||
logout}` unchanged. See `../website/BACKEND_DESIGN.md`.
|
||||
- **Part 2 — app client:** register the `runicgateway://auth/callback` intent-filter; generate
|
||||
`code_verifier`/`code_challenge` + `state`; open the Custom Tab at `/auth/mobile/sso/start`;
|
||||
verify `state` on the callback; `POST …/exchange`; store the returned pair in the existing
|
||||
`TokenStore` (M3). No new token-storage or refresh code — it feeds the M3 session machinery.
|
||||
- **Follow-up — App Links (opt-in hardening on top of Part 2):** a website `GET
|
||||
/.well-known/assetlinks.json` route behind the `mobile_app_links_enabled` admin toggle, a
|
||||
self-origin HTTPS entry added to the redirect-URI allowlist when enabled, and an app-side
|
||||
`autoVerify` intent-filter for `https://<host>/mobile/callback` driven by a **build-time**
|
||||
`appLinkHost` (a single multi-tenant APK cannot autoVerify open-ended shard domains, so the
|
||||
generic build stays custom-scheme; white-label/first-party builds bake one host). The custom
|
||||
scheme remains the permanent fallback on every build. Full spec + rollout in
|
||||
[`APP_LINKS.md`](./APP_LINKS.md).
|
||||
11. **M10 — Native SSO fixes + staff operations** (post-v1; decided 2026-07-21). Two threads found
|
||||
during on-device QA:
|
||||
- **SSO discovery + reachability fixes (app-only, done):** the native login screen only rendered
|
||||
provider buttons when `GET /auth/providers` was non-empty and otherwise fell back to the desktop
|
||||
website login (which can't deep-link a mobile session back → it hung). `AuthRepository.ssoProviders()`
|
||||
now returns `Available` / `None` / `Unavailable` (retry once); the login screen shows native
|
||||
buttons / a loading hint / a retry, and the website-login fallback is removed. The pending PKCE
|
||||
`{state,verifier}` is persisted (encrypted `PendingSsoStore`) so the exchange survives Custom-Tab
|
||||
process death. The nav drawer is now `verticalScroll`-wrapped so a signed-in session's longer menu
|
||||
(which includes **Notifications**) can't clip on short screens. Dev testing uses a **stub OAuth IdP**
|
||||
(`website/scripts/dev/`), since dev configures no real provider. Verified end-to-end on an emulator.
|
||||
- **Staff operations (§1, §6.4):** a new `STAFF`/`ADMIN` menu access level reveals a staff section
|
||||
for `admin`/`moderator`, with native screens over the existing `/api/v1/admin/**`: **moderation**
|
||||
(`POST /admin/shard/{kick,ban,unban,broadcast}`), **support queue** (`GET /admin/shard/pages`,
|
||||
`POST /admin/shard/pages/:id/{respond,close}`), **dashboard + site-mode** (`GET /admin/dashboard`,
|
||||
`PUT /admin/site-mode`), and **content** (news posts under `/admin/posts*`, wiki categories/tags
|
||||
under `/admin/wiki/*`). No backend routes added (they already accept bearer + re-check role);
|
||||
shard-write actions degrade gracefully when the sidecar is offline. Excluded: hero/CMS block
|
||||
editor, Discord-bot config, uo-link config, OAuth-provider setup.
|
||||
|
||||
### Deferred (not a milestone)
|
||||
|
||||
- **`/api/mobile` facade migration + app-version floor** — briefly planned as M11 (2026-07-22), now
|
||||
**deferred with no app work scheduled**. The website's router refactor is being done in place with
|
||||
every URL byte-identical and `/api/v1` is not being retired, so the app's ~70 hardcoded `api/v1/…`
|
||||
endpoints, its SSE path, and its SSO URLs keep working untouched. If the mobile contract ever needs
|
||||
to diverge from web, the migration comes back — starting from a one-line alias mount on the server,
|
||||
not a hand-written delegate layer. Reasoning and revival triggers:
|
||||
[`../website/API_V2_PLAN.md`](../website/API_V2_PLAN.md) § Deferred: the `/api/mobile` facade.
|
||||
|
||||
---
|
||||
|
||||
@@ -914,10 +544,6 @@ first release. Users **opt in per stream**: nothing is pushed unless subscribed.
|
||||
- **Primary: UnifiedPush, delivered by a self-hosted `ntfy` service added to the website's
|
||||
`docker-compose.yml`.** FOSS, no Google Play Services dependency, works for the sideloaded APK on any
|
||||
device, and keeps delivery under the org's own infrastructure — consistent with the self-hosted ethos.
|
||||
- **The app embeds its own distributor — no second app (decided; see M7 Part 2 work item 1).** ntfy is
|
||||
purely the relay *server*; the Runic Gateway app receives notifications itself via an in-app embedded
|
||||
UnifiedPush distributor (a foreground-service persistent connection to the shard's ntfy). The user
|
||||
installs one APK — never a separate distributor app — and no Google Play Services is involved.
|
||||
- **FCM stays optional and Play-only.** If/when a Play build wants it, add FCM as a **build flavor**;
|
||||
the direct-APK flavor stays Google-free. The backend fan-out is **transport-agnostic** and dispatches
|
||||
to whatever endpoint a device registered, so adding FCM later touches no core logic.
|
||||
@@ -928,11 +554,8 @@ first release. Users **opt in per stream**: nothing is pushed unless subscribed.
|
||||
path/tag at implementation.
|
||||
- **All config is declarative** — a committed `ntfy` config file and/or `NTFY_*` env vars baked into
|
||||
compose. No `docker exec`, no interactive `ntfy user add`, no post-deploy manual steps. Bringing the
|
||||
stack up provisions a working push relay. Reachable via the existing reverse proxy on its own
|
||||
hostname/path — the container publishes `:80` on a host port (`NTFY_HOST_PORT`, default `2586`)
|
||||
because the proxy runs *outside* the compose network and can only reach a service through a
|
||||
published host port (the same reason `app` publishes `3000`). Both devices and the backend publisher
|
||||
reach ntfy on that public origin.
|
||||
stack up provisions a working push relay. Reachable to devices via the existing reverse proxy on its
|
||||
own hostname/path; internal-only for the backend publisher.
|
||||
- **No per-user ntfy accounts to administer.** The security model (below) removes the need for ntfy ACL
|
||||
provisioning, which is exactly what keeps setup interaction-free. ntfy topics are the random,
|
||||
unguessable endpoints UnifiedPush hands out; the backend treats ntfy as an **untrusted relay**.
|
||||
@@ -980,17 +603,6 @@ The ntfy relay is treated as **untrusted infrastructure**, and the design makes
|
||||
write to `/auth/me/notifications/subscriptions`. Tapping a notification deep-links to the relevant
|
||||
screen (§ open item below).
|
||||
|
||||
> **Gotcha — the PUT body must always carry `streams`, even when empty.** The backend validator
|
||||
> requires the field (`body('streams').isArray()`), so an empty set has to be sent as
|
||||
> `{"streams":[]}`, never `{}`. kotlinx.serialization omits a property equal to its default
|
||||
> (`encodeDefaults=false`), so a DTO field like `streams: List<String> = emptyList()` gets *dropped*
|
||||
> from the body when the set is empty — the app then sends `{}` and the server rejects it `400`.
|
||||
> Symptom: clearing your **last** subscription fails with "could not save" and the toggle sticks
|
||||
> (any non-empty set still includes the field, so only the final toggle-off breaks). Fix: give the
|
||||
> request DTO field **no default** so kotlinx always encodes it (Android-app
|
||||
> `fix/notifications-empty-subscriptions`). The same trap applies to any "replace the full set"
|
||||
> `PUT`/`POST` whose empty value equals a DTO default — prefer no default on required request fields.
|
||||
|
||||
## 12. Build & CI (Gitea Actions)
|
||||
|
||||
Builds run on the org's existing self-hosted runners (`runs-on: ubuntu-latest`, same label the other
|
||||
@@ -1003,38 +615,16 @@ repos use), on a bare `ubuntu:latest` container.
|
||||
frequent: run the job under a prebuilt Android-SDK `container:` image so nothing installs per-run.)
|
||||
- **PR gate** (`.gitea/workflows/pr-checks.yml`, on PR → `main`): `./gradlew lint test assembleDebug`.
|
||||
Debug builds are auto-signed, so the gate needs no secrets. Mirrors `website/`'s pre-merge gate.
|
||||
- **Release** (`.gitea/workflows/release.yml`, M6+): build a **signed release APK** and attach it to a
|
||||
Gitea release (mirrors `link/`'s release job). The **keystore is a base64 Gitea Actions secret**
|
||||
decoded in CI; store/key passwords are secrets. The keystore never lives in the repo. Keep the
|
||||
signing identity stable from the first release (Play later requires consistency).
|
||||
- Semantic `versionName` + monotonic `versionCode`; tag releases.
|
||||
|
||||
### 12.1 Code quality — SonarQube (`Runic-Gateway-Android-app`)
|
||||
|
||||
Analysis runs post-merge and non-blocking (`.gitea/workflows/sonarqube.yml`); the gate is
|
||||
informational. The project is clean (0 bugs / 0 vulns / 0 hotspots, Maintainability **A**); the only
|
||||
gate failure is **coverage = 0% on new code**, which is a *reporting* gap, not a testing gap — the
|
||||
25-file JVM unit suite exists, but the source-only Sonar scan never received a JaCoCo report.
|
||||
|
||||
> Once the wiring below landed (`Android-app#26`), real coverage measured **16.4%** on new code —
|
||||
> still under the 50% gate. The plan to raise it (exclude non-unit-testable framework/UI code + test
|
||||
> ViewModels/DTOs/repositories) lives in [`COVERAGE_PLAN.md`](./COVERAGE_PLAN.md).
|
||||
|
||||
**Coverage wiring (the fix):**
|
||||
- Apply the `jacoco` plugin in `app/build.gradle.kts` + a `jacocoTestReport` task fed by
|
||||
`testDebugUnitTest`, emitting XML. Exclude generated/DI/Compose scaffolding
|
||||
(`**/*_Hilt*`, `**/*_Factory*`, `**/di/**`, `**/*ComposableSingletons*`, `R`/`BuildConfig`).
|
||||
- `sonar-project.properties`: set `sonar.coverage.jacoco.xmlReportPaths` to the report, and
|
||||
`sonar.coverage.exclusions` for pure-`@Composable` UI (JVM unit tests can't execute composable
|
||||
bodies without Robolectric, so counting those lines would unfairly sink new-code coverage).
|
||||
- `sonarqube.yml` must now run a real Gradle build before the scan (JDK 17 + Android SDK, mirroring
|
||||
`pr-checks.yml`): `./gradlew testDebugUnitTest jacocoTestReport` → then the scan step.
|
||||
|
||||
**Issue triage (2026-07-22):** of 15 code smells, **3 fixed in code** — remove an unused import
|
||||
(`AdminContentScreen.kt`), remove an unused `page` param (`AdminSupportScreen.RespondDialog`), and
|
||||
decompose `LoginViewModel` (cognitive complexity 20). The remaining **12 marked *Won't Fix*** via the
|
||||
Sonar API with rationale: 5 snake_case DTO fields (intentionally mirror the JSON wire contract) and 7
|
||||
Compose/nav cognitive-complexity + hoisted-callback param-count smells (idiomatic for Jetpack Compose).
|
||||
- **Release** (`.gitea/workflows/release.yml`, M6): mirrors `link/`'s release engine — on every push to
|
||||
`main` it computes the next version from conventional commits since the last `v*` tag, generates a
|
||||
changelog, bumps `build.gradle.kts`, builds a **signed release APK**, commits the bump `[skip ci]`,
|
||||
tags `vX.Y.Z`, and creates the Gitea release with the notes + APK + `SHA256SUMS`. The **keystore is a
|
||||
base64 Gitea Actions secret** decoded in CI (`ANDROID_KEYSTORE_BASE64`); store/key passwords + alias
|
||||
are secrets too. The keystore never lives in the repo. `REGISTRY_USER`/`REGISTRY_TOKEN`
|
||||
(`write:repository`) push the bump + create the release, so `main` must allow that account to push.
|
||||
Keep the signing identity stable from the first release (Play later requires consistency).
|
||||
- Semantic `versionName` (bumped by the engine) + derived monotonic `versionCode`
|
||||
(`major*10000+minor*100+patch`); the engine tags each release.
|
||||
|
||||
## 13. Open questions (revisit as we go)
|
||||
|
||||
@@ -1043,8 +633,7 @@ password+TOTP only, with registration/invite/reset/SSO **handled by the website*
|
||||
reset built on backend + web first**, before app work (§8); minSdk 29, compile/target 35 (§2); no
|
||||
telemetry in v1 (§2); strings externalized from day one, English-only bundled (§2); **text-only** game
|
||||
data in v1, pretty paperdoll is future (§6.3); **no offline cache in v1** (§7); push via self-hosted
|
||||
ntfy / UnifiedPush (§11) with the **distributor embedded in the app — no second app to install**
|
||||
(M7 Part 2 work item 1); **biometric app-lock descoped from v1** (tokens already encrypted at rest, so
|
||||
ntfy / UnifiedPush (§11); **biometric app-lock descoped from v1** (tokens already encrypted at rest, so
|
||||
it is a UX convenience, not a v1 requirement — deferred at M3, descoped at M6; revisit only if requested).
|
||||
|
||||
**Still open:**
|
||||
@@ -1052,22 +641,10 @@ it is a UX convenience, not a v1 requirement — deferred at M3, descoped at M6;
|
||||
`runicgateway.app` domain (needed for a verified app-link host and a matching package namespace). Also
|
||||
the fixed launcher name (baked at build even though in-app branding is per-shard — one APK, any shard).
|
||||
Since SSO/invite/reset are website-handled, the app mostly *opens* website URLs rather than needing its
|
||||
own verified app links. **App Links resolved (M9 follow-up):** the SSO callback is the one place a
|
||||
verified deep-link-back helps; the server side (`assetlinks.json` + toggle) ships for any shard, but
|
||||
the app-side `autoVerify` needs a **literal build-time host**, so it is a white-label/first-party build
|
||||
opt-in (`-PappLinkHost=<host>`) — the generic multi-tenant build stays custom-scheme. A canonical
|
||||
`runicgateway.app` relay host, if secured, would let the generic build autoVerify one central domain.
|
||||
See [`APP_LINKS.md`](./APP_LINKS.md).
|
||||
- ntfy: exact upstream image + pinned tag (Part-1 landed the compose service — confirm the tag), and
|
||||
its reverse-proxy hostname/path. The hostname must land in `NTFY_ALLOWED_ORIGINS` before M7 Part 2 is
|
||||
end-to-end testable (the app registers an endpoint on that origin; the SSRF guard rejects others). The
|
||||
reverse proxy forwards that hostname to the ntfy container's published host port (`NTFY_HOST_PORT`,
|
||||
default `2586`) — the container publishes `:80` because the proxy runs outside the compose network.
|
||||
No backend publish token — **decided** (the content-free-tickle design does not require one; optional
|
||||
`NTFY_PUBLISH_TOKEN` is honored if ever set).
|
||||
- FCM flavor: build it for the Play release or ship Play on UnifiedPush too? Decide at M8. (The M7
|
||||
Part 2 `PushTransport` seam keeps this swap cheap.)
|
||||
- Deep-link / share targets for wiki pages and posts (share/open-in-app). *Notification-tap* deep-links
|
||||
are **resolved** for M7 Part 2 (stream→screen map, work item 7).
|
||||
own verified app links — confirm whether any deep-link-back is wanted at all for v1.
|
||||
- ntfy: exact upstream image + pinned tag, its reverse-proxy hostname/path, and whether to add a
|
||||
backend publish token (optional hardening — the content-free-tickle design does not require one).
|
||||
- FCM flavor: build it for the Play release or ship Play on UnifiedPush too? Decide at M8.
|
||||
- Deep-link / share targets for wiki pages, posts, and notification taps.
|
||||
- iOS: none planned (this is the Android-only choice); revisit only if cross-platform is later
|
||||
required (would change §2 — and push, which would then favor a cross-platform transport).
|
||||
|
||||
@@ -1,348 +0,0 @@
|
||||
# Android App — Project Tree
|
||||
|
||||
> **Auto-generated.** This file is maintained by the `sync-project-tree` CI workflow in
|
||||
> the [`RunicGateway/Android-app`](https://gitea.whitlocktech.com/RunicGateway/Android-app) repository, which
|
||||
> opens a pull request here whenever the tracked file layout on `main` changes. Do not edit
|
||||
> by hand — changes will be overwritten by the next sync.
|
||||
|
||||
A snapshot of the tracked files in the repository (build output, dependencies, and other
|
||||
git-ignored paths are excluded).
|
||||
|
||||
```text
|
||||
android-app/
|
||||
├── .gitea/
|
||||
│ ├── scripts/
|
||||
│ │ └── gen_tree.py
|
||||
│ └── workflows/
|
||||
│ ├── pr-checks.yml
|
||||
│ ├── release.yml
|
||||
│ ├── sonarqube.yml
|
||||
│ └── sync-project-tree.yml
|
||||
├── app/
|
||||
│ ├── licenses/
|
||||
│ │ └── Cinzel-OFL.txt
|
||||
│ ├── src/
|
||||
│ │ ├── debug/
|
||||
│ │ │ └── res/
|
||||
│ │ │ └── xml/
|
||||
│ │ │ └── network_security_config.xml
|
||||
│ │ ├── main/
|
||||
│ │ │ ├── java/
|
||||
│ │ │ │ └── com/
|
||||
│ │ │ │ └── runicgateway/
|
||||
│ │ │ │ └── app/
|
||||
│ │ │ │ ├── core/
|
||||
│ │ │ │ │ ├── auth/
|
||||
│ │ │ │ │ │ ├── sso/
|
||||
│ │ │ │ │ │ │ ├── EncryptedPendingSsoStore.kt
|
||||
│ │ │ │ │ │ │ ├── PendingSsoStore.kt
|
||||
│ │ │ │ │ │ │ ├── Pkce.kt
|
||||
│ │ │ │ │ │ │ └── SsoAuthManager.kt
|
||||
│ │ │ │ │ │ ├── DeviceNameProvider.kt
|
||||
│ │ │ │ │ │ ├── EncryptedTokenStore.kt
|
||||
│ │ │ │ │ │ ├── EncryptedTrustTokenStore.kt
|
||||
│ │ │ │ │ │ ├── Session.kt
|
||||
│ │ │ │ │ │ ├── SessionManager.kt
|
||||
│ │ │ │ │ │ ├── TokenStore.kt
|
||||
│ │ │ │ │ │ └── TrustTokenStore.kt
|
||||
│ │ │ │ │ ├── net/
|
||||
│ │ │ │ │ │ ├── AuthInterceptor.kt
|
||||
│ │ │ │ │ │ ├── BaseUrlHolder.kt
|
||||
│ │ │ │ │ │ ├── HostSelectionInterceptor.kt
|
||||
│ │ │ │ │ │ ├── Http.kt
|
||||
│ │ │ │ │ │ ├── ServerUrl.kt
|
||||
│ │ │ │ │ │ ├── ShardStream.kt
|
||||
│ │ │ │ │ │ ├── ShardStreamClient.kt
|
||||
│ │ │ │ │ │ ├── ShardStreamEvent.kt
|
||||
│ │ │ │ │ │ ├── TokenAuthenticator.kt
|
||||
│ │ │ │ │ │ └── UserAgentInterceptor.kt
|
||||
│ │ │ │ │ ├── prefs/
|
||||
│ │ │ │ │ │ └── ServerPreferences.kt
|
||||
│ │ │ │ │ ├── push/
|
||||
│ │ │ │ │ │ ├── NtfyStreamClient.kt
|
||||
│ │ │ │ │ │ ├── NtfyTopic.kt
|
||||
│ │ │ │ │ │ ├── PushManager.kt
|
||||
│ │ │ │ │ │ ├── PushNotifier.kt
|
||||
│ │ │ │ │ │ ├── PushPreferences.kt
|
||||
│ │ │ │ │ │ ├── PushService.kt
|
||||
│ │ │ │ │ │ ├── PushStreams.kt
|
||||
│ │ │ │ │ │ └── PushTickle.kt
|
||||
│ │ │ │ │ ├── result/
|
||||
│ │ │ │ │ │ └── ApiResult.kt
|
||||
│ │ │ │ │ ├── web/
|
||||
│ │ │ │ │ │ ├── WebHandoff.kt
|
||||
│ │ │ │ │ │ └── WebsiteUrls.kt
|
||||
│ │ │ │ │ └── AppConfig.kt
|
||||
│ │ │ │ ├── data/
|
||||
│ │ │ │ │ ├── api/
|
||||
│ │ │ │ │ │ ├── dto/
|
||||
│ │ │ │ │ │ │ ├── AccountDto.kt
|
||||
│ │ │ │ │ │ │ ├── AdminDto.kt
|
||||
│ │ │ │ │ │ │ ├── AuthDto.kt
|
||||
│ │ │ │ │ │ │ ├── ContactDto.kt
|
||||
│ │ │ │ │ │ │ ├── NotificationsDto.kt
|
||||
│ │ │ │ │ │ │ ├── PageDto.kt
|
||||
│ │ │ │ │ │ │ ├── PlayerShardDto.kt
|
||||
│ │ │ │ │ │ │ ├── PostDto.kt
|
||||
│ │ │ │ │ │ │ ├── PublicDto.kt
|
||||
│ │ │ │ │ │ │ ├── ShardDto.kt
|
||||
│ │ │ │ │ │ │ ├── SsoDto.kt
|
||||
│ │ │ │ │ │ │ └── WikiDto.kt
|
||||
│ │ │ │ │ │ ├── AdminApi.kt
|
||||
│ │ │ │ │ │ ├── AuthApi.kt
|
||||
│ │ │ │ │ │ ├── AuthRefreshApi.kt
|
||||
│ │ │ │ │ │ ├── MeApi.kt
|
||||
│ │ │ │ │ │ ├── NotificationsApi.kt
|
||||
│ │ │ │ │ │ ├── PlayerShardApi.kt
|
||||
│ │ │ │ │ │ ├── PublicApi.kt
|
||||
│ │ │ │ │ │ └── SsoApi.kt
|
||||
│ │ │ │ │ └── repository/
|
||||
│ │ │ │ │ ├── AccountRepository.kt
|
||||
│ │ │ │ │ ├── AdminRepository.kt
|
||||
│ │ │ │ │ ├── AuthRepository.kt
|
||||
│ │ │ │ │ ├── ConnectionRepository.kt
|
||||
│ │ │ │ │ ├── ContactRepository.kt
|
||||
│ │ │ │ │ ├── ContentRepository.kt
|
||||
│ │ │ │ │ ├── NotificationsRepository.kt
|
||||
│ │ │ │ │ ├── PlayerShardRepository.kt
|
||||
│ │ │ │ │ ├── SettingsRepository.kt
|
||||
│ │ │ │ │ ├── ShardRepository.kt
|
||||
│ │ │ │ │ └── WikiRepository.kt
|
||||
│ │ │ │ ├── di/
|
||||
│ │ │ │ │ ├── AppModule.kt
|
||||
│ │ │ │ │ ├── NetworkModule.kt
|
||||
│ │ │ │ │ └── StorageModule.kt
|
||||
│ │ │ │ ├── ui/
|
||||
│ │ │ │ │ ├── admin/
|
||||
│ │ │ │ │ │ ├── AdminContentScreen.kt
|
||||
│ │ │ │ │ │ ├── AdminContentViewModel.kt
|
||||
│ │ │ │ │ │ ├── AdminDashboardScreen.kt
|
||||
│ │ │ │ │ │ ├── AdminDashboardViewModel.kt
|
||||
│ │ │ │ │ │ ├── AdminModerationScreen.kt
|
||||
│ │ │ │ │ │ ├── AdminModerationViewModel.kt
|
||||
│ │ │ │ │ │ ├── AdminSupportScreen.kt
|
||||
│ │ │ │ │ │ └── AdminSupportViewModel.kt
|
||||
│ │ │ │ │ ├── auth/
|
||||
│ │ │ │ │ │ ├── AccountScreen.kt
|
||||
│ │ │ │ │ │ ├── AccountViewModel.kt
|
||||
│ │ │ │ │ │ ├── LoginScreen.kt
|
||||
│ │ │ │ │ │ ├── LoginViewModel.kt
|
||||
│ │ │ │ │ │ ├── RecoveryCodesScreen.kt
|
||||
│ │ │ │ │ │ ├── RecoveryCodesViewModel.kt
|
||||
│ │ │ │ │ │ ├── TrustedDevicesScreen.kt
|
||||
│ │ │ │ │ │ └── TrustedDevicesViewModel.kt
|
||||
│ │ │ │ │ ├── components/
|
||||
│ │ │ │ │ │ ├── HtmlText.kt
|
||||
│ │ │ │ │ │ ├── StateViews.kt
|
||||
│ │ │ │ │ │ └── ThemeComponents.kt
|
||||
│ │ │ │ │ ├── connect/
|
||||
│ │ │ │ │ │ ├── ConnectScreen.kt
|
||||
│ │ │ │ │ │ └── ConnectViewModel.kt
|
||||
│ │ │ │ │ ├── contact/
|
||||
│ │ │ │ │ │ ├── ContactScreen.kt
|
||||
│ │ │ │ │ │ └── ContactViewModel.kt
|
||||
│ │ │ │ │ ├── home/
|
||||
│ │ │ │ │ │ ├── HomeScreen.kt
|
||||
│ │ │ │ │ │ └── HomeViewModel.kt
|
||||
│ │ │ │ │ ├── navigation/
|
||||
│ │ │ │ │ │ ├── Menu.kt
|
||||
│ │ │ │ │ │ └── Routes.kt
|
||||
│ │ │ │ │ ├── news/
|
||||
│ │ │ │ │ │ ├── NewsScreen.kt
|
||||
│ │ │ │ │ │ ├── NewsViewModel.kt
|
||||
│ │ │ │ │ │ ├── PostScreen.kt
|
||||
│ │ │ │ │ │ └── PostViewModel.kt
|
||||
│ │ │ │ │ ├── notifications/
|
||||
│ │ │ │ │ │ ├── NotificationsScreen.kt
|
||||
│ │ │ │ │ │ └── NotificationsViewModel.kt
|
||||
│ │ │ │ │ ├── page/
|
||||
│ │ │ │ │ │ ├── BlockRenderer.kt
|
||||
│ │ │ │ │ │ ├── PageScreen.kt
|
||||
│ │ │ │ │ │ └── PageViewModel.kt
|
||||
│ │ │ │ │ ├── player/
|
||||
│ │ │ │ │ │ ├── CharacterSheetScreen.kt
|
||||
│ │ │ │ │ │ ├── CharactersScreen.kt
|
||||
│ │ │ │ │ │ ├── CharactersViewModel.kt
|
||||
│ │ │ │ │ │ ├── CharacterViewModel.kt
|
||||
│ │ │ │ │ │ ├── MyHousesScreen.kt
|
||||
│ │ │ │ │ │ ├── MyHousesViewModel.kt
|
||||
│ │ │ │ │ │ ├── VendorsScreen.kt
|
||||
│ │ │ │ │ │ └── VendorsViewModel.kt
|
||||
│ │ │ │ │ ├── session/
|
||||
│ │ │ │ │ │ └── SessionViewModel.kt
|
||||
│ │ │ │ │ ├── shard/
|
||||
│ │ │ │ │ │ ├── ChampsScreen.kt
|
||||
│ │ │ │ │ │ ├── ChampsViewModel.kt
|
||||
│ │ │ │ │ │ ├── FrameFields.kt
|
||||
│ │ │ │ │ │ ├── GovernorsScreen.kt
|
||||
│ │ │ │ │ │ ├── GovernorsViewModel.kt
|
||||
│ │ │ │ │ │ ├── GuildsScreen.kt
|
||||
│ │ │ │ │ │ ├── GuildsViewModel.kt
|
||||
│ │ │ │ │ │ ├── HousesScreen.kt
|
||||
│ │ │ │ │ │ ├── HousesViewModel.kt
|
||||
│ │ │ │ │ │ ├── LiveBoard.kt
|
||||
│ │ │ │ │ │ ├── ShardComponents.kt
|
||||
│ │ │ │ │ │ ├── ShardEventText.kt
|
||||
│ │ │ │ │ │ ├── ShardScreen.kt
|
||||
│ │ │ │ │ │ └── ShardViewModel.kt
|
||||
│ │ │ │ │ ├── theme/
|
||||
│ │ │ │ │ │ ├── BrandColor.kt
|
||||
│ │ │ │ │ │ ├── Color.kt
|
||||
│ │ │ │ │ │ ├── Font.kt
|
||||
│ │ │ │ │ │ ├── Theme.kt
|
||||
│ │ │ │ │ │ └── Type.kt
|
||||
│ │ │ │ │ ├── wiki/
|
||||
│ │ │ │ │ │ ├── WikiPageScreen.kt
|
||||
│ │ │ │ │ │ ├── WikiPageViewModel.kt
|
||||
│ │ │ │ │ │ ├── WikiScreen.kt
|
||||
│ │ │ │ │ │ └── WikiViewModel.kt
|
||||
│ │ │ │ │ ├── AppViewModel.kt
|
||||
│ │ │ │ │ ├── LocalAssetResolver.kt
|
||||
│ │ │ │ │ ├── RunicApp.kt
|
||||
│ │ │ │ │ └── UiState.kt
|
||||
│ │ │ │ ├── MainActivity.kt
|
||||
│ │ │ │ └── RunicGatewayApp.kt
|
||||
│ │ │ ├── res/
|
||||
│ │ │ │ ├── drawable/
|
||||
│ │ │ │ │ └── ic_launcher_background.xml
|
||||
│ │ │ │ ├── drawable-anydpi/
|
||||
│ │ │ │ │ └── ic_stat_name.xml
|
||||
│ │ │ │ ├── drawable-hdpi/
|
||||
│ │ │ │ │ └── ic_stat_name.png
|
||||
│ │ │ │ ├── drawable-mdpi/
|
||||
│ │ │ │ │ └── ic_stat_name.png
|
||||
│ │ │ │ ├── drawable-xhdpi/
|
||||
│ │ │ │ │ └── ic_stat_name.png
|
||||
│ │ │ │ ├── drawable-xxhdpi/
|
||||
│ │ │ │ │ └── ic_stat_name.png
|
||||
│ │ │ │ ├── font/
|
||||
│ │ │ │ │ └── cinzel_variable.ttf
|
||||
│ │ │ │ ├── mipmap-anydpi-v26/
|
||||
│ │ │ │ │ ├── ic_launcher.xml
|
||||
│ │ │ │ │ └── ic_launcher_round.xml
|
||||
│ │ │ │ ├── mipmap-hdpi/
|
||||
│ │ │ │ │ ├── ic_launcher.webp
|
||||
│ │ │ │ │ ├── ic_launcher_foreground.webp
|
||||
│ │ │ │ │ └── ic_launcher_round.webp
|
||||
│ │ │ │ ├── mipmap-mdpi/
|
||||
│ │ │ │ │ ├── ic_launcher.webp
|
||||
│ │ │ │ │ ├── ic_launcher_foreground.webp
|
||||
│ │ │ │ │ └── ic_launcher_round.webp
|
||||
│ │ │ │ ├── mipmap-xhdpi/
|
||||
│ │ │ │ │ ├── ic_launcher.webp
|
||||
│ │ │ │ │ ├── ic_launcher_foreground.webp
|
||||
│ │ │ │ │ └── ic_launcher_round.webp
|
||||
│ │ │ │ ├── mipmap-xxhdpi/
|
||||
│ │ │ │ │ ├── ic_launcher.webp
|
||||
│ │ │ │ │ ├── ic_launcher_foreground.webp
|
||||
│ │ │ │ │ └── ic_launcher_round.webp
|
||||
│ │ │ │ ├── mipmap-xxxhdpi/
|
||||
│ │ │ │ │ ├── ic_launcher.webp
|
||||
│ │ │ │ │ ├── ic_launcher_foreground.webp
|
||||
│ │ │ │ │ └── ic_launcher_round.webp
|
||||
│ │ │ │ ├── values/
|
||||
│ │ │ │ │ ├── colors.xml
|
||||
│ │ │ │ │ ├── strings.xml
|
||||
│ │ │ │ │ └── themes.xml
|
||||
│ │ │ │ └── xml/
|
||||
│ │ │ │ ├── backup_rules.xml
|
||||
│ │ │ │ ├── data_extraction_rules.xml
|
||||
│ │ │ │ └── network_security_config.xml
|
||||
│ │ │ ├── AndroidManifest.xml
|
||||
│ │ │ └── ic_launcher-playstore.png
|
||||
│ │ └── test/
|
||||
│ │ └── java/
|
||||
│ │ └── com/
|
||||
│ │ └── runicgateway/
|
||||
│ │ └── app/
|
||||
│ │ ├── core/
|
||||
│ │ │ ├── auth/
|
||||
│ │ │ │ ├── sso/
|
||||
│ │ │ │ │ ├── PkceTest.kt
|
||||
│ │ │ │ │ └── SsoAuthManagerTest.kt
|
||||
│ │ │ │ └── SessionManagerTest.kt
|
||||
│ │ │ ├── net/
|
||||
│ │ │ │ ├── HostRewriteTest.kt
|
||||
│ │ │ │ ├── ServerUrlTest.kt
|
||||
│ │ │ │ └── ShardStreamClientTest.kt
|
||||
│ │ │ ├── push/
|
||||
│ │ │ │ ├── NtfyTopicTest.kt
|
||||
│ │ │ │ └── PushTickleTest.kt
|
||||
│ │ │ └── result/
|
||||
│ │ │ ├── ApiResultExtrasTest.kt
|
||||
│ │ │ └── ApiResultTest.kt
|
||||
│ │ ├── data/
|
||||
│ │ │ ├── api/
|
||||
│ │ │ │ ├── dto/
|
||||
│ │ │ │ │ ├── AccountDtoTest.kt
|
||||
│ │ │ │ │ ├── AdminDtoTest.kt
|
||||
│ │ │ │ │ ├── AuthDtoTest.kt
|
||||
│ │ │ │ │ ├── AuthRequestDtoTest.kt
|
||||
│ │ │ │ │ ├── ContentDtoTest.kt
|
||||
│ │ │ │ │ ├── NotificationsDtoTest.kt
|
||||
│ │ │ │ │ ├── PlayerGameDataDtoTest.kt
|
||||
│ │ │ │ │ ├── PlayerShardDtoTest.kt
|
||||
│ │ │ │ │ ├── PublicDtoTest.kt
|
||||
│ │ │ │ │ ├── ShardBoardDtoTest.kt
|
||||
│ │ │ │ │ ├── ShardDtoTest.kt
|
||||
│ │ │ │ │ ├── SsoDtoTest.kt
|
||||
│ │ │ │ │ └── WikiDtoTest.kt
|
||||
│ │ │ │ └── fake/
|
||||
│ │ │ │ ├── FakeAdminApi.kt
|
||||
│ │ │ │ ├── FakePlayerShardApi.kt
|
||||
│ │ │ │ ├── FakePublicApi.kt
|
||||
│ │ │ │ └── FakeShardStream.kt
|
||||
│ │ │ └── repository/
|
||||
│ │ │ ├── AccountTrustedDevicesTest.kt
|
||||
│ │ │ └── ConnectionVersionGuardTest.kt
|
||||
│ │ ├── ui/
|
||||
│ │ │ ├── admin/
|
||||
│ │ │ │ ├── AdminContentViewModelTest.kt
|
||||
│ │ │ │ ├── AdminDashboardViewModelTest.kt
|
||||
│ │ │ │ ├── AdminModerationViewModelTest.kt
|
||||
│ │ │ │ └── AdminSupportViewModelTest.kt
|
||||
│ │ │ ├── contact/
|
||||
│ │ │ │ └── ContactViewModelTest.kt
|
||||
│ │ │ ├── navigation/
|
||||
│ │ │ │ └── MenuAccessTest.kt
|
||||
│ │ │ ├── notifications/
|
||||
│ │ │ │ └── NotificationRoutingTest.kt
|
||||
│ │ │ ├── player/
|
||||
│ │ │ │ ├── CharacterSheetHelpersTest.kt
|
||||
│ │ │ │ ├── CharactersViewModelTest.kt
|
||||
│ │ │ │ └── PlayerViewModelTest.kt
|
||||
│ │ │ ├── shard/
|
||||
│ │ │ │ ├── FrameFieldsTest.kt
|
||||
│ │ │ │ ├── LiveBoardTest.kt
|
||||
│ │ │ │ ├── ShardBoardViewModelTest.kt
|
||||
│ │ │ │ └── ShardEventTextTest.kt
|
||||
│ │ │ ├── theme/
|
||||
│ │ │ │ └── BrandColorTest.kt
|
||||
│ │ │ ├── ContentViewModelTest.kt
|
||||
│ │ │ └── UiStateTest.kt
|
||||
│ │ ├── util/
|
||||
│ │ │ ├── FakeApiSupport.kt
|
||||
│ │ │ └── MainDispatcherRule.kt
|
||||
│ │ └── ScaffoldSanityTest.kt
|
||||
│ ├── build.gradle.kts
|
||||
│ └── proguard-rules.pro
|
||||
├── gradle/
|
||||
│ ├── wrapper/
|
||||
│ │ ├── gradle-wrapper.jar
|
||||
│ │ └── gradle-wrapper.properties
|
||||
│ └── libs.versions.toml
|
||||
├── .gitattributes
|
||||
├── .gitignore
|
||||
├── build.gradle.kts
|
||||
├── CODE_OF_CONDUCT.md
|
||||
├── CONTRIBUTING.md
|
||||
├── CONTRIBUTORS.md
|
||||
├── gradle.properties
|
||||
├── gradlew
|
||||
├── gradlew.bat
|
||||
├── LICENSE.md
|
||||
├── README.md
|
||||
├── SECURITY.md
|
||||
├── settings.gradle.kts
|
||||
└── sonar-project.properties
|
||||
```
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 124 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 130 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 118 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 194 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 126 KiB |
@@ -1,49 +0,0 @@
|
||||
# Android app — trusted devices & recovery codes (live smoke test)
|
||||
|
||||
Screenshots from a live end-to-end smoke test of the trusted-device + MFA feature on
|
||||
the Android client (app PR `RunicGateway/Android-app#23`), captured against the local
|
||||
Node server (`127.0.0.1:3000`) and a `uomysticmoon` MariaDB, on an API 36 emulator.
|
||||
See `../PLAN.md §4.1.1` for the design and `../../website/TRUSTED_DEVICES_MFA.md` for
|
||||
the canonical contract.
|
||||
|
||||
| # | Screenshot | Shows |
|
||||
|---|---|---|
|
||||
| 1 | [`01-login-2fa-trust-device.png`](01-login-2fa-trust-device.png) | The `401 { totpRequired }` login step: the **authentication code** field, the **"Use a recovery code instead"** toggle, and the **"Trust this device (skip codes for 30 days)"** checkbox (ticked). |
|
||||
| 2 | [`02-account-security-section.png`](02-account-security-section.png) | The new **Security** section on the account screen linking to Trusted devices and Recovery codes. |
|
||||
| 3 | [`03-trusted-devices.png`](03-trusted-devices.png) | The **Trusted Devices** screen listing this device (`Google sdk_gphone64_x86_64` — the `device_name` sent at login) with revoke / trust-this-device / untrust-all. |
|
||||
| 4 | [`04-recovery-codes-show-once.png`](04-recovery-codes-show-once.png) | The **Recovery Codes** screen after a password-stepped regenerate: the one-time batch shown once with copy / share, and the updated remaining count. |
|
||||
| 5 | [`05-recovery-code-login.png`](05-recovery-code-login.png) | Signing in with a **single-use recovery code** instead of the authenticator code. |
|
||||
|
||||
## Verified flows (all passed)
|
||||
|
||||
1. **2FA login + "Trust this device"** → `200`, trust token stored; server logged `device trusted`.
|
||||
2. **Trust survives logout** — after signing out, a **password-only** sign-in skipped the
|
||||
TOTP step entirely (server: a clean `200` with **no** preceding `401 totpRequired`).
|
||||
This is the headline behaviour: the trust token is *only* consulted at a fresh login,
|
||||
so it must outlive logout (see PLAN §4.1.1).
|
||||
3. **Trusted Devices** — list, and the device's `last_used` stamp advancing after the
|
||||
trust-skip login.
|
||||
4. **Untrust all** — cleared the server rows **and** the local token; the next
|
||||
password-only sign-in correctly required the TOTP step again (server: `401`).
|
||||
5. **Recovery codes** — generate (password step-up, shown once) and a successful
|
||||
**recovery-code login** (server: `mobile login via recovery code` → `200`).
|
||||
|
||||
## Full step-by-step walkthrough
|
||||
|
||||
The five images above are the curated highlights. These `walkthrough-*` frames are the
|
||||
rest of the same smoke-test session, in flow order, for a complete record. (Pure
|
||||
automation-artifact frames — soft-keyboard popups, mid-transition spinners, and
|
||||
duplicate Home landings — are omitted; the raws that the highlights above supersede are
|
||||
not repeated here.)
|
||||
|
||||
| # | Screenshot | Shows |
|
||||
|---|---|---|
|
||||
| 1 | [`walkthrough-01-home-connected.png`](walkthrough-01-home-connected.png) | Home with the shard **Online** (already connected to the local server), signed out. |
|
||||
| 2 | [`walkthrough-02-drawer-signed-out.png`](walkthrough-02-drawer-signed-out.png) | Navigation drawer while signed out — **Sign in** entry. |
|
||||
| 3 | [`walkthrough-03-login-screen.png`](walkthrough-03-login-screen.png) | The native login screen (no SSO providers configured in dev). |
|
||||
| 4 | [`walkthrough-04-login-2fa-step.png`](walkthrough-04-login-2fa-step.png) | The `401 { totpRequired }` step with the fields empty — the **"Use a recovery code instead"** toggle and **"Trust this device"** checkbox before entry (companion to highlight #1, which shows them filled). |
|
||||
| 5 | [`walkthrough-05-drawer-signed-in.png`](walkthrough-05-drawer-signed-in.png) | Drawer once signed in — **My account**, Notifications, player groups, **Sign out**. |
|
||||
| 6 | [`walkthrough-06-account-overview.png`](walkthrough-06-account-overview.png) | Top of the account screen: identity, username/password, **two-factor ENABLED**. |
|
||||
| 7 | [`walkthrough-07-recovery-codes-before-generate.png`](walkthrough-07-recovery-codes-before-generate.png) | Recovery Codes screen before generating — **0 codes remaining** + the password-step-up form. |
|
||||
| 8 | [`walkthrough-08-trusted-devices-empty-after-untrust.png`](walkthrough-08-trusted-devices-empty-after-untrust.png) | Trusted Devices after **Untrust all** — "All devices untrusted." and the empty state. |
|
||||
| 9 | [`walkthrough-09-login-2fa-required-after-untrust.png`](walkthrough-09-login-2fa-required-after-untrust.png) | The next sign-in **re-prompting for the TOTP step** — proof that untrust-all cleared the local trust token. |
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 100 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 76 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 77 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 120 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 109 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 125 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 95 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 79 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 120 KiB |
@@ -1,59 +0,0 @@
|
||||
# SonarQube static analysis
|
||||
|
||||
Each code repo in the Runic Gateway org reports static-analysis results to the
|
||||
self-hosted **SonarQube** server for review. Analysis is **non-blocking**: it
|
||||
runs on push to `main` (i.e. *after* merge), never on pull requests, so it never
|
||||
gates a PR. It complements each repo's PR gate and release pipeline — it only
|
||||
feeds the dashboard.
|
||||
|
||||
## Server
|
||||
|
||||
- **URL:** `https://sonar.whitlocktech.com`
|
||||
- Each repo is a separate SonarQube project, keyed as below.
|
||||
|
||||
## Projects
|
||||
|
||||
| Repo | Project key | Sources analysed | Language |
|
||||
|---|---|---|---|
|
||||
| `website` | `runic-gateway-website` | `server/src`, `client/src`, `bot/src` | JS/TS |
|
||||
| `link` | `runic-gateway-link` | `sidecar/src` | Rust |
|
||||
| `Android-app` | `runic-gateway-android-app` | `app/src/main` | Kotlin |
|
||||
|
||||
## How it's wired
|
||||
|
||||
Each repo carries two files, identical in shape across repos:
|
||||
|
||||
- **`sonar-project.properties`** (repo root) — declares the project key, sources,
|
||||
tests, and exclusions. The Sonar scanner reads this.
|
||||
- **`.gitea/workflows/sonarqube.yml`** — a `SonarQube` workflow that, on push to
|
||||
`main` (and via manual `workflow_dispatch`), checks out with full history
|
||||
(`fetch-depth: 0`, needed for accurate blame + "new code") and runs
|
||||
`sonarsource/sonarqube-scan-action@v4`.
|
||||
|
||||
The scan is **source-based** — it does not build the project or run a language
|
||||
toolchain, so the workflows are lightweight (checkout + scan only). Richer
|
||||
signals (Rust Clippy, Android Lint, JaCoCo coverage) are left as documented,
|
||||
commented-out enrichment in each repo's `sonar-project.properties`; enable them
|
||||
per repo when wanted.
|
||||
|
||||
## One-time setup per repo (Gitea UI → Repo → Settings → Actions)
|
||||
|
||||
Both are consumed by the scan action via `env:` in the workflow:
|
||||
|
||||
- **Secret `SONAR_TOKEN`** — a SonarQube *Analysis* token (My Account →
|
||||
Security in SonarQube; project-scoped or global).
|
||||
- **Variable `SONAR_HOST_URL`** — the SonarQube base URL reachable from the
|
||||
self-hosted runner. Kept as a **variable, not committed**, so the internal
|
||||
address stays out of git.
|
||||
|
||||
The self-hosted `ubuntu-latest` runner must be able to reach `SONAR_HOST_URL` on
|
||||
the network. Nothing waits on the SonarQube Quality Gate, so a failing gate does
|
||||
not fail the job — check the dashboard.
|
||||
|
||||
## Adding a new repo
|
||||
|
||||
1. Create the project in SonarQube; note its key.
|
||||
2. Add `sonar-project.properties` (copy an existing repo's, adjust key + sources).
|
||||
3. Add `.gitea/workflows/sonarqube.yml` (copy verbatim — it's language-agnostic).
|
||||
4. Set the `SONAR_TOKEN` secret and `SONAR_HOST_URL` variable in the repo's
|
||||
Gitea Actions settings.
|
||||
@@ -43,20 +43,6 @@ Pin the version you built against and compare it to the header (or `/health.prot
|
||||
|
||||
**v2 (Protocol 2.0)** added the account-provisioning surface (§6.x: `POST /accounts/create`, `DELETE /link/{account}`) and the `account.*` events. Outbound event kinds are **additive** — a v1 client that ignores unknown kinds keeps working against the live feed — but the new *endpoints* require a v2 sidecar. If you send `X-UOLink-Version: 1`, calls to the new endpoints are refused with the 409 above.
|
||||
|
||||
**v3 (Protocol 3.0) is being built and the version has not been bumped yet.** It is defined as *adds
|
||||
`world.ruleset`, `points.board`, `vendor.listing` / `vendor.listing.remove`*, and the bump to
|
||||
`X-UOLink-Version: 3` happens **exactly once**, at the end, when [`v3.md`](v3.md) §4's `edge` → `main`
|
||||
cutover lands — because a bump is an operator-visible hard break (409 on every protected route, and
|
||||
the website's WS closes on the `ws.hello` mismatch), so doing it per phase would break the site
|
||||
repeatedly.
|
||||
|
||||
Until then, sidecars on `edge` still report `2` while already carrying some v3 kinds and endpoints.
|
||||
That is safe in the direction that matters: event kinds are additive, and a client that ignores
|
||||
unknown kinds and tolerates a `404` on a not-yet-present endpoint keeps working. What you must **not**
|
||||
do is infer feature availability from the version number during this window — probe the endpoint, or
|
||||
treat a missing `world.ruleset` as "this shard hasn't published one". There is deliberately **no
|
||||
feature-negotiation array**: v3 implies all three kinds.
|
||||
|
||||
---
|
||||
|
||||
## 3. Health
|
||||
@@ -329,198 +315,6 @@ The house registry — one row per house, complementing the `house.decay` *trans
|
||||
|
||||
Render from `GET /houses` (§6) on connect, then keep live with these events.
|
||||
|
||||
#### Shard ruleset (Protocol 3.0)
|
||||
|
||||
How the shard is actually configured, published by the shard itself. **Not a sweep** — it changes only
|
||||
when an operator edits `Config/*.cfg`, so it is emitted once per shard↔sidecar connect (and on
|
||||
`[bridge reload`), exactly like `server.hello`.
|
||||
|
||||
| kind | fields | notes |
|
||||
|------|--------|-------|
|
||||
| `world.ruleset` | `rev`, `shard`, `expansion`, `connect?`, `systems`, `caps`, `housing`, `accounts`, `vetRewards`, `loot`, `vendors`, `champions?`, `treasureMaps`, `vvv?`, `store`, `schedule?` | The whole ruleset, always complete — **never a delta**, so the latest frame replaces the previous one outright. Every block except `shard`/`expansion` is optional and is **omitted when its system is off**, so absence means "not applicable here", not "unknown". |
|
||||
|
||||
`rev` is the shard's FNV-1a of the body: identical `rev` means the ruleset is unchanged and this frame
|
||||
is just a reconnect re-send, so a consumer can skip the write. It is deliberately **not**
|
||||
`String.GetHashCode()`, which is seeded per process and would change on every shard restart.
|
||||
|
||||
```json
|
||||
{"kind":"world.ruleset","rev":"1a2b3c4d","shard":"UOMysticmoon","expansion":"EJ",
|
||||
"systems":{"cityLoyalty":true,"vvv":true,"factions":false,"siege":false,"chat":true,
|
||||
"store":true,"dailyRares":true,"honesty":true,"shadowguard":true,
|
||||
"treasureMaps":true,"vetRewards":true,"testCenter":false},
|
||||
"caps":{"skill":1000,"totalSkill":7000,"stat":225,"str":125,"dex":125,"int":125,
|
||||
"strMax":150,"dexMax":150,"intMax":150},
|
||||
"housing":{"accountHouseLimit":1},
|
||||
"accounts":{"perIp":3,"charSlots":7,"autoCreate":true},
|
||||
"vetRewards":{"enabled":true,"rewardIntervalDays":30},
|
||||
"loot":{"feluccaLuckBonus":1000,"feluccaBudgetBonus":100,"feluccaMaxProps":11},
|
||||
"vendors":{"restockDelayMinutes":60,"maxSell":500,"economyStockAmount":500},
|
||||
"champions":{"powerScrolls":6,"statScrolls":16,"scrollChance":0.1,
|
||||
"transcendenceChance":50.0,"rankThresholds":[5,10,13]},
|
||||
"treasureMaps":{"enabled":true,"lootChance":0.01,"resetDays":30},
|
||||
"vvv":{"enabled":true,"startSilver":2000,"enhancedRules":false},
|
||||
"store":{"enabled":true,"currencyName":"Sovereigns"},
|
||||
"schedule":{"autoSaveEnabled":true,"autoSaveFrequencyMinutes":15,"autoRestartEnabled":false},
|
||||
"t":1752489280000}
|
||||
```
|
||||
|
||||
**Two things consumers get wrong.**
|
||||
|
||||
1. **`caps.skill` and `caps.totalSkill` are in tenths**, the way ServUO stores them: `1000` is `100.0`
|
||||
skill and `7000` is `700.0` total. Rendering the raw number is actively misleading. The other caps
|
||||
(`stat`, `str`, …) are plain integers.
|
||||
2. **`connect` is present only if the operator set `Bridge.PublicConnectAddress`.** The shard's real
|
||||
listen address (`Server.cfg`) is never published; nor are `Staff.cfg`, `Email.cfg`, `DataPath.cfg`,
|
||||
`Bridge.cfg`, `Compiler.cfg`, `Reports.cfg` or `Client.cfg`. The frame is built from an explicit
|
||||
allowlist in `BridgeRuleset.cs` — `Config.Entries` is never enumerated, because that would sweep in
|
||||
every key on the server.
|
||||
|
||||
Absent entirely if the shard runs `Bridge.RulesetEnabled=false` or an older plugin. Render from
|
||||
`GET /ruleset` (§6) on connect, then keep live with this event.
|
||||
|
||||
This **supersedes the `world.systems` frame** sketched in [`PROTOCOL_2.md`](PROTOCOL_2.md) §10.4 and
|
||||
never implemented; the `systems` block above is what that asked for.
|
||||
|
||||
#### Points / loyalty leaderboards (Protocol 3.0)
|
||||
|
||||
ServUO carries ~25 separate point currencies — Queen's Loyalty, Void Pool, Casino, Clean Up Britannia,
|
||||
the nine city loyalties, Blackthorn, the Doom / Khaldun / Kotl treasure systems — every one a standing
|
||||
players accumulate over months, and none of them visible outside an in-game gump before 3.0.
|
||||
|
||||
A diff sweep (default 300 s), **one frame per system** rather than one large frame for all of them,
|
||||
matching `champ.update` / `guild.update`. A system is emitted only when its top N or its participant
|
||||
count actually changes.
|
||||
|
||||
| kind | fields | notes |
|
||||
|------|--------|-------|
|
||||
| `points.board` | `system`, `nameString`, `nameNumber`, `maxPoints`, `showOnGump`, `players`, `top[]` | One system's complete board — **never a delta**. The latest frame for a `system` replaces the previous one outright. `top[]` entries are `{rank, serial, name, points}`. |
|
||||
|
||||
`system` is the shard's own `PointsType` enum name (`QueensLoyalty`, `CleanUpBritannia`, …) and is the
|
||||
board's stable key. There is deliberately **no `points.remove`**: the set of systems is fixed at startup
|
||||
by `PointsSystem.Configure`, so a system cannot disappear at runtime — the same argument `city.update`
|
||||
makes for cities.
|
||||
|
||||
```json
|
||||
{"kind":"points.board","system":"QueensLoyalty",
|
||||
"nameString":"Queen's Loyalty","nameNumber":1114938,
|
||||
"maxPoints":15000,"showOnGump":true,"players":842,
|
||||
"top":[{"rank":1,"serial":"0x1A2B","name":"Darrow","points":29500},
|
||||
{"rank":2,"serial":"0x1A2C","name":"Mireille","points":21000}],
|
||||
"t":1752489280000}
|
||||
```
|
||||
|
||||
**Four things consumers get wrong.**
|
||||
|
||||
1. **`maxPoints` of `0` means UNCAPPED, not "zero points allowed".** ServUO's idiom for an uncapped
|
||||
system is `double.MaxValue` (`DespiseCrystals`, `ShameCrystals` and `VoidPool` all use it), which
|
||||
the plugin normalises to `0` rather than emitting a nonsense integer. On a real shard **most
|
||||
systems are uncapped**, so a UI that renders `points / maxPoints` must special-case this or it will
|
||||
divide by zero on the common path.
|
||||
2. **`nameString` is usually `null`.** The shard's `Name` is a `TextDefinition`, which may carry a
|
||||
literal *or* a cliloc id, and in practice most systems use the cliloc — so `nameNumber` is set and
|
||||
`nameString` is `null`. Resolve clilocs consumer-side; failing that, humanising the `system` key
|
||||
("CleanUpBritannia" → "Clean Up Britannia") reads better than showing a bare number. This is the
|
||||
same contract `titles.reward` already documents.
|
||||
3. **`players` counts players who actually hold points**, not the size of the system's table. Ten of
|
||||
the ~25 systems have `AutoAdd = true` and therefore keep a zero-point row for every character that
|
||||
has ever logged in, so the raw table size would report the shard's entire character census as that
|
||||
system's participants.
|
||||
4. **Entries carry `serial` and `name` only — never `acct` or `webId`.** A board is the widest-audience
|
||||
surface the bridge has, so the account name of every ranked player deliberately does not cross the
|
||||
wire; resolve serial → site user from your own link mirror if you need it.
|
||||
|
||||
Absent entirely if the shard runs `Bridge.PointsLeaderboardEnabled=false` or an older plugin. Render
|
||||
from `GET /points` (§6) on connect, then keep live with this event.
|
||||
|
||||
##### `char.profile` gains a `points` block
|
||||
|
||||
Read-model enrichment on the existing kind — there is **no** request kind for one character's points,
|
||||
the same precedent `titles` set in [`PROTOCOL_2.md`](PROTOCOL_2.md) §10.3:
|
||||
|
||||
```json
|
||||
"points":[{"system":"QueensLoyalty","nameString":"Queen's Loyalty","nameNumber":1114938,
|
||||
"points":29500,"maxPoints":15000}]
|
||||
```
|
||||
|
||||
Systems where the character has no entry, or an entry at zero, are **omitted** — otherwise every sheet
|
||||
would carry ~25 zeroes. `maxPoints` follows the same `0 == uncapped` rule as the board.
|
||||
|
||||
`rank` is **absent by default** and appears only when the shard runs `Bridge.PointsProfileRank=true`:
|
||||
a points lookup stops at the character's own row, but a rank must count every row that beats them, in
|
||||
every system, on every profile build. Derive rank from `points.board` instead for anyone in the top N.
|
||||
|
||||
#### Player-vendor marketplace (Protocol 3.0)
|
||||
|
||||
The shard-wide shop index: every player vendor's shop name, owner, location and priced inventory —
|
||||
the same set the in-game **Vendor Search** gump reads, published so a site can offer the same search
|
||||
from outside the game.
|
||||
|
||||
An **amortized round-robin diff sweep**, not a snapshot RPC, and the distinction is load-bearing:
|
||||
`rpc.rs::try_route` correlates a reply on the FIRST frame carrying a matching `reqId`, so a chunked
|
||||
reply sharing one `reqId` would deliver chunk 1 to the HTTP caller and leak chunks 2..N onto the
|
||||
broadcast feed. A whole-world snapshot could not fit in one frame inside the 10 s reply timeout
|
||||
either. The per-account `vendor.snapshot` RPC (§5) is unaffected and still serves the player portal.
|
||||
|
||||
Each tick inventories at most `Bridge.MarketSweepBatch` vendors (default 25) starting from a
|
||||
persistent cursor, so **per-tick cost is bounded independently of world size**; full coverage takes
|
||||
`ceil(vendors / batch) × MarketSweepSeconds`. A vendor is emitted only when its contents, prices,
|
||||
shop name or location actually change.
|
||||
|
||||
| kind | fields | notes |
|
||||
|------|--------|-------|
|
||||
| `vendor.listing` | `serial`, `shopName`, `ownerSerial`, `ownerName`, `location{}`, `count`, `total`, `truncated`, `items[]` | One vendor's complete shop — **never a delta**. The latest frame for a `serial` replaces the previous one outright. |
|
||||
| `vendor.listing.remove` | `serial` | The shop is gone from the index: dismissed, expired, or its owner switched off the in-game Vendor Search flag. |
|
||||
|
||||
```json
|
||||
{"kind":"vendor.listing","serial":"0x40001234",
|
||||
"shopName":"Darrow's Bargains","ownerSerial":"0x1A2B","ownerName":"Darrow",
|
||||
"location":{"map":"Trammel","x":1421,"y":1699,"z":0,
|
||||
"region":"Britain","house":"Darrow's Villa"},
|
||||
"count":2,"total":2,"truncated":false,
|
||||
"items":[{"serial":"0x40012ABC","itemId":3922,"hue":0,"amount":1,
|
||||
"price":25000,"name":null,"cliloc":1023721},
|
||||
{"serial":"0x40012ABD","itemId":7026,"hue":1157,"amount":3,
|
||||
"price":500,"name":"a shard sigil","cliloc":1041243}],
|
||||
"t":1752489280000}
|
||||
```
|
||||
|
||||
**Six things consumers get wrong.**
|
||||
|
||||
1. **`name` is `null` for nearly every item; `cliloc` is the real label.** Items carry a
|
||||
`LabelNumber`, not a name. The plugin deliberately never calls `VendorSearch.GetItemName`, which
|
||||
builds an `ObjectPropertyList`, serialises it and byte-parses the packet **per item** — a
|
||||
multi-hundred-millisecond stall across a full pass. (It would not work anyway: every current
|
||||
client ships its cliloc files compressed and ServUO's bundled `Ultima.StringList` cannot read
|
||||
them, so the in-game gump has the same gap.) Resolve clilocs consumer-side; a non-null `name` is a
|
||||
player-set literal and is strictly more specific, so **prefer it over the cliloc**.
|
||||
2. **`location` is one nested object, and it may be absent entirely.** It is nested so that a
|
||||
consumer gating vendor whereabouts gates one field rather than five that can drift apart — the
|
||||
website's `market.location` rule removes the whole object. Treat a missing `location` as "not
|
||||
published", not as an error.
|
||||
3. **`truncated` means the shop holds more than the frame carries.** `count` is what was published,
|
||||
`total` is what the shop actually holds, capped by `Bridge.MarketMaxListings` (default 250). A
|
||||
commodity reseller with thousands of stacked resources is real and an uncapped frame for one is
|
||||
measured in megabytes. Say "showing 250 of 3,104" rather than presenting a partial shop as
|
||||
complete.
|
||||
4. **`child: true` means the price buys the ENCLOSING CONTAINER.** ServUO prices a container as a
|
||||
unit and everything inside inherits that price with no `VendorItem` of its own; `DoSearch`
|
||||
surfaces the same flag. A UI that prints the container's price against each item inside it is
|
||||
lying about the shard.
|
||||
5. **Opted-out vendors are absent, and that is a privacy control.** `pv.VendorSearch` is the player's
|
||||
own in-game toggle and the sweep honours it — hide your vendor in game and it is hidden here too.
|
||||
The same goes for `Map.Internal` and a null backpack, matching `DoSearch`. Process
|
||||
`vendor.listing.remove` promptly: it is how a player *revoking* that consent reaches you.
|
||||
6. **Prices are inherently stale, by design.** The round-robin sweep means a shop can be a full cycle
|
||||
behind. Any UI over this must say how old the data may be — the website derives it from the oldest
|
||||
vendor row.
|
||||
|
||||
Entries carry `ownerSerial`/`ownerName` and **never `acct` or `webId`**, the same rule `points.board`
|
||||
follows. Absent entirely if the shard runs `Bridge.MarketEnabled=false` or an older plugin. Render
|
||||
from `GET /market` (§6) on connect, then keep live with these events — though note that a live
|
||||
firehose of whole vendor inventories is the largest stream the bridge produces, and a consumer that
|
||||
only needs a browsable index (as the website does) is better served by the REST read plus the
|
||||
periodic re-sweep.
|
||||
|
||||
---
|
||||
|
||||
## 5. REST — read queries
|
||||
@@ -560,7 +354,7 @@ Full character sheet: stats, all trained skills, worn equipment with flattened i
|
||||
Field notes:
|
||||
- `skills[].base` is trained value, `value` includes item/temp bonuses, `cap` is the cap. **Do not assume `base <= cap`** — GM characters can exceed it.
|
||||
- `equipment[].mods` is a flattened map of every non-zero AOS attribute on the item (weapon or armor). Empty `{}` for plain items.
|
||||
- Item names are usually **clilocs**, not strings: use `name` when present, otherwise resolve `cliloc` against a UO cliloc table on the site. **Do not expect the shard to resolve them for you** — on any modern client ServUO's own `Ultima.StringList` cannot read the client's compressed cliloc files, so `VendorSearch.GetItemName` returns `item.Name` and the in-game Vendor Search gump has the same gap. Building that table is a consumer-side job; the website's is described in [`website/CLILOCS.md`](../website/CLILOCS.md).
|
||||
- Item names are usually **clilocs**, not strings: use `name` when present, otherwise resolve `cliloc` against a UO cliloc table on the site.
|
||||
- `titles` (Protocol 2.0): `selected` is the index into `reward` currently displayed (`-1` if none). `fameKarma`/`skill` are computed display titles, omitted when the character has none. `reward` entries may be a **cliloc number as a string** or a literal string — resolve numeric ones against your cliloc table, same as item names.
|
||||
- Errors: unknown account → **404** `{"kind":"bridge.error","reason":"unknown account"}`; bad slot → **404**/**400** similarly.
|
||||
|
||||
@@ -864,72 +658,6 @@ GET /houses
|
||||
|
||||
Every house's latest snapshot — owner→houses map. Served from the sidecar's projection, kept current by the `house.*` stream (§4). Ordered by name. Survives a sidecar restart.
|
||||
|
||||
### Shard ruleset (Protocol 3.0)
|
||||
|
||||
```
|
||||
GET /ruleset
|
||||
→ { "ruleset": {"kind":"world.ruleset","rev":"1a2b3c4d","shard":"UOMysticmoon",
|
||||
"expansion":"EJ","systems":{...},"caps":{...},"accounts":{...}, ... } }
|
||||
```
|
||||
|
||||
The shard's published ruleset (§4 for the full frame and its two gotchas). Served from the sidecar's
|
||||
store, so it **answers while the shard is down** — a rules page that goes blank during a restart is
|
||||
worse than one that is briefly stale. Keep it current with the `world.ruleset` stream.
|
||||
|
||||
`{"ruleset": null}` means the shard has never published one — an older plugin, or
|
||||
`Bridge.RulesetEnabled=false`. That is a real answer distinct from a published ruleset, and worth
|
||||
rendering differently ("not published yet") rather than as an empty ruleset.
|
||||
|
||||
### Points / loyalty leaderboards (Protocol 3.0)
|
||||
|
||||
```
|
||||
GET /points
|
||||
→ { "boards": [ {"kind":"points.board","system":"QueensLoyalty","nameString":"Queen's Loyalty",
|
||||
"nameNumber":1114938,"maxPoints":15000,"showOnGump":true,"players":842,
|
||||
"top":[{"rank":1,"serial":"0x1A2B","name":"Darrow","points":29500}, ...],"t":...}, ... ] }
|
||||
|
||||
GET /points/{system} # e.g. /points/QueensLoyalty
|
||||
→ {"kind":"points.board","system":"QueensLoyalty", ... }
|
||||
```
|
||||
|
||||
Every system's latest board, or one by its `PointsType` name (§4 for the frame and its four gotchas).
|
||||
Served from the sidecar's projection, kept current by the `points.board` stream, ordered by display
|
||||
name. Survives a sidecar restart — which matters more here than for live state, since these are
|
||||
standings built over months and blanking them during a restart reads as data loss.
|
||||
|
||||
`GET /points/{system}` returns **404** for a system the shard has never published (an unknown name, or
|
||||
one excluded by `Bridge.PointsSystems`). That is distinct from a published board nobody has scored in
|
||||
yet, which is **200** with an empty `top[]` — and the two are worth rendering differently.
|
||||
|
||||
### Player-vendor marketplace (Protocol 3.0)
|
||||
|
||||
```
|
||||
GET /market?limit=200&offset=0
|
||||
→ { "vendors": [ {"kind":"vendor.listing","serial":"0x40001234",
|
||||
"shopName":"Darrow's Bargains","ownerSerial":"0x1A2B","ownerName":"Darrow",
|
||||
"location":{"map":"Trammel","x":1421,"y":1699,"z":0,
|
||||
"region":"Britain","house":"Darrow's Villa"},
|
||||
"count":2,"total":2,"truncated":false,"items":[ ... ],"t":...}, ... ],
|
||||
"total": 137, "limit": 200, "offset": 0 }
|
||||
```
|
||||
|
||||
Every vendor's latest shop, exactly as `vendor.listing` published it (§4 for the frame and its six
|
||||
gotchas). Served from the sidecar's projection, so it answers while the shard is down.
|
||||
|
||||
**This is the only PAGED read the sidecar serves**, because it is the only board that can be a whole
|
||||
world's inventory. `limit` is clamped to 1..1000 (default 200); `total` is returned so a caller knows
|
||||
when to stop rather than paging until it sees a short page, which would race a concurrent sweep.
|
||||
Ordering is by **serial**, not by shop name — a serial is stable while a shop name is renameable, so
|
||||
a rename mid-walk cannot make a vendor skip or repeat a page.
|
||||
|
||||
The route is `/market` and deliberately **not** `/vendors`: `/vendors/{account}` next door is the
|
||||
per-account RPC (§5), and two routes a prefix apart meaning "this player's shops" and "every shop on
|
||||
the shard" is a trap nobody wins.
|
||||
|
||||
Frames are served **verbatim**, owner names and coordinates included. That is not an oversight: the
|
||||
sidecar defines no audiences. Deciding who may see what is the consuming site's job — see
|
||||
[`v3.md`](v3.md) §3 for how the website does it.
|
||||
|
||||
---
|
||||
|
||||
## 7. Status codes
|
||||
|
||||
39
link/PLAN.md
39
link/PLAN.md
@@ -284,15 +284,6 @@ Counts in `hello` are a live snapshot taken on the Core thread, not a cached val
|
||||
|
||||
`Item.Name` is frequently `null`; the display name is `LabelNumber`, a cliloc id. **There is no `Data/Cliloc.enu` in this repo** — `BRIDGE_FINDINGS.md` §IV.4 is wrong about this. Cliloc data lives in the client install, which `DataPath` resolves to `D:\Games\Electronic Arts\Ultima Online Classic\`. Ship **both** `name` (when non-null) and `cliloc`, and resolve the number **on the website** against a cliloc map. That avoids a server-side dependency on the client directory.
|
||||
|
||||
**Update (3.0).** That recommendation held, and the reason it had to hold turned out to be stronger
|
||||
than "avoids a dependency": **ServUO cannot resolve clilocs either.** Every current client ships its
|
||||
`Cliloc.*` files compressed, and the bundled `Ultima.StringList` reads only the older plain layout —
|
||||
so `VendorSearch.StringList` is null and `VendorSearch.GetItemName` returns `item.Name` on any modern
|
||||
shard. The in-game Vendor Search gump has the same gap, which is why `vendor.listing` never calls it.
|
||||
Pushing name resolution to the plugin was never an option. See [`v3.md`](v3.md) §8.6 and
|
||||
`docs/website/CLILOCS.md` for how the site gets a table instead (the operator converts one from their
|
||||
own client, once).
|
||||
|
||||
---
|
||||
|
||||
## 8. Corrections to `BRIDGE_FINDINGS.md`
|
||||
@@ -324,29 +315,6 @@ own client, once).
|
||||
7. **Core edit: `PlayerVendorSale`** (§6). Then the cheat-detection feed.
|
||||
8. **Cheat signals.** `FastWalk`, `OnPropertyChanged` audit, vendor-sale anomaly detection in the sidecar.
|
||||
|
||||
**Beyond 1.0.** Phases above are the 1.0 read/event plane. Protocol 2.0's phasing (provisioning +
|
||||
world-state boards) is [`PROTOCOL_2.md`](PROTOCOL_2.md) §13; Protocol 3.0's (visibility framework,
|
||||
shard content and standings) is [`v3.md`](v3.md) §9, which also tracks what has landed. Shipped from
|
||||
3.0 so far: **Part A** — the visibility framework — **`world.ruleset`** ([`v3.md`](v3.md) §5),
|
||||
`BridgeRuleset.cs`, the first bridge stream that is neither an event subscription nor a sweep (it is
|
||||
emitted once per connect, like `server.hello`, because shard config changes only when an operator
|
||||
edits a file) — the **spawn atlas** ([`v3.md`](v3.md) §6), which is website-only and touches no wire
|
||||
at all — and **`points.board`** ([`v3.md`](v3.md) §7), `BridgePoints.cs`, the loyalty/points
|
||||
leaderboards. `BridgePoints` is the widest read the bridge performs: ten of ServUO's ~25 point systems
|
||||
keep a row for every character ever created, so it selects the top N in a single bounded pass rather
|
||||
than sorting, and runs on a deliberately slow 300 s interval.
|
||||
|
||||
Also shipped: **`vendor.listing`** ([`v3.md`](v3.md) §8), `BridgeMarket.cs`, the shard-wide
|
||||
player-vendor index. It introduces the one sweep pattern the bridge did not previously have — an
|
||||
**amortized round-robin**. Every other sweep walks its whole collection per tick, which is fine for
|
||||
tens of houses or a fixed set of point systems and is not fine for a world of shops whose inventories
|
||||
recurse into containers. `BridgeMarket` inventories at most `MarketSweepBatch` vendors per tick from
|
||||
a persistent cursor, so the per-tick cost is bounded by the batch rather than by world size, and full
|
||||
coverage takes `ceil(vendors / batch) x MarketSweepSeconds`. Measured at **15.4 ms** for a cold tick
|
||||
of 25 vendors x 40 listings and **0.3 ms** in steady state (the per-vendor diff), on a shard of 209k
|
||||
items / 43k mobiles. It is also the first stream to honour a per-player privacy toggle: ServUO's own
|
||||
`PlayerVendor.VendorSearch` flag, so a shop hidden in game is hidden on the site.
|
||||
|
||||
### Config keys (`Config/Bridge.cfg`)
|
||||
|
||||
```ini
|
||||
@@ -360,13 +328,6 @@ EconomySweepSeconds=300
|
||||
|
||||
Read in `Configure()` via `Config.Get<T>("Bridge.<Key>", default)`. Key scope is the filename: `Bridge.cfg` + `StatSweepSeconds` → `Bridge.StatSweepSeconds`.
|
||||
|
||||
The set above is the 1.0 sample, not the current one — every later phase added keys (sweep intervals
|
||||
for each board, the town-crier/news caps, the admin write plane, account provisioning, and 3.0's
|
||||
`RulesetEnabled` / `PublicConnectAddress` / `RulesetIncludeSchedule`, and the `Points*` and `Market*`
|
||||
blocks).
|
||||
**`servuo-plugins/overlay/Config/Bridge.cfg`
|
||||
is the authoritative, commented list**; `BridgeConfig.cs` holds the defaults.
|
||||
|
||||
---
|
||||
|
||||
## 11. Phase 1 acceptance
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
# uo-link — Project Tree
|
||||
|
||||
> **Auto-generated.** This file is maintained by the `sync-project-tree` CI workflow in
|
||||
> the [`RunicGateway/link`](https://gitea.whitlocktech.com/RunicGateway/link) repository, which
|
||||
> opens a pull request here whenever the tracked file layout on `main` changes. Do not edit
|
||||
> by hand — changes will be overwritten by the next sync.
|
||||
|
||||
A snapshot of the tracked files in the repository (build output, dependencies, and other
|
||||
git-ignored paths are excluded).
|
||||
|
||||
```text
|
||||
link/
|
||||
├── .gitea/
|
||||
│ ├── ISSUE_TEMPLATE/
|
||||
│ │ ├── bug_report.md
|
||||
│ │ ├── config.yaml
|
||||
│ │ └── feature_request.md
|
||||
│ ├── scripts/
|
||||
│ │ └── gen_tree.py
|
||||
│ ├── workflows/
|
||||
│ │ ├── release.yml
|
||||
│ │ ├── sonarqube.yml
|
||||
│ │ └── sync-project-tree.yml
|
||||
│ └── PULL_REQUEST_TEMPLATE.md
|
||||
├── sidecar/
|
||||
│ ├── src/
|
||||
│ │ ├── config.rs
|
||||
│ │ ├── main.rs
|
||||
│ │ ├── rpc.rs
|
||||
│ │ ├── shard.rs
|
||||
│ │ ├── store.rs
|
||||
│ │ └── web.rs
|
||||
│ ├── .gitignore
|
||||
│ ├── Cargo.lock
|
||||
│ ├── Cargo.toml
|
||||
│ ├── README.md
|
||||
│ └── sidecar.toml.example
|
||||
├── .gitignore
|
||||
├── CODE_OF_CONDUCT.md
|
||||
├── CONTRIBUTING.md
|
||||
├── CONTRIBUTORS.md
|
||||
├── LICENSE.md
|
||||
├── README.md
|
||||
├── SECURITY.md
|
||||
└── sonar-project.properties
|
||||
```
|
||||
@@ -285,18 +285,6 @@ City titles and faction/VvV merchant titles (`CityLoyaltySystem.ApplyCityTitle`,
|
||||
|
||||
### 10.4 Factions / Vice vs Virtue
|
||||
|
||||
> **Status update (Protocol 3.0, 2026-07-28).**
|
||||
>
|
||||
> - **The deferred question is answered.** This shard runs **Vice vs Virtue** (`VvV.cfg Enabled=True`);
|
||||
> old Factions is off, and in stock ServUO that is not a coincidence —
|
||||
> `Services/Factions/Core/Faction.cs` sets `Settings.Enabled = !ViceVsVirtueSystem.Enabled`, so the
|
||||
> two are mutually exclusive by construction. The `vvv.standings` / `vvv.battle` streams below are
|
||||
> therefore unblocked, but are **not** scoped for 3.0 (see [`v3.md`](v3.md) §2 row 5).
|
||||
> - **`world.systems` is superseded by `world.ruleset`** ([`v3.md`](v3.md) §5), which shipped in 3.0.
|
||||
> It was never implemented under this name. `world.ruleset` carries the same
|
||||
> `systems{cityLoyalty, vvv, factions, …}` sub-object this section asked for, plus the rest of the
|
||||
> shard's published ruleset, so no orphan kind is left behind. Do not implement `world.systems`.
|
||||
|
||||
**Which system is live is a shard decision — verify before building.** Two exist:
|
||||
|
||||
- **Old Factions** (`Scripts/Services/Factions`): `Faction.Commander` (leader, `Faction.cs:160`), `Faction.Election`, `Faction.Members` (`List<PlayerState>`), and faction-controlled **Towns** (`Town.cs` — each town has an owning faction, a sheriff, and finance). Config-gated and, on most modern shards, **off**.
|
||||
@@ -312,10 +300,7 @@ City titles and faction/VvV merchant titles (`CityLoyaltySystem.ApplyCityTitle`,
|
||||
{"kind":"vvv.standings","order":142000,"chaos":138500,"leaderSide":"Order"}
|
||||
```
|
||||
|
||||
> Start by detecting which system is enabled at boot and streaming only that one. ~~emit a one-time
|
||||
> `world.systems` frame (what's on: cityLoyalty, vvv, factions) so the website renders the right panels
|
||||
> instead of guessing.~~ — **superseded: `world.ruleset` already carries that `systems` block** (see the
|
||||
> status note at the top of this section).
|
||||
> Start by detecting which system is enabled at boot and streaming only that one; emit a one-time `world.systems` frame (what's on: cityLoyalty, vvv, factions) so the website renders the right panels instead of guessing.
|
||||
|
||||
## 11. Further integration points — a menu to pick from
|
||||
|
||||
|
||||
960
link/v3.md
960
link/v3.md
@@ -1,960 +0,0 @@
|
||||
# Protocol 3.0 — Shard content, standings & the visibility framework
|
||||
|
||||
**Status:** In progress. All work lands on an `edge` branch in each repo; `edge` → `main` is the v3 cutover.
|
||||
**Date:** 2026-07-28
|
||||
**Codebase:** ServUO 57.4, `<servuo>`, net48 / x64, Expansion **EJ**.
|
||||
**Companion to** [`PLAN.md`](PLAN.md) (1.0 read/event plane), [`PROTOCOL_2.md`](PROTOCOL_2.md) (2.0 provisioning + world-state streams), [`ADMIN_CONTROLS.md`](ADMIN_CONTROLS.md) (staff write plane), [`INTEGRATION.md`](INTEGRATION.md) (website API).
|
||||
|
||||
### Progress
|
||||
|
||||
Each part is marked off here as it lands on `edge`. §9 carries the same state per sequencing row.
|
||||
|
||||
| Order | Part | State | Landed on `edge` |
|
||||
|---|---|---|---|
|
||||
| 1 | **A** — visibility framework + actor-leak fix (§3) | ✅ **Done** | website [#109](https://gitea.whitlocktech.com/RunicGateway/website/pulls/109) + [#110](https://gitea.whitlocktech.com/RunicGateway/website/pulls/110), docs [#64](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/64) + [#65](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/65) |
|
||||
| 2 | **B/1** — `world.ruleset` (§5) | ✅ **Done** | servuo-plugins [#3](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/3), link [#17](https://gitea.whitlocktech.com/RunicGateway/link/pulls/17), website [#111](https://gitea.whitlocktech.com/RunicGateway/website/pulls/111), docs [#66](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/66) |
|
||||
| 3 | **C** — spawn atlas (§6) | ✅ **Done** | website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112) (parsers + CLI + tables) + [#113](https://gitea.whitlocktech.com/RunicGateway/website/pulls/113) (API + pages + admin panel), docs [#67](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/67) + [#68](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/68) |
|
||||
| 4 | **B/2** — `points.board` (§7) | ✅ **Done** | servuo-plugins [#4](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/4), link [#18](https://gitea.whitlocktech.com/RunicGateway/link/pulls/18), website [#114](https://gitea.whitlocktech.com/RunicGateway/website/pulls/114), docs [#69](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/69) |
|
||||
| 5a | **B/3 dependency** — cliloc table (§8.6) | ✅ **Done** | website [#115](https://gitea.whitlocktech.com/RunicGateway/website/pulls/115), docs [#70](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/70) |
|
||||
| 5b | **B/3** — `vendor.listing` (§8) | 🟨 In review | servuo-plugins [#5](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/5), link [#19](https://gitea.whitlocktech.com/RunicGateway/link/pulls/19), website [#116](https://gitea.whitlocktech.com/RunicGateway/website/pulls/116), docs [#71](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/71) |
|
||||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3 (§4) | ⬜ Not started | — |
|
||||
|
||||
Order 5 split in two once §8.6's cliloc dependency turned out to be a client-format problem rather
|
||||
than a parser (see §8.6). 5a is website-only and lands first so the marketplace ships with real item
|
||||
names; 5b is the four-repo wire change.
|
||||
|
||||
---
|
||||
|
||||
## 1. Why 3.0
|
||||
|
||||
A survey of the live ServUO tree against everything the bridge already surfaces end-to-end found that
|
||||
**the bridge covers live *activity* well and covers shard *content and standings* almost not at all.**
|
||||
|
||||
Covered by 1.0 + 2.0: presence/online, region transitions, char vitals + profile + roster, house
|
||||
registry + IDOC decay, champion spawns, guild board, city governors + term history, help-page queue,
|
||||
total gold supply, player-vendor sales log, deaths/murders/kills, skill gains, fame/karma, quest
|
||||
completes, staff/cheat audit, account linking + creation, town crier + news.
|
||||
|
||||
Not covered by anything: every leaderboard, every ruleset fact, every "where do I find X", and the
|
||||
entire player economy outside a player's own vendors.
|
||||
|
||||
3.0 has **three scope areas**:
|
||||
|
||||
- **A — The visibility framework (§3).** Admin-configurable, per-feature and per-field audience
|
||||
control over every shard-derived surface on the website. Ships first; the rest depends on it.
|
||||
- **B — Three new wire streams (§5, §7, §8).** `world.ruleset`, `points.board`,
|
||||
`vendor.listing`/`vendor.listing.remove`.
|
||||
- **C — One website-only feature (§6).** The spawn atlas, built from static ServUO data files with no
|
||||
wire involvement at all.
|
||||
|
||||
---
|
||||
|
||||
## 2. Survey: the full gap list
|
||||
|
||||
Recorded so the items *not* scoped for 3.0 aren't re-derived later.
|
||||
|
||||
| # | Gap | Source on the shard | Value | Cost | Status |
|
||||
|---|---|---|---|---|---|
|
||||
| 1 | **Points/loyalty leaderboards** — 25 point currencies | `Scripts/Services/PointsSystems/PointsSystem.cs` → `static List<PointsSystem> Systems`, each `List<PointsEntry>{Player,Points}` | Very high | Low | **3.0 §7** |
|
||||
| 2 | **Shard ruleset page** | `Config/*.cfg` via `Server.Config.Get` | High | Very low | **3.0 §5** |
|
||||
| 3 | **Shard-wide marketplace** | `PlayerVendor.PlayerVendors` + `VendorSearch.cs` | Very high | High | **3.0 §8** |
|
||||
| 4 | **Spawn atlas / bestiary** | `Spawns/*.xml` (6,455 spawners), `RevampedSpawns/*.xml` (333), `Data/Regions.xml`, `Data/Locations/*.xml`, `Config/ChampionSpawns.xml`, `Data/teleporters.csv`, `Data/HarvestLocs/*` | High | Medium | **3.0 §6** |
|
||||
| 5 | VvV standings + battle status | `Services/ViceVsVirtue/{ViceVsVirtueSystem,GuildStats,VvVBattle}.cs` | High | Medium | `PROTOCOL_2.md` §10.4 deferred this pending "which PvP system does this shard run?" — **now answered: `VvV.cfg Enabled=True`, `Factions.cfg` off.** Unblocked, not scoped here |
|
||||
| 6 | Skill leaderboards + shard census | `Services/Reports/Reports.cs` → `GetSkillDistribution()`, `CompileGeneralStats()`, `StaffHistory` | High | Low | Spec'd `PROTOCOL_2.md` §14 (Part B phase 6), unbuilt. Shares §7's UI — fold in after |
|
||||
| 7 | Custom mounts/pets codex — ~35 across 4 tiers | `Scripts/Custom/{Companions,Legendary,Named,New Legacy}` | Medium-high | Very low | Pure wiki/CMS content, zero bridge work. The shard's most distinctive content, with zero site presence |
|
||||
| 8 | Community Collections progress | `Services/CommunityCollections/CollectionsSystem.cs` | Medium | Low | Natural public "community goal" widget |
|
||||
| 9 | Seasonal/holiday event calendar | `Services/Seasonal Events/SeasonalEventSystem.cs`, Krampus, Forsaken Foes | Medium | Low | "What's live now / what's next" |
|
||||
| 10 | Crafting / taming / harvesting feeds | `EventSink.CraftSuccess` / `TameCreature` / `ResourceHarvestSuccess` | Medium | Low | Spec'd `PROTOCOL_2.md` §11 #3/#4/#5, unbuilt |
|
||||
| 11 | Virtue progression | `EventSink.VirtueLevelChange`, `Services/Ethics/` | Medium | Low | Spec'd §11 #6, unbuilt |
|
||||
| 12 | Bulk Order Deeds + reward tables | `Services/BulkOrders/`, `Data/Bulk Orders/*` | Medium | Low | Feed spec'd §11 #7; the static reward tables are a free wiki page |
|
||||
| 13 | Guild wars | war state on `Guild` | Low-medium | Low | Spec'd §11 #8, unbuilt |
|
||||
| 14 | Astronomy discovery log | `Services/Astronomy/AstronomySystem.cs` (104 KB save) | Low | Low | Niche completion leaderboard |
|
||||
| 15 | In-game chat relay | `Services/Chat/`, `Logs/Chat/{General,Help,Trade,LFG}` | Low | Medium | Privacy-sensitive; staff-only at most |
|
||||
| 16 | Shard health telemetry | Crash logs, `LayerConflict.log`, `throttle.log`, `world.save.after` counts, AutoSave/AutoRestart schedule | Low-medium | Low | `world.save.after` is already ingested but never charted — world-size-over-time is nearly free |
|
||||
| 17 | Ultima Store / Sovereigns balance | `Store.cfg Enabled=True, CurrencyName=Sovereigns`; `UltimaStore.GetCurrency` | ? | Medium | Only worth it if sovereigns are actually sold |
|
||||
|
||||
**Excluded permanently** — see [`ADMIN_CONTROLS.md`](ADMIN_CONTROLS.md) Tier H/N: firewall/IP-block,
|
||||
kill/resurrect, jail, item/gold grants, set-access-level, arbitrary `[set`/`[add`.
|
||||
|
||||
**Noticed during the survey, out of scope:** `Scripts/Custom/PerryOwnerFix.cs` hardcodes an
|
||||
`EventSink.Login` hook granting `AccessLevel.Owner` to account `"ShardOnwerPerry"`. Worth reviewing
|
||||
independently of this work.
|
||||
|
||||
---
|
||||
|
||||
## 3. Part A — The visibility framework ✅ Done
|
||||
|
||||
*Landed on `edge`: website [#109](https://gitea.whitlocktech.com/RunicGateway/website/pulls/109) (the framework)
|
||||
and [#110](https://gitea.whitlocktech.com/RunicGateway/website/pulls/110) (the REST-projection gap §3.6.1
|
||||
records), docs [#64](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/64) + [#65](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/65).
|
||||
Smoke-tested across all five rungs per §11.*
|
||||
|
||||
### 3.1 The leak this replaces (verified 2026-07-28)
|
||||
|
||||
`BridgeJson.Actor()` (`BridgeJson.cs:85-117`) writes `serial`, `name`, **`acct`**, **`webId`**,
|
||||
`player`. `shardState.model.js:346 shapeGuild()` returns `r.payload` verbatim, and
|
||||
`GET /api/v1/public/shard/guilds` (anonymous, `shard.controller.js:131`) serves it. **A guild
|
||||
leader's game account name and website user id are readable on an anonymous public endpoint today.**
|
||||
The same path exists for `shapeGovernor` → `/public/shard/governors`. `Actor` also feeds
|
||||
`guild.join`, `city.update` and `region.enter`, all three in `PUBLIC_KINDS` on the anonymous SSE
|
||||
stream.
|
||||
|
||||
The framework below is the vehicle for the fix, and the reason it ships before anything else.
|
||||
|
||||
### 3.2 Where visibility lives
|
||||
|
||||
**On the website, never in the sidecar.** The sidecar's job for 3.0 is unchanged in character: accept
|
||||
frames, persist them to its SQLite store, forward them verbatim over WS, and serve store-backed reads
|
||||
that survive a shard outage. It defines no access parameters, no audiences, no field projection, and
|
||||
advertises no capabilities.
|
||||
|
||||
### 3.3 The audience ladder
|
||||
|
||||
`anonymous → logged_in → player → staff → admin`, each rung implying the ones below it.
|
||||
|
||||
`viewerLevel(req)` resolves: no session ⇒ `anonymous`; authenticated ⇒ `logged_in`; authenticated
|
||||
with a linked shard account ⇒ `player`; moderator/admin role ⇒ `staff`/`admin`. **Staff always
|
||||
satisfy the `player` rung** even without a linked game account, consistent with the existing rule
|
||||
that `/player/*` is role-agnostic self-service.
|
||||
|
||||
### 3.4 Two limits an admin cannot override
|
||||
|
||||
1. **`acct` and `webId` are admin-only, always.** They are not in-game-visible and are not exposed as
|
||||
configurable fields.
|
||||
2. **A kind absent from the kind→feature map is never broadcast below `admin`.** Fail closed. This
|
||||
preserves the property that today's static `PUBLIC_KINDS` allowlist is a security boundary rather
|
||||
than a convenience filter.
|
||||
|
||||
### 3.5 Configuration
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS shard_feature_visibility (
|
||||
feature VARCHAR(48) NOT NULL PRIMARY KEY,
|
||||
enabled TINYINT(1) NOT NULL DEFAULT 1,
|
||||
audience VARCHAR(20) NOT NULL DEFAULT 'anonymous',
|
||||
field_rules JSON NULL, -- {"<field>": "<rung>"} for sensitive fields only
|
||||
updated_by INT NULL,
|
||||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||||
```
|
||||
|
||||
**Not** seeded on boot (this changed during implementation): an **absent row means "use the compiled
|
||||
default"**, so the table starts empty and only ever holds rows an admin has actually touched. The
|
||||
defaults live in one place — `FEATURES` in `shardVisibility.js` — instead of being duplicated into a
|
||||
seeder that could drift from it, and a DB blip degrades to those same defaults rather than to
|
||||
"everything is public". **All ten shard features are covered — the four new ones and the six that
|
||||
already ship — and every default reproduces today's behavior, so the retrofit is a no-op until an
|
||||
admin changes something.**
|
||||
|
||||
| Feature | Default audience | Sensitive fields (default rung) |
|
||||
|---|---|---|
|
||||
| `status`, `activity`, `champs`, `guilds`, `governors` | `anonymous` | guilds/governors: `leaderAcct` / `leaderWebId` → **admin (locked)** |
|
||||
| `houses` | `anonymous` | `owner` → `staff`, `price` → `staff` (matches today's IDOC-only public view) |
|
||||
| `presence` | `anonymous` | `location` → `staff` (matches today's staff-only, location-gated `/online`) |
|
||||
| `ruleset` (new) | `anonymous` | `connect` → `anonymous` |
|
||||
| `atlas` (new) | `anonymous` | — |
|
||||
| `leaderboards` (new) | `anonymous` | `characterName` → `anonymous` |
|
||||
| `market` (new) | `anonymous` | `ownerName` → `anonymous`, `location` → `anonymous` |
|
||||
|
||||
### 3.6 Enforcement — three points, one config
|
||||
|
||||
New `website/server/src/utils/shardVisibility.js`:
|
||||
|
||||
- `LADDER = ['anonymous','logged_in','player','staff','admin']`, `rank()`, `meets(viewer, required)`
|
||||
- `viewerLevel(req)` (§3.3)
|
||||
- `KIND_FEATURE` — every event kind → its feature; unmapped ⇒ admin-only (§3.4)
|
||||
- `getConfig()` — DB-backed, cached ~5 s like `uoLinkClient`'s config cache, busted on admin `PUT`
|
||||
- `requireFeature(name)` — **404 when disabled** (don't leak existence), **403 when enabled but the
|
||||
viewer is below the audience**
|
||||
- `projectFeature(name, payload, viewerLevel)` — strips fields whose rung the viewer doesn't meet;
|
||||
`acct`/`webId` always stripped below `admin`
|
||||
|
||||
Applied at:
|
||||
|
||||
1. **Routes** — `requireFeature(…)` on every `/public/shard/*`, `/public/atlas/*` and the
|
||||
shard-derived player routes; `projectFeature` in the controllers, replacing the ad-hoc
|
||||
`shapeGuild`-returns-payload-verbatim path.
|
||||
2. **SSE** — `shardBroadcast.js` moves from *"one public channel with a static `PUBLIC_KINDS`
|
||||
allowlist plus one admin channel"* to **per-connection filtering**: each subscriber carries its
|
||||
`viewerLevel`; each frame is mapped kind→feature, gated on `enabled && meets(...)`, then passed
|
||||
through `projectFeature` before write. `PUBLIC_KINDS` becomes the seed data for `KIND_FEATURE`
|
||||
rather than a hardcoded gate. **This is the largest single change in Part A and where the security
|
||||
boundary now lives.**
|
||||
3. **Nav** — `GET /api/v1/public/shard/features` returns only the features the calling viewer can
|
||||
see, so the SPA hides nav entries rather than rendering links that 403.
|
||||
|
||||
### 3.6.1 What the first implementation missed (found by the §11 smoke test, fixed)
|
||||
|
||||
Part A shipped enforcement on the SSE path and on `/guilds` + `/governors`, but the **remaining public
|
||||
REST reads never called into it** — so the same event was projected live and served verbatim from
|
||||
history. Recorded because each miss is a shape the next phase can repeat:
|
||||
|
||||
- **`/public/shard/feed` returned the stored payload as-is.** `actor.acct` / `actor.webId` were
|
||||
readable *anonymously* for every logged kind (`player.death`, `mob.killed`, `skill.gain`,
|
||||
`guild.join`, …) — broader than the §3.1 leak, which was limited to board holders.
|
||||
- **`/public/shard/idoc` returned `ownerAcct`.** Rule 1 keyed on the exact strings `acct`/`webId`,
|
||||
but `shapeHouse` flattens the actor into `ownerAcct` / `ownerName` / `ownerSerial`. The lock is now
|
||||
on the field's **meaning** — a key that is or ends in `acct`/`webId`, case-insensitively — so
|
||||
flattened spellings are covered and unwritten shapes fail closed.
|
||||
- **The `houses` field rules were dead config.** Neither `getIdoc` nor `getHouses` projected, so the
|
||||
panel offered toggles that did nothing. **Every feature's declared fields must name the keys the
|
||||
read model actually emits**, not just the wire frame's.
|
||||
- **`/feed` filtered on `PUBLIC_KINDS`**, a module-load constant derived from the compiled defaults,
|
||||
so live audience changes never reached it. `visibleKinds(level, config)` resolves the readable set
|
||||
from live config; it deliberately ignores the `stream` flag, which governs SSE fan-out only (market
|
||||
history stays readable with its firehose off).
|
||||
- **`shardEvents.db.list` treated an empty `kinds` array as "no filter"** and fell through to an
|
||||
unfiltered `SELECT`. A fully-gated config would have dumped the whole event log, staff audit
|
||||
included. An empty allowlist now serves nothing.
|
||||
- **`projectValue` recursed into every object**, so a `Date` column came back as `{}`. It walks
|
||||
arrays and plain objects only. The unit tests used JSON fixtures and could not have caught this —
|
||||
the live read did, which is the argument for §11's smoke test over tests alone.
|
||||
|
||||
**The rule this leaves behind:** *a read path that returns shard data and does not call
|
||||
`projectFeature` is a bug.* Every new surface in Parts B and C — `/ruleset`, `/points`, `/market`,
|
||||
`/atlas` — must project, and must gate its kind set on live config rather than on `PUBLIC_KINDS`.
|
||||
|
||||
### 3.7 Admin surface
|
||||
|
||||
`GET` / `PUT /api/v1/admin/shard/visibility` (admin-only). Validate feature names against the known
|
||||
set and rungs against the ladder; reject any attempt to set a locked field below `admin` — including
|
||||
its flattened spellings (`ownerAcct`, `leaderWebId`), see §3.6.1. Writes an
|
||||
`admin.audit`-style row so visibility changes are traceable. New client panel
|
||||
`routes/admin/ShardVisibility.jsx` at `/admin/shard-visibility`, linked from `ShardAdmin.jsx`.
|
||||
|
||||
---
|
||||
|
||||
## 4. The version bump and the rollout
|
||||
|
||||
`PROTOCOL_VERSION` **2 → 3** in `link/sidecar/src/main.rs:27`. v3 is defined as *"adds
|
||||
`world.ruleset`, `points.board`, `vendor.listing` / `vendor.listing.remove`"*.
|
||||
|
||||
A bump is an operator-visible hard cutover — `web.rs::gate` returns 409 on every protected route on
|
||||
mismatch, `uoLinkSocket.js::handleHello` closes the WS, and the website's declared version is the
|
||||
admin-set `uo_link_config.protocol` column — so it happens **exactly once**, at the end:
|
||||
|
||||
- Cut an **`edge`** branch from `main` in each of `website/`, `link/`, `servuo-plugins/`, `docs/`.
|
||||
- Every phase PRs into `edge`, never `main`. Feature branches are cut from `edge`.
|
||||
- Part A lands first, alone.
|
||||
- When all parts are built and tested, one `edge` → `main` PR per repo, merged together. **That merge
|
||||
is the v3 cutover.**
|
||||
- A schema migration sets `uo_link_config.protocol` (the existing row **and** the column default)
|
||||
from 2 to 3, so the cutover doesn't require a manual admin edit. `UOLINK_PROTOCOL` still overrides.
|
||||
- No feature-negotiation array anywhere — v3 implies all three kinds.
|
||||
|
||||
---
|
||||
|
||||
## 5. Part B/1 — `world.ruleset` ✅ Done
|
||||
|
||||
*Landed on `edge`: servuo-plugins [#3](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/3), link [#17](https://gitea.whitlocktech.com/RunicGateway/link/pulls/17), website [#111](https://gitea.whitlocktech.com/RunicGateway/website/pulls/111), docs [#66](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/66). Implementation notes worth keeping:*
|
||||
|
||||
- ***`shadowguard` is derived, not configured.*** `Shadowguard.cfg` carries only `ReadyDuration` and
|
||||
`RandomizeInstances` — there is no `Enabled` key — so the systems block reports `Core.TOL`
|
||||
(the expansion gate) instead. Same shape for `factions`: `Factions.cfg` has no `Enabled` either, and
|
||||
`Services/Factions/Core/Faction.cs` sets `Settings.Enabled = !ViceVsVirtueSystem.Enabled`, so the
|
||||
frame reads that static rather than inventing a key. **Where a system's on/off state is derived, read
|
||||
the system's own static; only read `Config.Get` where the .cfg key IS the truth.**
|
||||
- **`caps.skill` / `caps.totalSkill` are in tenths** (1000 = 100.0), the way ServUO stores them.
|
||||
Documented in `INTEGRATION.md` and converted in the client, because the raw number is actively
|
||||
misleading rather than merely unhelpful.
|
||||
- **`Config.Get` re-parses when the cached type differs.** `InternalGet<T>` caches the parsed value on
|
||||
the entry and re-parses if `entry.Object is T` fails, so reading `PlayerCaps.SkillCap` as an `int`
|
||||
where ServUO reads it as a `double` is correct (both parse) — it just re-parses. Harmless, but worth
|
||||
knowing before assuming a shared cache.
|
||||
- **The plugin CAN be compile-verified**, contrary to "no standalone build": point Roslyn
|
||||
(`dotnet sdk/*/Roslyn/bincore/csc.dll`, `/langversion:7.3`, net48 reference assemblies) at the whole
|
||||
ServUO `Scripts` tree with `overlay/Scripts/Custom/Bridge/*.cs` substituted for the deployed copy,
|
||||
excluding `Scripts/obj` and `Scripts/bin`. 6,205 files, ~40 s, and it catches every signature error
|
||||
a boot would. Worth doing before every plugin PR.
|
||||
|
||||
`PROTOCOL_2.md` §10.4 sketches a `world.systems` capability frame that was never implemented
|
||||
(`grep` returns nothing across all four repos). **`world.ruleset` subsumes it**, carrying a `systems`
|
||||
sub-object with the `cityLoyalty` / `vvv` / `factions` booleans §10.4 asked for. §10.4 is marked
|
||||
superseded; no orphan kind is left behind.
|
||||
|
||||
### 5.1 Plugin
|
||||
|
||||
NEW `servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeRuleset.cs`, modelled on
|
||||
`BridgeBoot.EmitHello` — **not** a sweep. Subscribes `BridgeLink.Connected_Core += Emit` so a sidecar
|
||||
that comes up second still learns the ruleset.
|
||||
|
||||
Built from an **explicit allowlist** of `Server.Config.Get<T>` calls. **Never enumerate
|
||||
`Config.Entries`** (`Server/Config.cs:162`) — it would sweep in secrets. An FNV-1a `rev` over the body
|
||||
makes an unchanged reconnect a site-side no-op (`String.GetHashCode()` is not stable across runs and
|
||||
must not be used).
|
||||
|
||||
`BridgeConfig.cs` + `overlay/Config/Bridge.cfg`: `RulesetEnabled=true`, `PublicConnectAddress=""`,
|
||||
`RulesetIncludeSchedule=true`. `BridgeBoot.cs`: `reload` → re-emit, `status` → rev/bytes. Not wired
|
||||
to `sweepnow`; it isn't a sweep.
|
||||
|
||||
### 5.2 Payload
|
||||
|
||||
Every block optional, omitted when its system is off:
|
||||
|
||||
`shard`, `expansion`, `connect` (only from `PublicConnectAddress`),
|
||||
`systems{cityLoyalty,vvv,factions,siege,chat,store,dailyRares,honesty,shadowguard,treasureMaps,vetRewards,testCenter}`,
|
||||
`caps{skill:1000,totalSkill:7000,stat:225,str/dex/int:125,strMax/dexMax/intMax:150}`,
|
||||
`housing{accountHouseLimit:1}`, `accounts{perIp:3,charSlots:7,autoCreate}`,
|
||||
`vetRewards{enabled,rewardIntervalDays:30}`,
|
||||
`loot{feluccaLuckBonus:1000,feluccaBudgetBonus:100,feluccaMaxProps:11}`,
|
||||
`vendors{restockDelayMinutes,maxSell,economyStockAmount}`,
|
||||
`champions{powerScrolls:6,statScrolls:16,scrollChance,transcendenceChance,rankThresholds}`,
|
||||
`treasureMaps`, `vvv{enabled,startSilver:2000,enhancedRules}`, `store{enabled,currencyName}`,
|
||||
`schedule{autoSaveFrequencyMinutes,autoRestart*}`.
|
||||
|
||||
**Excluded by name — in a code comment and here:** `Server.cfg` (Address/Listen/Port; only
|
||||
`PublicConnectAddress` is published), `Staff.cfg`, `Email.cfg`, `DataPath.cfg`, `Bridge.cfg`,
|
||||
`Compiler.cfg`, `Reports.cfg`, `Client.cfg`.
|
||||
|
||||
### 5.3 Sidecar and website
|
||||
|
||||
Sidecar — `store.rs`: singleton `ruleset(id CHECK(id=1), rev, json, updated_t)` + upsert/get;
|
||||
`main.rs`: new arm in the board-projection match; `web.rs`: `GET /ruleset` served from the store, so
|
||||
it answers during a shard outage (`PROTOCOL_2.md` §12.2).
|
||||
|
||||
Website — `uoLinkClient.getRuleset()`; `uoLinkSocket.backfill()` (object-shaped, so follow the
|
||||
`getPresence()` block's explicit form, not the array-only `snapshot()` helper); `shardIngest.js` →
|
||||
`shardState.setRuleset`, **not** in `LOGGED_KINDS` (it re-arrives every reconnect and `server.hello`
|
||||
already marks those); `KIND_FEATURE['world.ruleset'] = 'ruleset'`; `shard_ruleset` singleton table
|
||||
(`rev`, `expansion`, `payload JSON`, `t`); `GET /public/shard/ruleset` behind
|
||||
`requireFeature('ruleset')`, returning `null` ⇒ "not published yet".
|
||||
|
||||
Client — NEW `routes/public/Rules.jsx` at `/site/rules`, alongside
|
||||
`/site/champs|guilds|governors|houses`; live via `useShardFeed({ filter: new Set(['world.ruleset']) })`.
|
||||
|
||||
### 5.4 Risk
|
||||
|
||||
Perf is nil (~3 KB per connect). The only real risk is publishing a secret, mitigated by the explicit
|
||||
allowlist, the no-`Config.Entries` rule, the named exclusion list, and a manual eyeball of the emitted
|
||||
frame during verification.
|
||||
|
||||
---
|
||||
|
||||
## 6. Part C — Spawn atlas / bestiary (website-only)
|
||||
|
||||
**No plugin, no sidecar, no `Bridge.cfg` knob, no new kinds.** Not part of the v3 wire change.
|
||||
|
||||
> **Status:** complete on `edge` — website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112)
|
||||
> (parsers, import CLI, tables) and [#113](https://gitea.whitlocktech.com/RunicGateway/website/pulls/113)
|
||||
> (the six public routes, the five admin ones, `/site/atlas` + `/site/atlas/:slug`, and the
|
||||
> Admin → Spawn Atlas panel).
|
||||
> Part C ships as **two** website PRs, not one: the parsing half is where the correctness risk
|
||||
> lives, and burying it under routes and React would have meant reviewing it in a 10k-line diff.
|
||||
> Full operator documentation: [`docs/website/SPAWN_ATLAS.md`](../website/SPAWN_ATLAS.md).
|
||||
>
|
||||
> **§6 below is the original design and is partly superseded.** §6.1 records two decisions that were
|
||||
> rejected in review and replaced (the committed artifact, and the fixed facet list); §6.2 records
|
||||
> the corrections the real ServUO data forced. Read both before trusting §6.
|
||||
|
||||
**Decision (revised at implementation time): the shard's ServUO tree is the single source of truth,
|
||||
re-derived on every server boot.** The original plan here was a committed generated artifact plus an
|
||||
idempotent import. That was rejected in review for two reasons, recorded in §6.1: a snapshot in the
|
||||
repo goes stale as a shard's maps change, and the design leaned on a fixed facet list that no shard
|
||||
is obliged to keep. Still not a browser-served blob; still parsed server-side only.
|
||||
|
||||
New in `website/server/`:
|
||||
|
||||
- `src/utils/spawnAtlasParse.js` — **pure functions, no fs**, so they are unit-testable in CI without
|
||||
a ServUO tree: `parseObjects2()`, `parsePoints()`, `parseRegions()`, `parseLocations()`,
|
||||
`resolveRegion()`.
|
||||
- ~~`scripts/buildSpawnAtlas.js` and a committed `db/data/spawnAtlas.*.json` artifact~~ — dropped,
|
||||
see §6.1 R1. Replaced by `src/utils/spawnAtlasSource.js` (the only thing that reads a ServUO tree,
|
||||
shared by the boot path and the CLI) and a `scripts/importSpawnAtlas.js` that is a thin CLI over
|
||||
the model. `package.json` gains `atlas:import` only.
|
||||
- `src/model/shardAtlas/{shardAtlas.db.js,shardAtlas.model.js}` following the `shardState` split.
|
||||
- `src/router/v1/public/atlas.{router,controller}.js`; `test/spawnAtlas.parse.test.js`.
|
||||
|
||||
Two parsing notes that matter:
|
||||
|
||||
- `<Objects2>` is `Type:MX=n:SB=…` segments joined by `:OBJ=` — verified against `trammel.xml`, where
|
||||
a single point carries six types. Split on `:OBJ=`; the token before the first `:` is the type.
|
||||
- **The high-value transform:** point-in-rect each spawn against the facet's `Regions.xml` rects
|
||||
(highest `priority` wins), falling back to the nearest `Data/Locations` landmark, else
|
||||
`"Wilderness"`. This is what turns *"lizardman at 5411,1234"* into ***"Despise, Felucca"*** and is
|
||||
the entire reason the page is worth building. `Regions.xml` is genuinely nested and needs a ~120-line
|
||||
recursive tokenizer **or** one devDependency (`fast-xml-parser`) — the server has zero XML deps
|
||||
today, so that is an explicit call to make at implementation time. The flat `<Points>` files need
|
||||
only regex/streaming; **do not** put 10.5 MB through a DOM parser.
|
||||
|
||||
Tables: `shard_spawn_creatures` (slug PK, name, total, facets JSON), `shard_spawn_points` (slug,
|
||||
facet, x, y, region, landmark, max_count, tod_*), `shard_regions`, `shard_landmarks`,
|
||||
`shard_champion_spawns`, `shard_atlas_meta`. Plain `INDEX` on name, **not `FULLTEXT`** — ~1,500
|
||||
creature rows makes a `LIKE` scan free, and FULLTEXT brings min-token-length trouble for names like
|
||||
"orc". No FKs, consistent with every existing `shard_*` table.
|
||||
|
||||
Routes at `/api/v1/public/atlas`, **not** under `/shard` — the atlas is static shard *content*, not
|
||||
live shard *state*; it must not look sidecar-dependent, and unlike `/shard/*` it *should* be
|
||||
`siteMode`-gated like `/posts` and `/wiki`. `GET /creatures?q=&facet=`, `/creatures/:slug`,
|
||||
`/regions`, `/landmarks`, `/champions`, `/meta`, all behind `requireFeature('atlas')`. Admin:
|
||||
`GET /admin/shard/atlas/status` (artifact-vs-DB drift) and `POST /admin/shard/atlas/import`. **Build
|
||||
stays CLI-only.**
|
||||
|
||||
Client: `routes/public/Atlas.jsx` (`/site/atlas`) and `AtlasCreature.jsx` (`/site/atlas/:slug`).
|
||||
|
||||
**Payload risk** — *superseded by §6.1 R1; nothing is committed.* The field selection it describes
|
||||
still applies at parse time: every `<Points>` field the site cannot use (`UniqueId`, all
|
||||
trigger/refractory/proximity/sequential fields, sound ids) is dropped, keeping
|
||||
Name/Map/X/Y/W/H/Range/MaxCount/MinDelay/MaxDelay/TOD*/types. Parsed data never reaches the browser;
|
||||
the browser sees only paginated API responses.
|
||||
|
||||
**Operator re-run story** — *revised by §6.1 R1.* Spawns changed → restart, or
|
||||
`npm run atlas:import` / `POST /admin/shard/atlas/import` to apply without one. `shard_atlas_meta`
|
||||
holds a sha256 per source file, so the server can tell on boot whether anything changed, and
|
||||
`GET /admin/shard/atlas/status` reports drift. If the change would remove a facet it is staged for
|
||||
approval rather than applied (§6.1 R3). Full detail in `docs/website/SPAWN_ATLAS.md`.
|
||||
|
||||
### 6.1 What implementation changed
|
||||
|
||||
Two design decisions in §6 were rejected in review and replaced; the rest are corrections the real
|
||||
ServUO data forced. Kept as a diff rather than edited in place, because each is a trap the next
|
||||
person would otherwise re-enter.
|
||||
|
||||
**R1. The committed artifact is gone — the tree is re-parsed on every boot.** §6 proposed building a
|
||||
generated artifact, committing it, and importing it. Two problems. A shard's maps change over its
|
||||
life, so a snapshot in the repo silently drifts from the world players actually see; and the build/
|
||||
import split existed only to work around the website container not having a tree, which is a
|
||||
deployment question (mount it) rather than a reason to freeze data. The server now hashes the source
|
||||
files on boot and re-derives the atlas when they differ. `scripts/buildSpawnAtlas.js`, the 1.41 MB
|
||||
artifact, and the whole encode/decode seam it needed are deleted.
|
||||
|
||||
**R2. Nothing may name a facet.** The first implementation carried a lookup table of the six stock
|
||||
UO facets to reconcile the spelling drift between sources. A shard may add facets, replace them
|
||||
outright, or rename them when its maps are updated, and a built-in list mishandles all three
|
||||
silently. Reconciliation is now by *matching* against the facet set discovered from the shard's own
|
||||
spawn and region data — exact key, then prefix in either direction — with an unmatched name keeping
|
||||
its own rather than being forced into a wrong bucket.
|
||||
|
||||
**R3. Two contracts on the boot path.** It never blocks startup: no path, an unreadable mount, a
|
||||
malformed file or a database error is caught and logged, and the site comes up serving whatever
|
||||
atlas it had. And a refresh that would REMOVE a facet is never applied automatically — facet loss
|
||||
is indistinguishable at boot from a half-copied or mid-update tree, so it is staged in
|
||||
`shard_atlas_pending` for an admin to approve or reject. Only the decision is stored (source hashes
|
||||
+ the facet diff, a few KB); approving re-parses, so what lands matches the tree at approval time.
|
||||
A rejection is remembered against those hashes so it does not re-prompt every restart.
|
||||
|
||||
### 6.2 What the build against real data changed
|
||||
|
||||
Six corrections to the design above, from running it against stock ServUO 57.4. Kept as a diff
|
||||
rather than edited in place, because each one is a trap the next person would otherwise re-enter.
|
||||
|
||||
**1. Six facets, not thirteen.** The design said `spawnAtlas.<facet>.json ×13`, assuming one facet
|
||||
per spawn file. There are 13 files but only **6** facets — `Eodon.xml`, `GravewaterLake.xml`,
|
||||
`TreasuresOfKotl.xml` and the other named-area files carry TerMur/Trammel points. The facet comes
|
||||
from each record's own `<Map>`, never the file name, and the artifact shards 6 ways.
|
||||
|
||||
**2. The XML dependency call: hand-rolled, zero deps.** §6 left `fast-xml-parser` vs a ~120-line
|
||||
tokenizer open. Resolved as the tokenizer — a deliberate *subset* parser covering only what these
|
||||
files use. The server keeps zero XML dependencies at any tier.
|
||||
|
||||
**3. Facet names disagree between sources — a silent failure.** `Data/Locations/*.xml` spells them
|
||||
`Ter Mur` and `Tokuno Islands`; `<Map>` and `<Facet name>` say `TerMur` and `Tokuno`. Unreconciled,
|
||||
the landmark bucket is keyed differently from the points looking it up, so the fallback never fires
|
||||
and **every unregioned spawn in Ter Mur and Tokuno reads "Wilderness"** — a plausible-looking atlas
|
||||
that is quietly wrong for two facets. All facet names now pass through `normalizeFacet()`.
|
||||
|
||||
**4. Spawn type tokens carry XmlSpawner directives.** `<Objects2>` types are not always bare class
|
||||
names: `Fairy,{RND,4,8}`, `alchemist/z/-50`, `Agralem/Name/Agralem`, `greatape,true`. Taken literally
|
||||
they invent creatures that do not exist *and* split real ones in two, since `Fairy` and
|
||||
`Fairy,{RND,4,8}` slug apart. 71 of 845 entries were affected; stripping at the first `/` or `,`
|
||||
leaves **800** real creatures. (The design's "~1,500 creature rows" estimate was high; 800 only
|
||||
reinforces the plain-`INDEX`-not-`FULLTEXT` call.)
|
||||
|
||||
**5. The artifact would have been 1.41 MB, not "well under 1 MB" — and is now moot.** Dropping the
|
||||
unused `<Points>` fields as the design directed still left 4.40 MB; three further encodings brought
|
||||
it to 1.41 MB, and getting under 1 MB would have meant dropping the spawner `name`. The size budget
|
||||
in §6 was simply optimistic for 6,455 points. Superseded by §6.1 R1: there is no artifact, so there
|
||||
is no payload to budget and no encode/decode seam to keep in sync.
|
||||
|
||||
**6. `DELETE`, not `TRUNCATE`.** The design said "TRUNCATE + batched INSERT in one transaction",
|
||||
which does not hold: `TRUNCATE` is DDL in MariaDB and implicitly commits, so a mid-import failure
|
||||
would leave the atlas half-loaded. `DELETE` is transactional, and at ~7k rows the cost is
|
||||
irrelevant. Point ids are also assigned explicitly rather than by `AUTO_INCREMENT`, because the
|
||||
join rows need them and `conn.batch()` reports no usable `insertId`.
|
||||
|
||||
**Measured result:** 6,455 points, 800 creatures, 23,927 point/type rows, 387 regions, 558
|
||||
landmarks, 25 champion altars. The placement transform resolves **83.2%** of points (3,689 by
|
||||
region, 1,690 by landmark, 1,086 Wilderness).
|
||||
|
||||
**One thing the design got exactly right:** the point-in-rect transform really is the reason to
|
||||
build this. "Where does a lizardman spawn?" answers *Shrines, Isamu-Jima, Yew* across three facets.
|
||||
|
||||
### 6.3 What the API/client half added
|
||||
|
||||
The second website PR built the six public routes, the five admin ones, `/site/atlas` +
|
||||
`/site/atlas/:slug`, and the Admin → Spawn Atlas panel. Three things it changed or established:
|
||||
|
||||
**1. Respawn delays were being read in the wrong unit — sometimes.** XmlSpawner writes
|
||||
`MinDelay`/`MaxDelay` in minutes and switches to seconds only when a delay does not divide into
|
||||
whole minutes, flagging that per record with `DelayInSec`
|
||||
(`XmlSpawner2.cs:7462-7480`, read back at `:6345-6358`). So a `5` means five *minutes* on one
|
||||
spawner and five *seconds* on the next, both plausible, and the pipeline stored the raw number.
|
||||
170 of 6,455 stock spawners are second-flagged — few enough to look like noise on a page and be
|
||||
believed. The parser now normalises to **seconds**, and the API and UI carry seconds throughout.
|
||||
*This is the class of bug §6.2 is a list of: the atlas still builds, it is just quietly wrong.*
|
||||
|
||||
**2. The hash gate needed a parser version, and this generalises.** Fixing the parse exposed that
|
||||
"has the tree changed?" is the wrong question on its own — an install whose maps never change would
|
||||
have kept serving the old readings forever, because the only thing compared was the tree.
|
||||
`spawnAtlasSource.PARSER_VERSION` is stored in `shard_atlas_meta` beside the source hashes, and a
|
||||
mismatch counts as drift. Any future parse correction lands on the next boot without an operator
|
||||
having to know it happened. **Bump it whenever the parser derives different data from identical
|
||||
files.**
|
||||
|
||||
**3. `points` is a count; `spawners` is the list.** The first cut of the detail route spread the
|
||||
creature row and then set `points` to the array of spawn points — the same key meaning a number on
|
||||
the search route and an array on the detail route. Renamed before it shipped, and worth recording
|
||||
because the two names are one letter apart in meaning and it reads as correct.
|
||||
|
||||
**On projection.** The `atlas` feature declares no sensitive fields, so `projectFeature` is a no-op
|
||||
on every one of these routes today. Every handler calls it anyway, per §3.6.1's rule — the point of
|
||||
the rule is that the *first* field that needs gating is covered by construction rather than by a
|
||||
retrofit nobody remembers to do.
|
||||
|
||||
---
|
||||
|
||||
## 7. Part B/2 — `points.board` ✅ Done
|
||||
|
||||
*Landed on `edge`: servuo-plugins [#4](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/4),
|
||||
link [#18](https://gitea.whitlocktech.com/RunicGateway/link/pulls/18), website [#114](https://gitea.whitlocktech.com/RunicGateway/website/pulls/114),
|
||||
docs [#69](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/69). Verified against the real ServUO tree
|
||||
per §11 — see §7.5 for what that run changed.*
|
||||
|
||||
Two deliverables: a diff sweep for the boards, and a `points` block folded into `char.profile` —
|
||||
the `PROTOCOL_2.md` §10.3 `titles` precedent (read-model enrichment, no new request kind).
|
||||
|
||||
### 7.1 Plugin
|
||||
|
||||
NEW `BridgePoints.cs`, copying the `BridgeHousing.cs` diff-sweep shape (`Initialize` →
|
||||
`ServerStarted`, `Connected_Core += OnConnected` clearing `_last` + `Rearm()`, `SweepOnce()`,
|
||||
`Status()`, skip when `!BridgeLink.Connected`, try/catch throughout).
|
||||
|
||||
Which systems: default to `PointsSystem.Systems` filtered to `ShowOnLoyaltyGump == true` — reuse the
|
||||
shard's own "this is player-facing" signal rather than inventing one. `Bridge.cfg PointsSystems=`
|
||||
overrides. Null-guard `PointsSystem.Systems`; it is a mutable static populated by 25 separate
|
||||
subsystem constructors.
|
||||
|
||||
**The perf trap.** `PlayerTable` is a plain `List<PointsEntry>`, and `QueensLoyalty` has `AutoAdd`, so
|
||||
it can hold an entry for every `PlayerMobile` that ever existed. A naive
|
||||
`.OrderByDescending().Take(N)` across 25 systems is 25 full sorts — at 20,000 historical characters,
|
||||
~7.5 M comparisons, tens of ms on the Core thread. `BRIDGE_PLUGIN_PLAN.md` §1 found that nothing
|
||||
except bulk profile generation comes close to a frame budget; this would be the second thing that
|
||||
does.
|
||||
|
||||
**Mitigation — single-pass bounded selection** into a fixed N-element sorted array (N=10): O(n·N) with
|
||||
tiny constants and one allocation. Skip `Player == null || Deleted` and `Points <= 0`. ~500 k cheap
|
||||
iterations at a 300 s interval.
|
||||
|
||||
Diff signature per system: `concat(serial + ":" + (long)points)` over the top N, plus the entry count.
|
||||
**No `points.remove`** — the system set is fixed, the same argument `city.update` already uses.
|
||||
|
||||
### 7.2 Payload — one frame per system
|
||||
|
||||
25 × ~600 B rather than one 12 KB frame, matching `champ.update` / `guild.update`:
|
||||
|
||||
```jsonc
|
||||
{"t":…,"kind":"points.board","system":"QueensLoyalty",
|
||||
"nameString":"Queen's Loyalty","nameNumber":1114938,
|
||||
"maxPoints":30000,"showOnGump":true,"players":842,
|
||||
"top":[{"rank":1,"serial":"0x1A2B","name":"Darrow","points":29500}, …]}
|
||||
```
|
||||
|
||||
`nameString` **and** `nameNumber` are both emitted (a `TextDefinition` may be a cliloc), resolved
|
||||
website-side — the contract `titles.reward` already documents at `BridgeProfile.cs:107-110`.
|
||||
|
||||
**Entries are written inline as `{serial, name}` — never via `BridgeJson.Actor`.** Deliberate even
|
||||
though the website can now reveal fields by rung: `acct`/`webId` are not needed here, because the
|
||||
website resolves serial→user from its own `shard_account_links` mirror for staff views. Keep the wire
|
||||
minimal.
|
||||
|
||||
### 7.3 `char.profile` enrichment
|
||||
|
||||
`BridgeProfile.cs` gains `WritePoints(sb, m)` alongside `WriteTitles`:
|
||||
`"points":[{system,nameString,points,maxPoints}]`, omitting systems with no entry or 0 points.
|
||||
|
||||
**Deliberately no `rank`** — computing it means scanning each system's `PlayerTable` once per profile
|
||||
(25 × n), which would dominate the measured 0.069 ms/profile budget. The website derives rank from
|
||||
the board when the character appears in the top N. Gate behind `PointsProfileRank=false` if it is
|
||||
ever wanted.
|
||||
|
||||
### 7.4 Config, sidecar, website
|
||||
|
||||
`Bridge.cfg`: `PointsSweepSeconds=300`, `PointsLeaderboardEnabled=true`, `PointsTopN=10`,
|
||||
`PointsSystems=` (blank ⇒ auto), `PointsProfileEnabled=true`, `PointsProfileRank=false`.
|
||||
`BridgeBoot.cs`: `Rearm()` in `reload`, `SweepOnce()` in `sweepnow`, `Status()` in both.
|
||||
|
||||
Sidecar — `points_boards(system PK, name, json, updated_t)`; `main.rs` arm keyed on `system`;
|
||||
`GET /points` and `GET /points/:system`.
|
||||
|
||||
Website — `shard_points_boards(system PK, name, name_cliloc, max_points, players, show_on_gump,
|
||||
payload JSON, t)`. **The top-N list stays in `payload`** — a fixed-size list read whole, exactly like
|
||||
`shard_governors.candidates`. Do not normalize into a `shard_points_entries` table until a
|
||||
per-character reverse lookup is actually needed. `shardIngest.js` → `upsertPointsBoard`, **not** in
|
||||
`LOGGED_KINDS` (board state, like `guild.update`). `KIND_FEATURE['points.board'] = 'leaderboards'`,
|
||||
with `characterName` as its per-field rule. `GET /public/shard/points` and `/points/:system` behind
|
||||
`requireFeature('leaderboards')`; validate `system` ≤ 48 chars.
|
||||
|
||||
**No new player route** — per-character points ride inside `char.profile`, already served by
|
||||
`GET /player/shard/char/:serial` with its `shardLinks.ownsAccount` check.
|
||||
|
||||
Client — NEW `routes/public/Leaderboards.jsx` at `/site/leaderboards`; a "Loyalty & Points" section
|
||||
added to `components/CharacterSheet.jsx`, one edit serving both `PlayerCharacter.jsx` and
|
||||
`AdminCharacter.jsx`.
|
||||
|
||||
### 7.5 What the run against a real shard changed
|
||||
|
||||
The plan above was written from reading `PointsSystem.cs`. Booting the actual shard (ServUO 57.4, a
|
||||
43,011-mobile world) and letting one sweep run corrected four things — all of them invisible to a
|
||||
fake-shard test, because a fake shard emits whatever the spec says it should.
|
||||
|
||||
1. **`maxPoints` overflowed to `long.MinValue`.** `MaxPoints` is a `double`, and ServUO's idiom for an
|
||||
uncapped system is `double.MaxValue` — which `DespiseCrystals`, `ShameCrystals` and `VoidPool` all
|
||||
use. `(long)double.MaxValue` in C# is an **unchecked** conversion: it does not throw, it yields
|
||||
`long.MinValue`, and the first real sweep published
|
||||
`"maxPoints": -9223372036854775808` for three of the five live boards. Fixed with `Cap()` /
|
||||
`Score()` converters that normalise anything unrepresentable to `0`, which is now the wire's
|
||||
documented **"uncapped"** value. Worth stating plainly because it inverts the obvious reading:
|
||||
**on a real shard, `maxPoints: 0` is the common case, not an edge case**, so any UI dividing by it
|
||||
must special-case it.
|
||||
2. **`nameString` is usually `null`.** Most systems define their `Name` as a cliloc rather than a
|
||||
literal: four of the five boards on the live shard came back `nameString: null` with only
|
||||
`nameNumber` set. The humanise-the-`system`-key fallback is therefore the *primary* display path,
|
||||
not a defensive nicety, and both the leaderboards page and the character sheet lead with it.
|
||||
3. **`GetEntry`/`GetPoints` cannot be used in the read model.** `GetEntry(from, create: false)` still
|
||||
calls `AddEntry` when the system has `AutoAdd` (`PointsSystem.cs:207`) — it **mutates the world**.
|
||||
Ten of the ~25 systems have `AutoAdd = true`, so a profile built with the obvious accessor would
|
||||
have appended up to ten rows to the points save file every time anyone viewed a character sheet.
|
||||
`BridgeProfile.WritePoints` hand-rolls a read-only scan instead, and says so loudly.
|
||||
4. **`players` had to be redefined.** §7.2 called for "the entry count", but those same ten `AutoAdd`
|
||||
systems hold a zero-point row per character ever created — so the raw count reports the shard's
|
||||
whole census as one system's participants. It is now the number of players actually holding points,
|
||||
which is both the honest number and a strictly better diff signal (it moves when someone scores,
|
||||
not when someone logs in for the first time).
|
||||
|
||||
One deviation from the plan as written, for the same class of reason: §7.4 named the per-field
|
||||
visibility rule `characterName`, but `projectValue` matches on the **literal JSON key**, and the wire
|
||||
key is `name`. A rule under the descriptive name would have been silently inert — an admin tightening
|
||||
character names would have got no enforcement and no error, exactly the failure §3.6.1 records for the
|
||||
flattened `ownerAcct`. `FEATURES.leaderboards.fields` therefore keys on `name`, with a test that fails
|
||||
if it is renamed back.
|
||||
|
||||
---
|
||||
|
||||
## 8. Part B/3 — `vendor.listing` 🟨 In review
|
||||
|
||||
### 8.1 It cannot be an RPC, and this is load-bearing
|
||||
|
||||
`rpc.rs::try_route` correlates on the **first** frame carrying a matching `reqId` and resolves a
|
||||
single `oneshot`. A chunked reply sharing one `reqId` would deliver chunk 1 to the HTTP caller and
|
||||
**leak chunks 2..N onto the broadcast feed**. `REPLY_TIMEOUT` is 10 s (the client waits 12 s), so a
|
||||
whole-world snapshot could not fit regardless.
|
||||
|
||||
⇒ **a per-vendor diff sweep on the broadcast stream**, like `champ.update` / `house.update`. The
|
||||
existing per-account `vendor.snapshot` RPC is untouched; the player portal keeps using it.
|
||||
|
||||
Kinds: `vendor.listing` (one frame per vendor, authoritative for that vendor) and
|
||||
`vendor.listing.remove`. Payload: `serial, shopName, owner:{serial,name}, map, x, y, region, house,
|
||||
count, truncated, items:[{serial,itemId,hue,amount,price,name,cliloc,child}]`.
|
||||
|
||||
### 8.2 Two perf traps
|
||||
|
||||
Measured baseline (`BRIDGE_PLUGIN_PLAN.md` §1): 30 vendors / 1,200 listings = 0.343 ms via
|
||||
`pack.Items` + `GetVendorItem`; extrapolated to 500 vendors / 40,000 listings ≈ 12 ms per full pass.
|
||||
Except:
|
||||
|
||||
1. **`VendorSearch.GetItemName(Item)` is a packet builder, not a field read.** It constructs an
|
||||
`ObjectPropertyList`, calls `GetProperties`, serialises, then byte-parses the packet
|
||||
(`VendorSearch.cs:681-789`) — per item. Across 40,000 items in one tick that is a
|
||||
multi-hundred-millisecond stall. **Mandatory: never call it in the sweep.** Emit `itemId`, `hue`,
|
||||
`amount`, `price`, `item.Name` (the plain field, null for most) and `item.LabelNumber`, resolving
|
||||
display names website-side — exactly what `char.profile.equipment` already does
|
||||
(`BridgeProfile.cs:173`).
|
||||
2. **`VendorSearch.GetItems(PlayerVendor)` is private** (`:791`). The reusable public API is
|
||||
`GetItems(Container, List<Item>)` (`:807`), which recurses into sub-containers, so real item counts
|
||||
run above the top-level `pack.Items` the 0.343 ms measurement used. Budget accordingly.
|
||||
|
||||
### 8.3 Mitigations
|
||||
|
||||
- **Amortized round-robin sweep** — `MarketSweepSeconds=60`, at most `MarketSweepBatch=25` vendors per
|
||||
tick, with a persistent cursor over `PlayerVendor.PlayerVendors`. Full coverage in
|
||||
`ceil(vendors/25) × 60 s`, with **per-tick cost bounded independent of world size**. This is the one
|
||||
genuinely new pattern versus the existing sweeps and should be flagged in review.
|
||||
- **Per-vendor signature diff** (`count | Σ(serial ^ price) | x | y | shopName`), as `BridgeHousing`
|
||||
does — most vendors are static, so steady-state emission is near zero.
|
||||
- **`MarketMaxListings=250`**, then `"truncated":true`. `BridgeJson.Parse` caps *inbound* at 1 MB;
|
||||
outbound is uncapped and `shard.rs::read_line` will allocate whatever arrives.
|
||||
- On `Connected_Core`, clear `_last` **and reset the cursor**; the re-emit is self-throttled by the
|
||||
round-robin window.
|
||||
|
||||
### 8.4 Player opt-out and privacy
|
||||
|
||||
**Honour `pv.VendorSearch`** — ServUO's own per-vendor opt-out, which `DoSearch` filters on (`:62`).
|
||||
Skip opted-out vendors entirely; the seen-set removal then drops them from the board, so **a player
|
||||
who hid their vendor in game is hidden on the website too.** Also skip `Map == null || Map.Internal`
|
||||
and `Backpack == null`, matching `DoSearch`.
|
||||
|
||||
A vendor's shop name, owner character name and location are **already globally visible in-game** — the
|
||||
stock Vendor Search gump surfaces exactly this set to any player — which is why they default to
|
||||
`anonymous`. They remain per-field configurable (`ownerName`, `location`) so an admin can tighten
|
||||
them. Account name and website user id never go on the wire.
|
||||
|
||||
### 8.5 Sidecar and website
|
||||
|
||||
Sidecar — one table `vendors(serial PK, shop_name, owner_name, map, x, y, region, count, json,
|
||||
updated_t)` storing the whole-vendor blob. **No `vendor_items` table** — the sidecar's job here is
|
||||
outage resilience (`PROTOCOL_2.md` §12.2), not search; search lives in MariaDB. Endpoint is
|
||||
**`GET /market`**, not `/vendors` — axum would route the latter fine, but the collision with the
|
||||
per-account RPC is a readability trap.
|
||||
|
||||
Website — `shard_vendors` + `shard_vendor_items` (indexes on `vendor_serial`, `price`, `item_id`,
|
||||
`display_name`; delete-then-insert per vendor in one transaction; no FKs). `shardIngest.js` handles
|
||||
both kinds; **not** in `LOGGED_KINDS`.
|
||||
|
||||
`KIND_FEATURE['vendor.listing'] = 'market'`, but the market feature's **SSE mapping is disabled by
|
||||
default**: a live firehose of full vendor inventories would be the site's single biggest bandwidth
|
||||
consumer, and no page needs it live. The page is a paginated DB query with a staleness stamp; an
|
||||
admin can turn the stream on. `uoLinkSocket` paginates `/market` on reconnect, bounded by
|
||||
`MARKET_SNAPSHOT_MAX = 5000` vendors so a pathological world cannot hang startup.
|
||||
|
||||
`GET /public/shard/market?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=`
|
||||
(limit 1..100, default 50; `q` ≤ 60 chars; `sort ∈ {price_asc, price_desc, recent}`) and
|
||||
`/market/vendors/:serial`, behind `requireFeature('market')`. **Rate-limit it** — this is the first
|
||||
genuinely expensive public endpoint; `express-rate-limit` is already a dependency.
|
||||
|
||||
### 8.6 The open dependency — cliloc names ✅ Resolved (shipped ahead of §8)
|
||||
|
||||
`CharacterSheet.jsx:14-15` documented the gap ("without a cliloc table on the site we can only
|
||||
show literals") and rendered equipment as `id {itemId}`. Search-by-name needs that table.
|
||||
|
||||
**Resolved as its own website-only change, landed BEFORE the market so `/site/market` ships with real
|
||||
item names.** Full design and operator guide: [`docs/website/CLILOCS.md`](../website/CLILOCS.md).
|
||||
Ingest denormalizes the resolved name into `shard_vendor_items.display_name` as planned.
|
||||
|
||||
Two things in the original recommendation above turned out to be wrong, and both are worth recording
|
||||
because the reasoning generalises.
|
||||
|
||||
**1. The committed `db/data/clilocs.json` artifact was dropped.** It predates the two Part C
|
||||
corrections (§6.1) and violates both: no committed snapshot of derived content, and nothing
|
||||
EA-derived ever shipped. UO's strings are EA's, exactly as the creature sprites are. Replaced with
|
||||
the §6 pattern instead — parse on every boot from an operator-configured path, hash-gated, output
|
||||
gitignored, `PARSER_VERSION` counted as drift.
|
||||
|
||||
**2. `scripts/buildClilocs.js reads the UO client's Cliloc.enu` is not possible, and the reason
|
||||
matters.** **Every current client ships its cliloc files COMPRESSED** — all eight `Cliloc.*` files
|
||||
open with a DWORD whose high byte is `0x8E`, the "Mythic" container. The plain layout (`02 00 00 00
|
||||
01 00`, then `{int32 number, byte flag, uint16 length, UTF-8}`) is what those files looked like
|
||||
*before* that change. Parsing a compressed file as plain does not fail cleanly: it yields ~19k
|
||||
"records" with negative ids, 1,722 distinct keys out of 19,508, one 62 KB "string", and a truncation
|
||||
somewhere in the middle.
|
||||
|
||||
Decompressing means porting an inverse-BWT coder with a 1 KB frequency header — a few hundred lines
|
||||
whose failure mode is plausible-looking garbage rather than an error. Two facts closed off the
|
||||
alternatives:
|
||||
|
||||
- **ServUO cannot read it either.** Its bundled `Ultima.StringList` implements only the plain layout,
|
||||
so on a modern client `VendorSearch.StringList` is null and `VendorSearch.GetItemName` returns
|
||||
`item.Name`. **The in-game Vendor Search gump has the same gap** — which also means §8.2's warning
|
||||
never to call `GetItemName` in the sweep costs us nothing we could otherwise have had.
|
||||
- The shard therefore cannot supply names on our behalf, so this could not be pushed to the plugin.
|
||||
|
||||
⇒ **the operator converts once, from their own client, and the site reads the result.** Accepted
|
||||
shapes are the plain binary layout and a `number<TAB|,|;>text` export; the site sniffs which.
|
||||
`server/tools/cliloc-export/` drives UOFiddler's `Ultima.dll` (the decompressor that already exists)
|
||||
and writes the plain form. A shard that never converts is fully supported — names render as ids,
|
||||
exactly as before.
|
||||
|
||||
**Shards edit items and add new ones**, and those carry ids no stock client table has — so this reads
|
||||
a **set** of sources, not one file, hash-gated together and re-read on every boot exactly as §6 reads
|
||||
the ServUO tree: a base (the converted client table) plus every overlay under `custom/`, later
|
||||
winning. Adding one custom item therefore never means re-exporting a 5 MB client file. Measured on
|
||||
the live shard for scale: its script tree references **16,434** cliloc ids and only **37** are absent
|
||||
from stock — tens against a 67k base, which is why an overlay and not a second table. `custom/` is the
|
||||
one convention here that is ours rather than the shard's, because **ServUO has no server-side notion
|
||||
of a custom cliloc**: they live in the patched client a shard distributes, and nothing in the tree
|
||||
declares them.
|
||||
|
||||
That set also brings back a hazard a single file did not have, and §8.6 answers it the way §6 does. A
|
||||
corrupt source fails the parse loudly, but a source that has **vanished** parses perfectly and imports
|
||||
a table quietly missing everything it contributed — an unmounted volume is indistinguishable from a
|
||||
deliberate deletion. So it is **staged, not applied** (`status: 'needsReview'`), reported by both the
|
||||
import and `status()`, and accepted with `{approve:true}`. It is a flag rather than §6's
|
||||
approve/reject pair because the atlas stores a pending decision *so that approving re-parses*; here
|
||||
nothing is stored, so re-reading at approval time is automatic.
|
||||
|
||||
Five traps found by building it, all recorded in `CLILOCS.md`:
|
||||
|
||||
- **`StringList.SaveStringList` RE-COMPRESSES on save.** It looks exactly like the export path and is
|
||||
not; its output is byte-identical to its compressed input, because its purpose is round-tripping a
|
||||
file back into the client.
|
||||
- **Trimming a text line before splitting silently drops half the table.** Roughly half of a real
|
||||
cliloc table is empty strings (ids the client reserves), exported as `1005008<TAB>`. Trimming eats
|
||||
the trailing separator, leaving a bare number that then looks like a header row — 55,994 of 123,490
|
||||
entries vanished, and the import still looked successful.
|
||||
- **`Number('')` is `0`, not `NaN`.** A line starting with a separator imports as a bogus cliloc 0
|
||||
unless the empty field is rejected explicitly.
|
||||
- **Tidying punctuation unconditionally corrupts real names.** Stripping leftover brackets is right
|
||||
after a placeholder is removed (`[~1_stuff~]` → nothing) and wrong otherwise: a shard's custom
|
||||
`"Runic Gateway Sigil (v2)"` rendered as `"(v2"`. Same shape as the `%` rule. **Found only by
|
||||
running a shard-style overlay through it** — every stock-table fixture passed.
|
||||
- **Source labels must be forward-slashed and root-relative**, or the same directory fingerprints
|
||||
differently on Windows and Linux and every boot looks like a change. The identical bug §6 records.
|
||||
|
||||
Blank entries are dropped at import (123,490 parsed → **67,496** stored), which also makes the binary
|
||||
and text paths converge on identical content.
|
||||
|
||||
### 8.7 Client
|
||||
|
||||
`routes/public/Market.jsx` at `/site/market`, with a *"prices last refreshed N minutes ago"* banner
|
||||
driven by `staleAt` (the oldest `shard_vendors.updated_at`). The round-robin sweep means data is
|
||||
inherently up to one full cycle old, and the UI must say so.
|
||||
|
||||
Shipped with a second page, `routes/public/MarketVendor.jsx` at `/site/market/vendors/:serial` —
|
||||
where a search result points. It is the only surface that can render the two states the result list
|
||||
cannot: a `truncated` shop (*"showing 250 of 3,104 — this shop holds more than the shard
|
||||
publishes"*) and a `location` an admin has gated away, which is a real answer rather than an empty
|
||||
coordinate.
|
||||
|
||||
### 8.8 What the build changed
|
||||
|
||||
Four things the implementation settled differently from §8 as written, all of them found by building
|
||||
against the live shard.
|
||||
|
||||
**1. `location` is a nested object, not flat `map`/`x`/`y`/`region`.** §8.1's payload sketch had them
|
||||
flat, and it would have made `market.location` — a rule Part A pre-wired — **inert**, exactly like
|
||||
the `characterName` miss §7.5 records: `projectValue` matches literal JSON keys, so there is no
|
||||
`location` key for the rule to match. Flat keys would have needed five rules that could drift apart.
|
||||
Nesting makes one rule hide the facet, the coordinates, the region and the house together, on the
|
||||
live frame and the stored read model alike, because both now spell it the same way.
|
||||
|
||||
The other pre-wired rule, `market.ownerName`, checked out — it is a real key on the frame. Owner is
|
||||
written as flat `ownerSerial`/`ownerName` rather than through `BridgeJson.Actor`, which would add
|
||||
`acct` and `webId`; same argument `points.board` makes. `ownerSerial` was **added** to the
|
||||
configurable fields alongside `ownerName`, because an admin who hides the owner's name and leaves a
|
||||
serial every other board resolves back to that name has not hidden anything.
|
||||
|
||||
**2. The per-vendor diff signature is the full listing set, not §8.3's `count | Σ(serial ^ price)`.**
|
||||
That hash collides on the single most common change a shop makes: two items swapping prices, which
|
||||
is what re-pricing looks like. The signature is built over the same buffer the frame is written
|
||||
from, in the same order, so a match really does mean an identical frame.
|
||||
|
||||
**3. There is no `payload` column on `shard_vendors`.** §8.5 implied the board pattern (whole frame
|
||||
in JSON, columns hoisted for display). It does not apply here: the items ARE the searchable rows, so
|
||||
they are normalized into `shard_vendor_items` and there is nothing left worth duplicating. The
|
||||
sidecar keeps the whole blob, because outage resilience is its job and search is not.
|
||||
|
||||
**4. Sweep cost is reported, and a slow tick warns.** The batch cap is a *claim* about per-tick cost,
|
||||
and an operator tuning `MarketSweepBatch` was otherwise tuning blind. `[bridge status` now carries
|
||||
`lastMs`/`maxMs`, and a tick over 50 ms prints a rate-limited warning naming the knob.
|
||||
|
||||
Measured on the live shard (27 vendors × 40 listings, 209k items / 43k mobiles):
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| First tick — 25 vendors emitted cold | **15.4 ms** |
|
||||
| Second tick — the remaining 2 | **3.4 ms** |
|
||||
| Steady state — nothing changed | **0.3 ms** |
|
||||
| Website `/market` search over 1,040 listings | 1,040 total, names resolved |
|
||||
| Cliloc re-resolution pass over 1,040 rows | **50 ms** |
|
||||
|
||||
The diff is what makes the steady state ~free; the batch cap is what bounds the cold case. Note the
|
||||
arithmetic the warning exists for: at the default cap of 250 listings, a batch of 25 **full** shops
|
||||
is 6,250 items ≈ 95 ms — over budget. Real shops hold tens, which is why 25 is the default, but a
|
||||
shard of commodity resellers should lower the batch, and now it will be told to.
|
||||
|
||||
Two smaller things worth not rediscovering:
|
||||
|
||||
- **`BridgeJson.Escape` takes a NON-NULL string** — it dereferences `value.Length` immediately — and
|
||||
`BridgeJson.Str` writes its own `,"key":` prefix, so neither serves a value inside a hand-built
|
||||
object. Nearly everything this frame writes is legitimately null (an item's plain `Name` is null
|
||||
for almost every item; a vendor in the street has no house), so that is the common path, not an
|
||||
edge case. `BridgeMarket.Text()` is the two-line writer that was missing.
|
||||
- **The ServUO console writes in the OS code page**, so an em dash in a `Console.WriteLine` renders
|
||||
as `???` in the log an operator would paste into an issue. Bridge console output is ASCII.
|
||||
|
||||
Search-side, one thing the site had to fix rather than inherit: `%` and `_` in a user's query are
|
||||
**LIKE** metacharacters, not SQL ones, so parameterization does not neutralize them — a search for
|
||||
`%` would otherwise match every listing on the shard. `shardMarket.db.js` escapes them. (The atlas's
|
||||
`LIKE` searches predate this and have the same shape over a much smaller table; worth a follow-up,
|
||||
not a blocker here.)
|
||||
|
||||
---
|
||||
|
||||
## 9. Sequencing
|
||||
|
||||
| Order | Part | Repos touched | Wire change | State |
|
||||
|---|---|---|---|---|
|
||||
| 1 | **A** — visibility framework + actor-leak fix | website, docs | none | ✅ Done |
|
||||
| 2 | **B/1** — `world.ruleset` (§5) | all four | new kind | ✅ Done |
|
||||
| 3 | **C** — spawn atlas (§6) | website, docs | none | ✅ Done |
|
||||
| 4 | **B/2** — `points.board` (§7) | all four | new kind + `char.profile` field | ✅ Done |
|
||||
| 5a | **B/3 dependency** — cliloc table (§8.6) | website, docs | none | ✅ Done |
|
||||
| 5b | **B/3** — `vendor.listing` (§8) | all four | new kinds | 🟨 In review |
|
||||
| 6 | **Cutover** — `PROTOCOL_VERSION` 2→3, `edge` → `main` | all four | the bump | ⬜ |
|
||||
|
||||
---
|
||||
|
||||
## 10. Documentation obligations
|
||||
|
||||
- This file (`link/v3.md`) is the canonical 3.0 design.
|
||||
- `PROTOCOL_2.md` §10.4 gains a note that `world.systems` is superseded by `world.ruleset`, and that
|
||||
the deferred VvV question is answered (`VvV.cfg Enabled=True`, Factions off).
|
||||
- `INTEGRATION.md` — catalog entries and §6 consumer sections for each new kind, plus the v2→v3
|
||||
upgrade note for operators.
|
||||
- `PLAN.md` — phasing.
|
||||
- `website/BACKEND_DESIGN.md` — every new table and route, and **the visibility framework as a
|
||||
security contract**: the audience ladder, the two locked rules, and the fail-closed kind map belong
|
||||
in the security section.
|
||||
- NEW `website/SHARD_VISIBILITY.md` — admin-facing: what each feature exposes, what each rung means,
|
||||
what cannot be loosened.
|
||||
- NEW `website/SPAWN_ATLAS.md`, NEW `website/CLILOCS.md`, NEW `website/MARKETPLACE.md`.
|
||||
- `PROJECT_TREE.md` in each touched repo.
|
||||
- `npm run swagger` **and** `npm run routes:manifest` on every route-touching PR — both are committed
|
||||
artifacts, and `test/routeManifest.test.js` fails on drift.
|
||||
|
||||
**Follow-up, not scoped for 3.0:** the Android app consumes the same public/player shard API and will
|
||||
need `/public/shard/features` to hide its own nav. Track separately against `android-app/`.
|
||||
|
||||
---
|
||||
|
||||
## 11. Verification
|
||||
|
||||
**Plugin** — `servuo-plugins\deploy.ps1 -ServerPath <servuo> -Verify`, inspect the ADD/CHANGE list,
|
||||
then re-run without `-Verify` (ServUO must be stopped). Boot with `tools/stub_sidecar.ps1` listening
|
||||
and **confirm the compile banner in the console, not merely the absence of errors** —
|
||||
`BRIDGE_PLUGIN_PLAN.md` §1 warns that a failing build is silently ignored and the previous
|
||||
`Scripts.dll` reloads. Then `[bridge status`, `[bridge sweepnow`, `[bridge reload`.
|
||||
|
||||
- §5: eyeball the emitted `world.ruleset` frame for anything sourced from `Server.cfg`, `Staff.cfg`,
|
||||
`Email.cfg`, `DataPath.cfg` or `Bridge.cfg`.
|
||||
- §8: with a seeded world, time one sweep tick and confirm the batch cap holds it under ~1 ms.
|
||||
|
||||
**Sidecar** — `cargo build && cargo clippy`; `curl -H "Authorization: Bearer <token>"
|
||||
localhost:8080/ruleset` (and `/points`, `/market`); confirm `X-UOLink-Version: 3` and that a client
|
||||
declaring 2 receives a 409.
|
||||
|
||||
**Website server** — `DB_HOST=127.0.0.1 DB_PORT=59999 node --test`. New tests, each modelled on an
|
||||
existing sibling: `test/shardVisibility.test.js`, `test/shardBroadcast.visibility.test.js`,
|
||||
`test/shardIngest.{ruleset,points,market}.test.js` (after `shardIngest.protocol2.test.js` — stubbed
|
||||
deps, asserting routing and `logged` flags), `test/spawnAtlas.parse.test.js` (pure functions, inline
|
||||
fixtures). Then `npm run routes:manifest` and `npm run swagger`, committing both.
|
||||
|
||||
**Full stack** — against a local MariaDB: apply `db/schema.sql` (idempotent), start the server,
|
||||
confirm `uoLinkSocket` backfill logs the new snapshot lines and that `uo_link_config.protocol`
|
||||
migrated to 3, then load `/site/rules`, `/site/atlas`, `/site/leaderboards`, `/site/market`.
|
||||
|
||||
**Visibility smoke test** — for each of the five rungs, walk every shard page and confirm gating and
|
||||
field projection match the configured matrix. Same shape as the 200-routes × 5-access-levels sweep
|
||||
already run for the domain split.
|
||||
|
||||
---
|
||||
|
||||
## 12. Critical files
|
||||
|
||||
| File | Why |
|
||||
|---|---|
|
||||
| `website/server/src/utils/shardBroadcast.js` | The security boundary; reworked from a static allowlist to per-connection audience filtering. **The highest-risk file in 3.0.** |
|
||||
| `website/server/src/utils/shardVisibility.js` (new) | Ladder, kind→feature map, projection |
|
||||
| `website/server/src/utils/shardIngest.js` | The dispatcher every new kind routes through |
|
||||
| `website/server/src/model/shardState/shardState.model.js` | The `shape*` projections, including the `shapeGuild` leak §3.1 fixes |
|
||||
| `servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeHousing.cs` | Cleanest copy of the diff-sweep pattern; template for `BridgePoints.cs` and `BridgeMarket.cs` |
|
||||
| `link/sidecar/src/main.rs` | `PROTOCOL_VERSION` 2→3 and the board-projection match |
|
||||
| `website/server/db/schema.sql` | All new `shard_*` tables plus the `uo_link_config.protocol` migration |
|
||||
@@ -1,731 +0,0 @@
|
||||
# Website API — router domain split + CSP hardening
|
||||
|
||||
Status: **domain split complete** — PR 0 (route manifest), CSP report-only and split PRs 1–5 have all
|
||||
landed; only the CSP enforce PR remains, and it is blocked on soak data rather than on code ·
|
||||
Target repo: `website/` · Docs owner: this file + `BACKEND_DESIGN.md`
|
||||
|
||||
> **This file replaces the earlier "API v2" plan** (auth merge → CSP → domain split, with a parallel
|
||||
> `/api/v2` mount and an `/api/mobile` facade). Three of those four pieces are **not being built**:
|
||||
> the auth merge and the mobile facade are deferred with their reasoning recorded below, and the
|
||||
> parallel-version scaffold in [API_V2_SKELETON.md](./API_V2_SKELETON.md) is superseded. The filename
|
||||
> is kept so existing links resolve. What remains is genuinely useful work:
|
||||
>
|
||||
> 1. **CSP hardening** — small, independent, ships on its own cadence.
|
||||
> 2. **The domain split** — `admin.routes.js` (1552 lines, 110 routes) broken into one router file per
|
||||
> business capability, **in place, with every URL unchanged**. This is the actual driver.
|
||||
|
||||
---
|
||||
|
||||
## Why the auth merge is out
|
||||
|
||||
The original plan replaced httpOnly session cookies with a bearer JWT + rotating refresh token for
|
||||
every client, so web and mobile would share one session model. Reasons that no longer hold up:
|
||||
|
||||
1. **The current model is the more secure one.** httpOnly + SameSite cookies are unreadable from JS
|
||||
and carry CSRF protection by default. Every migration target is a sideways or backwards move:
|
||||
- Refresh token in `localStorage` → any XSS becomes **persistent full account takeover**, not a
|
||||
bounded access-token window. A short access TTL does not help; the attacker mints new pairs.
|
||||
- Refresh token in an httpOnly cookie scoped to the refresh endpoint → safe, but that is
|
||||
*cookies with extra steps*. It concedes the premise.
|
||||
2. **"One session model everywhere" is already true where it matters.** `auth/session.service.js`
|
||||
unifies cookie and bearer into a single session, and `auth/token.js` already extracts from either
|
||||
`Cookie` or `Authorization: Bearer`. That abstraction is written, working, and paid for. The merge
|
||||
would move complexity *out* of `extractToken` and *into* the SPA.
|
||||
3. **The cookie codepath survives the merge anyway.** The SSO / email-connect redirect flow must keep
|
||||
its short-lived httpOnly tx / PKCE-verifier / pending-TOTP cookies — the browser leaves for the
|
||||
IdP and returns with no JS context. So the merge never actually delivered "cookies are gone."
|
||||
4. **It carried the plan's most bug-prone work as a dependency.** The admin SSE rewrite
|
||||
(`EventSource` → hand-rolled `fetch` + `ReadableStream` + SSE frame parser + reconnect/backoff +
|
||||
refresh-on-401) existed *only* to serve the bearer model. Without the merge, the admin stream stays
|
||||
on `EventSource` with `withCredentials` and that code is never written.
|
||||
5. **It is a contract change with no user-visible payoff**, competing for the same review attention as
|
||||
the domain split, which is the thing that actually hurts today.
|
||||
|
||||
### Dropped with it
|
||||
|
||||
- v2 bearer auth routes (`/api/v2/auth/{login,refresh,logout,login/totp}`).
|
||||
- Deletion of the `setAuthCookie` / `clearAuthCookie` path.
|
||||
- `rg_trust` cookie → `X-Trust-Token` header migration for web (the header stays available for native
|
||||
clients via `extractTrustToken`, unchanged).
|
||||
- `api/client.js` bearer + silent-refresh rewrite.
|
||||
- `lib/useShardFeed.js` admin-stream fetch rewrite. **The admin SSE stream stays as-is.**
|
||||
|
||||
### Kept from it
|
||||
|
||||
- **CSP hardening** — now its own phase (below). It was justified as a compensating control for a
|
||||
JS-held token; it is worth doing regardless, just no longer urgent.
|
||||
- **The public/admin SSE allowlist split** — unchanged security boundary, unrelated to session model.
|
||||
- **The tx-cookie carve-out reasoning** — recorded here so a future merge attempt doesn't rediscover it.
|
||||
|
||||
---
|
||||
|
||||
## Deferred: the auth merge
|
||||
|
||||
Not cancelled — parked behind trigger conditions. Revisit if **any** of these become true:
|
||||
|
||||
| Trigger | Why it changes the answer |
|
||||
|---|---|
|
||||
| The API becomes genuinely cross-origin (separate API host) | `SameSite` cookies stop being the easy path; bearer becomes the natural model. |
|
||||
| Third-party or OAuth clients are introduced | Cookies don't serve clients you don't control. |
|
||||
| Mobile and web session behavior diverge enough to cause real bugs | The unification argument gets teeth it currently lacks. |
|
||||
|
||||
If it is ever revived, two specs the original plan lacked must be written **first**:
|
||||
|
||||
- **Refresh-token reuse detection.** Rotation is only useful with it: replay of an already-consumed
|
||||
refresh must revoke the entire token family, not just fail the one request.
|
||||
- **Rollback procedure.** Once web clients have discarded their cookies, a bad deploy locks everyone
|
||||
out. Needs a documented path back.
|
||||
|
||||
---
|
||||
|
||||
## Why there is no `/api/v2`
|
||||
|
||||
The domain split reorganizes router *files*. It does not need to move a single URL — because the URL
|
||||
surface is **already grouped by capability**. Inventory taken from the live Express stack — 196
|
||||
`/api/v1` routes, plus three outside it (`GET /api/health`, `GET /api/docs.json`,
|
||||
`GET /.well-known/assetlinks.json`) and 2 on the internal port. Full machine-readable list:
|
||||
[`api-route-inventory.json`](./api-route-inventory.json).
|
||||
|
||||
| Group | Routes | Second segment → capability |
|
||||
|---|---|---|
|
||||
| `/admin` | 110 | `shard` 16 · `moderation` 15 · `users` 15 · `wiki` 14 · `posts` 9 · `pages` 7 · `account` 6 · `email` 6 · `uo-link` 5 · `auth` 4 · `invites` 3 · `bot-activity` 2 · `discord-bot` 2 · `settings` 2 · `activity` 1 · `dashboard` 1 · `site-mode` 1 · `uploads` 1 |
|
||||
| `/auth` | 42 | `me` 23 · `mobile` 5 · `sso` 4 · `password` 3 · `invite` 2 · `login` 2 · `logout` 1 · `providers` 1 · `register` 1 |
|
||||
| `/public` | 24 | `shard` 12 · `wiki` 4 · `pages` 2 · `posts` 2 · `contact` 1 · `settings` 1 · `status` 1 · `version` 1 |
|
||||
| `/player` | 20 | `account` 8 · `shard` 8 · `appeals` 4 |
|
||||
|
||||
Every capability already owns a URL prefix, so each new router file mounts at the prefix it already
|
||||
owns and the emitted paths are **byte-identical**. No URL change means no contract change, and no
|
||||
contract change means no reason to mount a parallel version.
|
||||
|
||||
Consequences of doing it in place:
|
||||
|
||||
- No `/api/v2`, no dual mount, no route-by-route migration, no v1-usage telemetry project, and no
|
||||
v1-retirement sequence.
|
||||
- `BASE = /api/v1` in `client/src/api/client.js` never changes. The Discord bot's `SITE_PUBLIC_URL`
|
||||
never changes. The Android app is untouched.
|
||||
- [API_V2_SKELETON.md](./API_V2_SKELETON.md) (the `router/v2/` scaffold) is **superseded and not
|
||||
scheduled**. It is kept as the concrete recipe if a real contract break ever forces a versioned API.
|
||||
|
||||
**Tripwire:** if any endpoint turns out to *need* a new URL, that is a contract change, not a
|
||||
refactor. List it explicitly, and reopen the versioning question before writing the code — do not
|
||||
smuggle a URL change into a "mechanical" PR.
|
||||
|
||||
---
|
||||
|
||||
## Deferred: the `/api/mobile` facade and the app-version floor
|
||||
|
||||
The earlier plan's Phase 0 stood up a version-agnostic `/api/mobile` namespace and migrated the
|
||||
Android app onto it, plus an app-version header and a server-side min-version floor.
|
||||
|
||||
**Why it was proposed:** the app hardcodes **69** distinct `api/v1/…` paths (`data/api/*.kt`,
|
||||
`core/net/ShardStreamClient.kt`, `core/net/HostSelectionInterceptor.kt`,
|
||||
`core/auth/sso/SsoAuthManager.kt`), has no version negotiation and no force-update, and installs in
|
||||
the wild cannot be forced forward. Under a parallel-`/api/v2` plan that made the app the load-bearing
|
||||
coupling: v1 could not be retired until the fleet aged out.
|
||||
|
||||
**Why it is deferred:** with the split done in place, no URL moves and nothing is being deleted — so
|
||||
there is no fleet to sunset and no coupling to break. A facade would add ~70 permanently maintained
|
||||
delegate routes plus a contract-test suite to solve a problem that does not currently exist. The
|
||||
version floor was scoped to sunsetting the pre-facade fleet, so it goes with it.
|
||||
|
||||
**If it is ever revived** (the trigger is the mobile contract genuinely needing to diverge from web —
|
||||
different response shapes, a mobile-only aggregation endpoint, a real breaking change):
|
||||
|
||||
- **Start with the alias mount, not a delegate layer:** `apiRouter.use('/mobile', v1Router)` gives the
|
||||
app a stable, version-agnostic namespace with identical wiring, identical middleware and zero
|
||||
per-route maintenance. Build hand-written delegates only for the routes that actually diverge.
|
||||
- **A facade is a security surface, not a convenience alias.** Any hand-written route must carry the
|
||||
*same* middleware chain as the route it mirrors (`requireAuth`, `staffOnly`/`adminOnly`, validators,
|
||||
the public/admin SSE allowlist split). A re-exposed admin route missing `adminOnly` is privilege
|
||||
escalation.
|
||||
- **It needs contract tests.** The moment the app pins a namespace, its response shapes are a
|
||||
committed contract; an internal refactor that changes a shape must fail a test before it ships to
|
||||
installed apps.
|
||||
- **The mobile SSE stream stays anonymous** under whatever path it gets — the app sends no
|
||||
`Authorization` header.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — CSP hardening (independent)
|
||||
|
||||
Previously bundled with the auth merge as a compensating control for a JS-held token. With no token in
|
||||
JS, this is **defense in depth on its own merits** — cheap, worth doing, blocking nothing. It has no
|
||||
dependency on the domain split and can ship at any time.
|
||||
|
||||
**Sequencing fix from the original plan:** the old version both "ships with the auth merge" and called
|
||||
for a one-release report-only soak. Those contradict. Correct order is **report-only first, observe one
|
||||
release, then enforce** — now trivially satisfiable since nothing waits on it.
|
||||
|
||||
The app already ships a tuned policy (`server/src/app.js`). Two directives are load-bearing and are
|
||||
**already correct** — the job is to keep them that way:
|
||||
|
||||
- `script-src 'self'` — no `'unsafe-inline'` / `'unsafe-eval'`. Primary defense.
|
||||
- `connect-src 'self'` — the exfiltration channel. Don't widen it unless the API genuinely becomes
|
||||
cross-origin (which would also reopen the auth-merge question — see the trigger table).
|
||||
|
||||
`style-src 'unsafe-inline'` stays — it permits inline styling, not script execution, and React's
|
||||
pervasive `style={{…}}` attributes can't be nonce'd. Not a meaningful hole. The `/api/docs` route keeps
|
||||
its deliberately looser policy (swagger-ui injects an inline bootstrap script); that carve-out is
|
||||
scoped to the one route and stays scoped.
|
||||
|
||||
Target enforced policy:
|
||||
|
||||
```
|
||||
default-src 'self';
|
||||
script-src 'self';
|
||||
connect-src 'self';
|
||||
img-src 'self' data: https:;
|
||||
style-src 'self' 'unsafe-inline'; /* + fonts.googleapis.com only until fonts are self-hosted */
|
||||
font-src 'self'; /* + fonts.gstatic.com only until fonts are self-hosted */
|
||||
object-src 'none';
|
||||
base-uri 'self';
|
||||
form-action 'self';
|
||||
frame-ancestors 'none';
|
||||
```
|
||||
|
||||
Delta vs. the policy in `server/src/app.js` today. This was written as two directives; on
|
||||
implementation it turned out to be **one**:
|
||||
|
||||
- ~~**Add `form-action 'self'`** (currently absent)~~ — **it was not absent.** The directives object in
|
||||
`app.js` does not list it, but the middleware is configured `useDefaults: true`, and helmet's default
|
||||
set already supplies `form-action 'self'` — so the header served in production has carried it all
|
||||
along. Verified by capturing the live `Content-Security-Policy` header from the running app rather
|
||||
than reading the config, which is how the plan got this wrong. **No behavioural change here.** It is
|
||||
now written out explicitly in `config/csp.js` anyway: a security directive should not depend on a
|
||||
third-party library's defaults surviving its next major version.
|
||||
- **Tighten `frame-ancestors`** `'self'` → `'none'` — nothing legitimately frames the site. **This is
|
||||
the entire behavioural delta of the phase.**
|
||||
- Unchanged: `default-src`, `script-src`, `connect-src`, `object-src 'none'`, `base-uri 'self'`, and
|
||||
`img-src … https:` (external `BRAND_*` logo/hero and `<img>` in sanitized wiki/news bodies rely on
|
||||
`https:`).
|
||||
|
||||
The soak is still worth running for that one directive, and arguably it is the directive that most
|
||||
needs one: a `frame-ancestors` report is generated by the browser of *whoever framed the site*, so it
|
||||
is the only way to discover that something legitimately embeds us before the enforcing policy breaks
|
||||
it. Nothing else can tell us that.
|
||||
|
||||
**Rollout:** ship via `Content-Security-Policy-Report-Only` with `report-to` for one release, watch for
|
||||
violations, then flip to enforce.
|
||||
|
||||
Before trusting `script-src 'self'`: Vite's build injects an inline modulepreload-polyfill `<script>`
|
||||
into `dist/index.html`, which that directive blocks (harmless, but throws a violation).
|
||||
**Already handled** — `client/vite.config.js` sets `modulePreload: { polyfill: false }`, so the build
|
||||
emits no inline bootstrap script. (The `renderIndexHtml` branding injection adds only `<meta>`/`<link>`
|
||||
tags — no inline script, no nonce needed.)
|
||||
|
||||
### Where reports go
|
||||
|
||||
`report-to` needs somewhere to point, so the report-only PR stands up a same-origin sink:
|
||||
**`POST /api/csp-report`** (`server/src/router/cspReport.controller.js`, wired in `app.js`). Same-origin
|
||||
on purpose — violation reports describe attacks against this site and are not handed to a third-party
|
||||
collector. It writes to the `csp` log tag and stores nothing.
|
||||
|
||||
It is mounted outside `/api/v1`, alongside `/api/health`: the browser learns the path from the policy
|
||||
header, never from a client build, so it is not part of the versioned client contract. This is the
|
||||
`+1` in the route manifest that made PR 0 go first (see § Sequencing).
|
||||
|
||||
Necessary properties, since it is an unauthenticated public `POST` (browsers send reports with no
|
||||
session, and gating it would silence exactly the anonymous visitors worth hearing about):
|
||||
|
||||
- **Both wire formats.** `report-uri` (Firefox, Safari) sends `application/csp-report` with a single
|
||||
hyphenated-key object; `report-to` (Chrome) sends `application/reports+json` with an array of
|
||||
camelCase envelopes. Handling one silently drops half the browsers. Both directives are emitted, and
|
||||
`report-to` additionally needs a `Reporting-Endpoints` response header or it is inert.
|
||||
- **Always 204, even for junk.** A 4xx would make the global error handler write an ERROR line quoting
|
||||
the attacker-supplied body — turning an open endpoint into a log-flood primitive. A browser cannot
|
||||
act on an error from a report sink anyway.
|
||||
- **Bounded everywhere:** 16 KB body cap, a per-IP rate limit, a fixed field allowlist, and every
|
||||
logged field truncated (`script-sample` is attacker-influenced and can carry a whole inline script).
|
||||
|
||||
**Retiring it:** the sink exists for the soak. When the tightened policy flips to enforced and the
|
||||
report-only twin is deleted, this endpoint goes with it — *unless* a `report-to` group is deliberately
|
||||
kept on the enforced policy, which is a reasonable thing to want. Decide that in the enforce PR rather
|
||||
than leaving an orphan route behind.
|
||||
|
||||
Tracked follow-ups (own PRs):
|
||||
|
||||
- **Self-host the Cinzel font** → drop `fonts.googleapis.com` from `style-src` and `fonts.gstatic.com`
|
||||
from `font-src`, removing two third-party origins from the trust surface.
|
||||
- **Trusted Types** — `require-trusted-types-for 'script'` + a `trusted-types` policy, report-only
|
||||
first. Audit `dangerouslySetInnerHTML` + the `sanitizeHtml` render path first.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — The domain split
|
||||
|
||||
**This is the reason the plan exists.** Everything else is supporting work.
|
||||
|
||||
**Rule:** one router file = one business capability; the URL names the domain; related endpoints live
|
||||
together regardless of HTTP method; no generic `admin.routes.js` catch-all. Controllers are **already**
|
||||
domain-split — this re-wires routes, not logic.
|
||||
|
||||
**Invariant:** each capability router mounts at the prefix it already owns, so the emitted URL set does
|
||||
not change. Proved per PR by the route manifest (§ PR 0).
|
||||
|
||||
Target tree — derived from the inventory above, **inside `router/v1/`** (no `v2/` directory):
|
||||
|
||||
```
|
||||
router/v1/
|
||||
admin/
|
||||
index.js # mounts the capability routers below under /admin, keeps the
|
||||
# shared `noindex, isLoggedIn, staffOnly` gate in one place
|
||||
users.router.js account.router.js invites.router.js
|
||||
authProviders.router.js moderation.router.js botActivity.router.js
|
||||
posts.router.js pages.router.js wiki.router.js
|
||||
uploads.router.js shard.router.js uoLink.router.js
|
||||
email.router.js discordBot.router.js settings.router.js
|
||||
activity.router.js # the staff audit log — landed in PR 2, not with dashboard
|
||||
dashboard.router.js # + the /site-mode singleton
|
||||
auth/
|
||||
login.router.js register.router.js password.router.js invite.router.js
|
||||
sso.router.js mobile.router.js me.routes.js (already split, 23 routes)
|
||||
public/
|
||||
news.router.js (posts) pages.router.js wiki.router.js shard.router.js
|
||||
site.router.js # status, settings, version, contact
|
||||
player/
|
||||
account.router.js shard.router.js appeals.router.js
|
||||
internal/ (unchanged — stays on the unpublished port, never mounted publicly)
|
||||
```
|
||||
|
||||
Steps:
|
||||
|
||||
1. **Land PR 0 (route manifest) first** — the mechanical proof that later PRs move no URL.
|
||||
2. Carve `admin.routes.js` into the per-capability files above, each requiring its already-existing
|
||||
controller. `admin/index.js` keeps the shared gate (`noindex, isLoggedIn, staffOnly`) and mounts
|
||||
each capability router at its existing prefix; the `adminOnly` / `modAccess` gates move with the
|
||||
routes that use them.
|
||||
3. Split `public.routes.js`, `player.routes.js`, and the remaining `auth.routes.js` groups the same
|
||||
way. `auth/me.routes.js` is already a separate file and stays.
|
||||
4. Keep `/internal` off the public listener exactly as today (separate `internalApp.js` port).
|
||||
5. Move each route's `#swagger.*` annotations **with** the route, then regenerate
|
||||
(`cd website/server && npm run swagger`).
|
||||
6. Update `BACKEND_DESIGN.md` §2 (folder structure) and §4 (API contract — it names
|
||||
`admin.routes.js → admin.controller.js` and friends) as routers move, plus `PROJECT_TREE.md`.
|
||||
|
||||
Because the auth model is untouched and the URLs are frozen, each PR is a **pure mechanical refactor
|
||||
with a green test suite and a zero-diff route manifest as its acceptance criteria** — which is what
|
||||
makes grouped PRs actually reviewable.
|
||||
|
||||
> **Step 6 correction:** `PROJECT_TREE.md` is no longer hand-edited. Since website#98 it is
|
||||
> auto-generated by the `sync-project-tree` CI workflow, which opens its own docs PR after a merge to
|
||||
> `main`. Leave it alone in split PRs. `BACKEND_DESIGN.md` §2/§4 are still manual.
|
||||
|
||||
### PR 1 — as landed
|
||||
|
||||
`admin/index.js` owns the shared `noindex, isLoggedIn, staffOnly` gate and the mount table, and
|
||||
declares no routes itself. The gate sits **ahead of every mount**, so a capability router extracted in
|
||||
a later PR cannot silently ship without it. Route counts:
|
||||
|
||||
| Router | Routes | Prefix | Extra gate |
|
||||
|---|---|---|---|
|
||||
| `account.router.js` | 6 | `/admin/account` | none — self-service, an editor manages their own 2FA |
|
||||
| `users.router.js` | 15 | `/admin/users` | `adminOnly` at router level |
|
||||
| `invites.router.js` | 3 | `/admin/invites` | `adminOnly` per route |
|
||||
| `authProviders.router.js` | 4 | `/admin/auth` (routes are `/providers[/:id]`) | `adminOnly` per route |
|
||||
| `admin.routes.js` (residual) | 82 | group root, mounted last | unchanged |
|
||||
|
||||
6 + 15 + 3 + 4 + 82 = the 110 inventoried admin routes. None of the four prefixes appears in the
|
||||
residual file, so nothing depends on mount ordering.
|
||||
|
||||
Three findings worth carrying into PRs 2–5:
|
||||
|
||||
- **The self-service `/shard/*` routes stay with `shard` (PR 4), despite their `Admin · Account`
|
||||
swagger tag.** The invariant is *prefix ownership*, not tag agreement: one router owns `/shard`, so
|
||||
splitting six routes off it by capability would mean two routers mounting under the same prefix and
|
||||
an ordering hazard for no gain. Retag them in PR 4 if the tag still grates.
|
||||
- **`usersRouter.use(adminOnly)` is exactly equivalent to the old `adminRouter.use('/users', adminOnly)`**
|
||||
now that the router is mounted at `/users` — but only because it is mounted at a prefix. Under a
|
||||
*pathless* mount, a bare `use(gate)` would run for every request passing through en route to a later
|
||||
mount, 403-ing an editor on `/admin/posts`. Do not "simplify" a prefix mount away.
|
||||
- **`routes.guards.json` came back zero-diff too**, not just the manifest — no route lost or gained a
|
||||
gate. Worth checking both every time; the guards file is the one that would catch a dropped
|
||||
`adminOnly` that the method+path freeze cannot see.
|
||||
|
||||
The OpenAPI spec was also byte-for-byte unchanged, which required a prerequisite fix — see below.
|
||||
|
||||
### PR 2 — as landed
|
||||
|
||||
`moderation`, `bot-activity` and `activity` — 18 routes, leaving 64 in the residual file.
|
||||
|
||||
| Router | Routes | Prefix | Extra gate |
|
||||
|---|---|---|---|
|
||||
| `moderation.router.js` | 15 | `/admin/moderation` | `modAccess` (admin + moderator) at router level |
|
||||
| `botActivity.router.js` | 2 | `/admin/bot-activity` | `adminOnly` per route |
|
||||
| `activity.router.js` | 1 | `/admin/activity` | none — staff-wide audit log |
|
||||
| `admin.routes.js` (residual) | 64 | group root, mounted last | unchanged |
|
||||
|
||||
All four gates came back zero-diff: `routes.manifest.json` (200 public + 2 internal),
|
||||
`routes.guards.json`, `swagger-output.json` (198 operations), and `docs/website/api-route-inventory.json`
|
||||
was already in sync. 434 server tests green.
|
||||
|
||||
Notes:
|
||||
|
||||
- **`/activity` gets its own file, deviating from the target tree above**, which parked it as a
|
||||
singleton inside `dashboard.router.js`. It is folded into PR 2 by the sequencing list, and PR 4 is
|
||||
where `dashboard` lands — so honouring the tree would have meant leaving one route in the residual
|
||||
file for two PRs to satisfy a filename. It is also a genuinely separate capability: `/activity` is
|
||||
the **staff audit log** (`activity.model.js`), while `/dashboard` is a stats overview and
|
||||
`/bot-activity` is the botScore middleware's in-memory ban state. Three different things that read
|
||||
alike. **PR 4 mounts `dashboard` and `site-mode` only.**
|
||||
- **`modAccess` moved to a router-level `use`; `adminOnly` on bot-activity deliberately did not.**
|
||||
Moderation was already gated by a *prefix* mount (`adminRouter.use('/moderation', modAccess)`), so
|
||||
`moderationRouter.use(modAccess)` is the exact equivalent (the PR 1 `users` case). Bot-activity's
|
||||
gate was per-route, and keeping it per-route is what holds the per-route handler count — the one
|
||||
number in `routes.guards.json` that would catch a dropped `adminOnly`, since `requireRole(...)`
|
||||
returns an anonymous arrow and never shows up by name. **Rule for PRs 3–5: move a gate to router
|
||||
level only where it was already a prefix mount; otherwise leave it on the route.**
|
||||
- **`modAccess` stays in the residual file** — the `/shard/*` in-game staff operations still use it
|
||||
and do not move until PR 4. Its comment there was retargeted rather than deleted.
|
||||
|
||||
### PR 3 — as landed
|
||||
|
||||
`posts`, `uploads`, `wiki` and `pages` — 31 routes, leaving 33 in the residual file. The content tier,
|
||||
and the first split PR where no gate moved at all: all four capabilities are editor-tier, so the shared
|
||||
`staffOnly` in `admin/index.js` is their whole gate.
|
||||
|
||||
| Router | Routes | Prefix | Extra gate |
|
||||
|---|---|---|---|
|
||||
| `posts.router.js` | 9 | `/admin/posts` | none — editor tier |
|
||||
| `uploads.router.js` | 1 | `/admin/uploads` | none — editor tier |
|
||||
| `wiki.router.js` | 14 | `/admin/wiki` | none — editor tier |
|
||||
| `pages.router.js` | 7 | `/admin/pages` | none — editor tier |
|
||||
| `admin.routes.js` (residual) | 33 | group root, mounted last | unchanged |
|
||||
|
||||
All four gates zero-diff: `routes.manifest.json` (200 public + 2 internal), `routes.guards.json`,
|
||||
`swagger-output.json` (198 operations), and `docs/website/api-route-inventory.json` was already in
|
||||
sync. 434 server tests green.
|
||||
|
||||
Notes:
|
||||
|
||||
- **The residual 33 is exactly PR 4's list** — `shard` 16, `email` 6, `uo-link` 5, `settings` 2,
|
||||
`discord-bot` 2, `dashboard` 1, `site-mode` 1. So `admin.routes.js` is deleted by PR 4, one PR
|
||||
earlier than the sequencing list implies, and PR 5 touches only `public/*`, `player/*` and `auth/*`.
|
||||
- **A shared module was unavoidable here, and it is the first one in the split.** The multer config
|
||||
(upload dir, mimetype→extension allowlist, 8 MB cap) was defined inline in `admin.routes.js` and used
|
||||
by *two* routes that this PR puts in different files: `POST /posts/upload` (→ `{image_url}`) and
|
||||
`POST /uploads` (→ `{url}`). It moved to `admin/imageUpload.js` rather than being duplicated —
|
||||
duplicating a security allowlist is how the two copies drift. It stays in `admin/` deliberately:
|
||||
`UPLOAD_DIR` is resolved `__dirname`-relative, so relocating the file would silently repoint the
|
||||
upload directory. Guard freshness is unaffected — multer's middleware is named `multerMiddleware`
|
||||
wherever it is constructed, so `routes.guards.json` did not move.
|
||||
- **`POST /uploads` keeps its `Admin · Posts` swagger tag**, which now disagrees with its filename. The
|
||||
acceptance criterion is a byte-identical spec, so retagging is a real OpenAPI diff and does not
|
||||
belong in a route-move PR. Same call as PR 1's `/shard/*` tag mismatch: fix tags in a PR that is
|
||||
*about* tags.
|
||||
- **The wiki router is the first one with load-bearing intra-file route order.** `/categories` and
|
||||
`/tags` are literal paths that must stay ahead of `/:slug`, or `GET /admin/wiki/categories` gets
|
||||
dispatched as a page whose slug is "categories". **The manifest cannot catch this — it sorts its
|
||||
entries, so a reordering is invisible in all three gates.** It was verified separately by
|
||||
introspecting the built router stack and asserting the last literal layer precedes the first `/:slug`
|
||||
layer. Any future PR moving `/:slug`-style routes needs the same explicit check.
|
||||
- **`/admin/pages` (CMS page builder) and `/admin/shard/pages` (in-game help-page queue) are unrelated
|
||||
capabilities that read alike** — the latter stays with `shard` in PR 4. Same trap as PR 2's
|
||||
`activity` / `dashboard` / `bot-activity` trio.
|
||||
|
||||
### PR 4 — as landed
|
||||
|
||||
`shard`, `uo-link`, `email`, `discord-bot`, `settings` and `dashboard`/`site-mode` — the whole residual
|
||||
33. **`admin.routes.js` is deleted**, so the admin group is fully split and every one of its 110 routes
|
||||
is declared in a capability router.
|
||||
|
||||
| Router | Routes | Prefix | Extra gate |
|
||||
|---|---|---|---|
|
||||
| `shard.router.js` | 16 | `/admin/shard` | none on the 7 self-service routes; `modAccess` per route on the 9 staff ops |
|
||||
| `uoLink.router.js` | 5 | `/admin/uo-link` | `adminOnly` per route |
|
||||
| `email.router.js` | 6 | `/admin/email` | `adminOnly` per route |
|
||||
| `discordBot.router.js` | 2 | `/admin/discord-bot` | `adminOnly` per route |
|
||||
| `settings.router.js` | 2 | `/admin/settings` | `adminOnly` per route |
|
||||
| `dashboard.router.js` | 2 | **group root** (`/dashboard`, `/site-mode`) | `adminOnly` per route on `/site-mode` only |
|
||||
| ~~`admin.routes.js`~~ | — | deleted | — |
|
||||
|
||||
16 + 5 + 6 + 2 + 2 + 2 = 33. All four gates zero-diff: `routes.manifest.json` (200 public + 2
|
||||
internal), `routes.guards.json`, `swagger-output.json` (198 operations), and
|
||||
`docs/website/api-route-inventory.json` was already in sync. 434 server tests green.
|
||||
|
||||
Notes:
|
||||
|
||||
- **`dashboard.router.js` is mounted at the group root, not a prefix — the one relaxation of the
|
||||
"always mount at a prefix" rule, and it is deliberate.** `GET /dashboard` and `PUT /site-mode` own no
|
||||
common path segment, so a prefix mount would mean two one-route files instead of the single file the
|
||||
target tree calls for. It is safe **only** because the file declares no router-level middleware: a
|
||||
bare `use(gate)` in a root-mounted router runs for every request passing through toward another
|
||||
mount and would 403 an editor on an unrelated route (the PR 1 finding). The file says so in a
|
||||
comment, because the next person to add a gate there is the one who needs to know.
|
||||
- **`/shard` is the first prefix where two tiers share one router**, and it is why prefix ownership
|
||||
beats swagger-tag grouping. The 7 self-service routes (`link`, `accounts`, `roster/:account`,
|
||||
`vendors/:account`, `char/:serial`, `sales`, `POST account`) are tagged `Admin · Account`, run with
|
||||
no gate beyond the shared `staffOnly`, and are served by the very same `player/shard.controller`
|
||||
handlers as `/player/shard` — staff are a superset of players, and the controller keys off
|
||||
`req.user.id`. The 9 in-game ops are tagged `Admin · Shard` and carry `modAccess`. Splitting them by
|
||||
tag would put two routers under one prefix for no gain; instead one router owns `/shard` and gates
|
||||
per route. The tag mismatch stays, on the PR 1 and PR 3 precedent: retagging is a real spec diff and
|
||||
belongs in a PR that is about tags.
|
||||
- **No gate moved to router level anywhere in this PR.** Every `adminOnly` in the residual file was
|
||||
per-route, and `modAccess` on `/shard` must stay per-route because half that router must *not* have
|
||||
it. This keeps the per-route handler count intact — the one number `routes.guards.json` can actually
|
||||
check, since `requireRole(...)` returns an anonymous arrow.
|
||||
- **The `/:param` shadowing check was run again and is clean**, since the manifest sorts and therefore
|
||||
cannot see declaration order. Introspecting the built stack, all 110 admin routes and all 59 literal
|
||||
admin paths dispatch to their own layer — nothing is captured first by a `:param` sibling. The
|
||||
near-misses worth naming: `GET /shard/pages` (help-page queue) sits alongside
|
||||
`POST /shard/pages/:id/respond|close`, and `POST /shard/towncrier` alongside
|
||||
`DELETE /uo-link/towncrier/:id` — different depths and methods, so neither collides.
|
||||
- **Deleting the file left dangling `see admin.routes.js` pointers**, which were repointed in the same
|
||||
PR: `botActivity.controller.js` → `botActivity.router.js`, `moderation.controller.js` →
|
||||
`moderation.router.js`, `announceJobs.logic.js`'s town-crier cap mirror → `admin/uoLink.router.js`,
|
||||
and the "route paths sit on the line *after* `adminRouter.get(`" rationale in
|
||||
`scripts/routeManifest.js`, `README.md` and `pr-checks.yml` was generalized (it was never about that
|
||||
one file).
|
||||
- **`/admin/shard/pages` vs `/admin/pages` stayed separate**, as PR 3 flagged: the former is the
|
||||
in-game help-page (support) queue and belongs to `shard`; the latter is the CMS page builder.
|
||||
|
||||
### PR 5 — as landed
|
||||
|
||||
`public`, `player` and the residual `auth` — 54 routes across three groups, the last split PR.
|
||||
`public.routes.js`, `player.routes.js` and `auth.routes.js` are all **deleted**, so every one of the
|
||||
200 routes in the manifest is now declared in a capability router and no monolithic route file
|
||||
remains anywhere in `router/v1/`.
|
||||
|
||||
| Group | Router | Routes | Prefix | Extra gate |
|
||||
|---|---|---|---|---|
|
||||
| `public` | `posts.router.js` | 2 | `/public/posts` | none — `siteMode` per route |
|
||||
| | `wiki.router.js` | 4 | `/public/wiki` | none — `siteMode` per route |
|
||||
| | `pages.router.js` | 2 | `/public/pages` | none — `siteMode` per route except the preview |
|
||||
| | `shard.router.js` | 12 | `/public/shard` | none — never `siteMode` gated |
|
||||
| | `site.router.js` | 4 | **group root** (`/settings`, `/status`, `/version`, `/contact`) | none |
|
||||
| `player` | `account.router.js` | 8 | `/player/account` | none beyond the group gate |
|
||||
| | `shard.router.js` | 8 | `/player/shard` | none beyond the group gate |
|
||||
| | `appeals.router.js` | 4 | `/player/appeals` | none beyond the group gate |
|
||||
| `auth` | `login.router.js` | 2 | `/auth/login` | `loginGuards` per route |
|
||||
| | `register.router.js` | 1 | `/auth/register` | `loginGuards` + `registerLimiter` |
|
||||
| | `invite.router.js` | 2 | `/auth/invite` | `loginGuards` + `registerLimiter` on accept |
|
||||
| | `password.router.js` | 3 | `/auth/password` | per-route reset limiters |
|
||||
| | `session.router.js` | 2 | **group root** (`/logout`, `/me`) | per route |
|
||||
|
||||
24 + 20 + 10 = 54. `auth/`'s other 32 routes (`me` 23, `mobile` 5, `sso` 4) were already in their own
|
||||
files and did not move. All four gates zero-diff: `routes.manifest.json` (200 public + 2 internal),
|
||||
`routes.guards.json`, `swagger-output.json` (198 operations), and `docs/website/api-route-inventory.json`
|
||||
was already in sync. 434 server tests green.
|
||||
|
||||
Notes:
|
||||
|
||||
- **Each group is now a directory with an `index.js`**, matching `admin/`: `public/index.js`,
|
||||
`player/index.js`, `auth/index.js` own the group gate (where there is one) and the mount table and
|
||||
declare no routes. `v1.router.js` requires the directories. The four tests that imported the
|
||||
deleted entry files were repointed.
|
||||
- **Two of the three groups have no group gate, and that is the security-relevant fact about them.**
|
||||
`player/index.js` carries `noindex, requireAuth` — authenticated, *any* role, because staff are a
|
||||
superset of players. `public/index.js` and `auth/index.js` carry **nothing**, deliberately: the
|
||||
public surface is anonymous by contract (logged-out SPA, Discord bot, and the Android
|
||||
`ShardStreamClient` on `/public/shard/stream`, none of which send credentials), and `/auth` is where
|
||||
an anonymous caller *becomes* authenticated. Both index files say so, because the obvious "hardening"
|
||||
edit to either one is an outage.
|
||||
- **`GET /auth/me` has a mount-order dependency, and it is the one genuinely non-obvious thing in this
|
||||
PR.** `authRouter.use('/me', meRouter)` matches the bare path `/me`, not just `/me/*` — so a request
|
||||
to `GET /auth/me` runs `meRouter`'s (and `notifRouter`'s) `noindex, requireAuth`, matches no route
|
||||
inside either, and falls through to its own handler. `session.router.js` must therefore stay mounted
|
||||
**last**. Verified by the counterfactual rather than by reading the mount table: moving the mount to
|
||||
the top of `auth/index.js` still answers `401`, but the response loses its `X-Robots-Tag` header. No
|
||||
gate file and neither manifest can see that — only a header assertion can.
|
||||
- **Two root-mounted routers, on the PR 4 `dashboard.router.js` precedent.** `public/site.router.js`
|
||||
(`/settings`, `/status`, `/version`, `/contact`) and `auth/session.router.js` (`/logout`, `/me`) hold
|
||||
the routes that own no path segment. Both are safe at the root **only** because they declare no
|
||||
router-level middleware — a bare `use(gate)` there runs for every request passing through toward
|
||||
another mount. Both files say so.
|
||||
- **`loginGuards` is the PR's one shared module**, the counterpart to PR 3's `imageUpload.js`. The
|
||||
`[backoffGuard, slowLogin, loginLimiter]` array was defined inline in `auth.routes.js` and spread by
|
||||
four routes that this PR puts in three different files — plus a fifth, already-duplicated copy in
|
||||
`sso.routes.js`. It moved to `auth/loginGuards.js` and `sso.routes.js` now imports it too, so there
|
||||
is one definition rather than five: duplicating a throttling stack is how the copies drift, and the
|
||||
copy that drifts is the one that stops throttling. It is exported `Object.freeze`d — it is
|
||||
module-level shared state, and a router that pushed onto it would silently add middleware to every
|
||||
other login surface. Guard freshness is unaffected: the same three named functions, so
|
||||
`routes.guards.json` did not move.
|
||||
- **The `/:param` shadowing check was run again and is clean.** All 86 public/player/auth routes and
|
||||
all 64 literal paths among them dispatch to their own layer. This was checked in *dispatch* order
|
||||
against the built stack, since the manifest sorts and therefore cannot see declaration order. The
|
||||
only ordering-sensitive pair is `GET /public/wiki/{categories,tags}` ahead of `/public/wiki/:slug` —
|
||||
the public twin of the `admin/wiki.router.js` trap PR 3 found, and `wiki.router.js` says so. The
|
||||
`/public/pages` preview route also stays ahead of `/:slug`, though at a different depth.
|
||||
- **Filename deviations from the target tree, both for prefix agreement.** The tree named the public
|
||||
posts router `news.router.js`; it is `posts.router.js`, matching the `/posts` prefix it owns and its
|
||||
`admin/posts.router.js` sibling. The tree also implied `sso.router.js` / `mobile.router.js`; those
|
||||
files already exist as `sso.routes.js` / `mobile.routes.js` and were not renamed — they did not move
|
||||
in this PR, and churning their names would add diff noise to a PR whose value is being reviewable.
|
||||
- **`auth/session.router.js` is a deviation the target tree did not anticipate**, the same shape as
|
||||
PR 2's `activity.router.js`. The tree listed `login.router.js` but had nowhere to put `/logout` and
|
||||
`GET /me`, which own no prefix. Folding them into `login.router.js` would have forced *that* router
|
||||
to the group root and given up prefix ownership for the four login routes; a separate root-mounted
|
||||
singleton file keeps `/login` a real prefix mount.
|
||||
- **Tag mismatches were left alone again**, on the PR 1 / PR 3 / PR 4 precedent: the acceptance
|
||||
criterion is a byte-identical spec, so retagging belongs in a PR that is about tags.
|
||||
- **`public.controller.js` was not split.** Unlike the admin controllers, it is still one file serving
|
||||
settings/status/version/contact *and* posts/wiki/pages. The plan's rule is that these PRs re-wire
|
||||
routes, not logic — splitting a controller is a separate change with a separate risk profile, and
|
||||
bundling it would have cost this PR its "pure mechanical refactor" acceptance criteria.
|
||||
|
||||
### The swagger path-normalization prerequisite (landed before PR 1)
|
||||
|
||||
swagger-autogen builds a path by string-concatenating the mount prefix with the route argument, so a
|
||||
capability router mounted at `/users` whose collection route is `router.get('/')` documents as
|
||||
`/api/v1/admin/users/` — advertising a URL no client calls while dropping the one the SPA, the Android
|
||||
app and the Discord bot all do. Express is indifferent; the published spec is not. It also emits path
|
||||
keys in *router-traversal order*, so moving a route between files rewrote most of the ~5k-line
|
||||
committed artifact even when the API was provably unchanged — burying the one line a reviewer needs.
|
||||
|
||||
Both are fixed once in `server/swagger/swagger.js`, which post-processes the generator's output to
|
||||
strip trailing slashes and sort path keys (throwing on a collision rather than silently dropping an
|
||||
operation). It shipped as its own PR ahead of PR 1, verified inert by the regenerated spec being
|
||||
byte-for-byte the sorted form of the previously committed one — 198 operations, none added or removed.
|
||||
`#swagger.path` was rejected as the fix: it bypasses the mount prefix, so every route would hardcode
|
||||
its absolute path in a comment that silently lies the moment a mount moves.
|
||||
|
||||
**Consequence for PRs 2–5: the swagger diff is now a signal.** With sorting in place, a pure route
|
||||
move produces *no* spec diff at all, so any diff there means an annotation actually changed. Treat
|
||||
`swagger-output.json`, `routes.manifest.json` and `routes.guards.json` as three zero-diff gates.
|
||||
|
||||
### PR 0 — the route manifest (prerequisite of the first split PR) — **landed**
|
||||
|
||||
> **Status: shipped.** `server/scripts/routeManifest.js` + `npm run routes:manifest`,
|
||||
> `server/routes.manifest.json` (199 public + 2 internal), `server/routes.guards.json`,
|
||||
> `server/test/routeManifest.test.js`, and a `routes:manifest -- --check` step in
|
||||
> `.gitea/workflows/pr-checks.yml`. No router file moved. **The generator reproduced
|
||||
> `api-route-inventory.json` byte-for-byte on first run**, so the freeze is in effect and the
|
||||
> committed baseline is confirmed accurate rather than merely asserted.
|
||||
>
|
||||
> Two deviations from the design below, both deliberate:
|
||||
>
|
||||
> - **The unauthenticated-status snapshot was tried and dropped**, exactly as this section allowed.
|
||||
> Firing unauthenticated GETs at every manifest path against the dead-port mariadb pool the tests
|
||||
> use does not fail fast — the pool sits on its acquire timeout, and a partial sweep had not
|
||||
> finished after two minutes. A flaky two-minute gate is worse than none. What replaced it is
|
||||
> cheap and deterministic: the test suite asserts from the introspected stack that every
|
||||
> `/api/v1/admin/**` and `/api/v1/player/**` route still carries `requireAuth`.
|
||||
> - **`routes.guards.json` is committed and staleness-checked**, though a diff in it is explicitly
|
||||
> *not* a contract change. Left ungenerated it would rot into a misleading review aid within a
|
||||
> release. The gate is on freshness; the meaning of a guards diff is still "read this", not
|
||||
> "justify this". The generator drops app-level plumbing (helmet, morgan, the JSON parser, the bot
|
||||
> guard) since it applies uniformly to all 199 routes and would bury the per-route gates.
|
||||
|
||||
|
||||
|
||||
"Every URL is unchanged" must be *proved by a diff*, not asserted in review. PR 0 lands the tool that
|
||||
proves it, with no router file moved.
|
||||
|
||||
- **Generator:** `server/scripts/routeManifest.js`, wired as `npm run routes:manifest`. It requires
|
||||
`src/app.js` (which exports the app and neither listens nor connects to the DB — `server.js` owns
|
||||
those), walks `app._router.stack` recursively through mounted routers, reconstructs each full path
|
||||
from the layer regexps, and writes a **sorted** array of
|
||||
`{ "method": "GET", "path": "/api/v1/admin/users/:id" }` to `server/routes.manifest.json`.
|
||||
`internalApp.js` is walked into a separate `internal` section so the unpublished port is inventoried
|
||||
without being confused for public surface.
|
||||
- **Scope it to the API surface, or it won't be deterministic.** Three mounts are *filesystem*
|
||||
conditional: the SPA catch-all `GET *` (only when `client/dist/index.html` exists), the `/brand`
|
||||
static mount, and `/api/docs*` (only when `swagger-output.json` is present — it is committed, so it
|
||||
is stable). The manifest keeps only `/api/**`, `/.well-known/**`, and the internal app's routes, so
|
||||
it does not change depending on whether CI built the client. Static mounts are not API contract.
|
||||
- **The baseline already exists:** [`api-route-inventory.json`](./api-route-inventory.json) in this
|
||||
directory is the frozen surface — **199 API routes** (110 of them `/api/v1/admin`) plus 2
|
||||
internal at the time PR 0 was written. PR 0's generator must **reproduce this file byte-for-byte**;
|
||||
that is PR 0's own acceptance test, and it means the freeze is already in effect before the first
|
||||
router moves. *(It did, on first run. The file has since moved to **200** — the CSP report-only PR
|
||||
added `POST /api/csp-report`, the first deliberate, reviewed manifest diff.)*
|
||||
- **Runtime introspection, not source parsing.** It is authoritative about mounts, and the route paths
|
||||
in `admin.routes.js` sit on the line *after* `adminRouter.get(`, which defeats naive greps.
|
||||
- **Not `swagger-output.json`.** That is annotation-derived (only annotated routes appear) and churns
|
||||
for unrelated reasons; it documents intent, the manifest records reality.
|
||||
- **Frozen key: method + path only.** That is exactly the contract being preserved. Handler names are
|
||||
useless as a guard check here — `requireRole(...)` returns an anonymous arrow, and router-level gates
|
||||
like `adminRouter.use(noindex, isLoggedIn, staffOnly)` never appear in a route's own stack.
|
||||
- **Guard coverage, separately.** The generator also emits a non-gated review aid: per route, the
|
||||
handler count plus any *named* middleware collected along the mount chain. If a stable behavioral
|
||||
check proves cheap, prefer it — a test that fires an **unauthenticated** request at every manifest
|
||||
path and snapshots the status code catches a dropped `adminOnly` (403 → 200) in a way names cannot.
|
||||
Try it in PR 0; if DB-touching public routes make it slow or noisy against the dead-port pool the
|
||||
tests use, drop it rather than ship a flaky gate.
|
||||
- **CI:** `.gitea/workflows/pr-checks.yml` runs `npm run routes:manifest` and
|
||||
`git diff --exit-code server/routes.manifest.json`. A PR that moves a URL fails unless it
|
||||
deliberately commits the new manifest — which puts the URL change in front of a reviewer instead of
|
||||
letting it pass silently.
|
||||
- **Published copy:** `docs/website/api-route-inventory.json` mirrors `server/routes.manifest.json` and
|
||||
is refreshed in each split PR's mandatory docs edit. The markdown table above is orientation for a
|
||||
human reader; **the manifest is the authoritative freeze.**
|
||||
|
||||
---
|
||||
|
||||
## Sequencing & PR breakdown
|
||||
|
||||
CSP and the split are independent; the only hard ordering is PR 0 before the first split PR.
|
||||
|
||||
**Resequenced during implementation: PR 0 ships first, before the CSP pair.** The CSP report-only PR
|
||||
has to stand up a violation collector (`POST /api/csp-report`) for `report-to` to point at — which is
|
||||
a new URL under `/api/**`. Landing it first would mean PR 0's generator emitting 200 routes against a
|
||||
199-route committed baseline, so PR 0 could no longer prove itself by reproducing
|
||||
`api-route-inventory.json` byte-for-byte. With PR 0 first, the collector shows up as a reviewed,
|
||||
deliberate `+1` in the manifest — which is exactly the mechanism working as designed.
|
||||
|
||||
1. **PR 0 — route manifest.** Generator + CI check + committed baseline of today's surface. No routers moved. ✅ landed
|
||||
2. **PR — CSP report-only.** Tightened policy behind `Content-Security-Policy-Report-Only` + `report-to`,
|
||||
plus the report collector (manifest `+1` — the first deliberate, reviewed manifest diff). ✅ landed
|
||||
3. **PR — CSP enforce.** One release later, assuming a clean violation report. **Blocked on real soak
|
||||
data**, not on code: watch the `csp` log tag for `frame-ancestors` reports across one release before
|
||||
flipping. Also decide there whether `/api/csp-report` is retired with the report-only twin or kept
|
||||
as a `report-to` group on the enforced policy.
|
||||
4. **PR 1 — admin:** `users`, `account`, `invites`, `auth` (providers). ✅ landed
|
||||
5. **PR 2 — admin:** `moderation`, `bot-activity`, `activity`. ✅ landed
|
||||
6. **PR 3 — admin (content):** `posts`, `pages`, `wiki`, `uploads`. ✅ landed
|
||||
7. **PR 4 — admin (ops/config):** `shard`, `uo-link`, `email`, `discord-bot`, `settings`, `site-mode`,
|
||||
`dashboard`. (`activity` went with PR 2 — see § PR 2 — as landed.) **This is the whole residual
|
||||
file** — `admin.routes.js` is deleted here, not by PR 5. ✅ landed
|
||||
8. **PR 5 — `public/*` + `player/*`** (and the residual `auth/*` grouping). ✅ landed — **the domain
|
||||
split is complete.** The only remaining item in this plan is the CSP enforce PR (3), which is
|
||||
blocked on soak data, not on code.
|
||||
|
||||
Each PR: **zero-line diff in `routes.manifest.json`**, server tests green
|
||||
(`cd website/server && npm test`), Swagger regenerated, matching `docs/` edit, Conventional Commit,
|
||||
AI-disclosure trailer, branch from a freshly-pulled `main`.
|
||||
|
||||
---
|
||||
|
||||
## Cross-component blast radius
|
||||
|
||||
Three independent clients consume the site's HTTP/SSE API, two of them in separate repos on separate
|
||||
release cadences. **With the auth merge and the version bump both gone, the blast radius is empty** —
|
||||
no client's URLs or authentication change at all.
|
||||
|
||||
| Consumer | Repo (cadence) | Pinning | Impact under this plan |
|
||||
|---|---|---|---|
|
||||
| Browser SPA | `website/client` (lockstep) | `BASE = /api/v1` in `client/src/api/client.js` | **None.** Same URLs, same cookie session. |
|
||||
| Android app | `android-app` (app-store cadence, un-updatable installs in the wild) | 69 hardcoded `api/v1/…` paths; SSE path in `ShardStreamClient.kt`; SSO in `SsoAuthManager.kt` | **None.** No repoint, no release required. |
|
||||
| Discord bot | `website/bot` (separate deploy, env-configured) | `SITE_PUBLIC_URL` env → `/api/v1/public` | **None.** Anonymous public reads on unchanged paths. |
|
||||
|
||||
Standing constraints, unchanged:
|
||||
|
||||
- **The public SSE stream stays anonymous.** Consumed by logged-out browser visitors *and* the Android
|
||||
`ShardStreamClient`, neither of which sends an `Authorization` header. Adding `requireAuth` blacks
|
||||
out the public live boards on web and mobile. The single most likely regression in a careless
|
||||
refactor is reflexively wrapping *both* shard streams in auth.
|
||||
- **The admin SSE stream keeps its current cookie-based gating** (`isLoggedIn`, `EventSource` +
|
||||
`withCredentials`) — the rewrite that would have changed this went out with the auth merge.
|
||||
- **The public/admin allowlist split is a security boundary**, not an implementation detail. Preserve
|
||||
it verbatim in `utils/shardBroadcast.js` / `utils/shardIngest.js` as routes move.
|
||||
- **SSO / email-connect transaction cookies are load-bearing for web *and* native.** The Android SSO
|
||||
flow opens a Custom Tab to the website's `/auth/…/sso/start` and rides the same server-side redirect
|
||||
transaction and the same tx cookies. Nothing in this plan touches them; don't let a future "cookies
|
||||
go away" push delete them.
|
||||
- **`link/` is out of scope.** The sidecar contract (`uoLinkConfig`, `utils/uoLinkClient.js`,
|
||||
`utils/shardIngest.js`, `X-UOLink-Version`) is a separate versioning axis. `PROTOCOL_VERSION` does
|
||||
**not** bump for this work.
|
||||
|
||||
---
|
||||
|
||||
## Appendix: mapping from the previous plan
|
||||
|
||||
| Previous | Now |
|
||||
|---|---|
|
||||
| Phase 0 — `/api/mobile` facade + app-version floor | **Deferred.** See § Deferred: the `/api/mobile` facade |
|
||||
| Phase 1 — auth merge | **Removed.** See § Why the auth merge is out and § Deferred: the auth merge |
|
||||
| Phase 1b — CSP hardening (shipped with the auth merge) | **Phase 1**, standalone; report-only-first ordering fixed |
|
||||
| Phase 2 — domain split under `router/v2/` | **Phase 2**, in place under `router/v1/`; now the primary driver |
|
||||
| PR 1 — `/api/v2` scaffold ([API_V2_SKELETON.md](./API_V2_SKELETON.md)) | **Superseded**, kept as the recipe if a versioned API is ever forced |
|
||||
| PR final — retire v1 | **Not applicable** — v1 is never replaced |
|
||||
@@ -1,131 +0,0 @@
|
||||
# Website API v2 — `/api/v2` Skeleton (PR 1)
|
||||
|
||||
> ## ⚠ Superseded — not scheduled
|
||||
>
|
||||
> The router domain split is being done **in place**, with every URL byte-identical, so there is no
|
||||
> parallel version to stand up and this scaffold will not be built. See
|
||||
> [API_V2_PLAN.md](./API_V2_PLAN.md) § Why there is no `/api/v2`.
|
||||
>
|
||||
> The file is kept, unedited below, as the concrete recipe **if** a real contract break ever forces a
|
||||
> versioned API. Nothing here describes current or planned work.
|
||||
|
||||
Companion to [API_V2_PLAN.md](./API_V2_PLAN.md) — this is the concrete scaffold for **PR 1** in that
|
||||
plan's sequencing. It stands up `/api/v2` **empty but wired**, next to a frozen `/api/v1`, with **no
|
||||
behavior change**. Endpoints are filled in by the later PRs (auth merge, then the domain split).
|
||||
|
||||
## Scope
|
||||
|
||||
- Create the `router/v2/` tree of empty, domain-named routers.
|
||||
- Mount `/v2` alongside `/v1` in `api.router.js`.
|
||||
- Add a single trivial `GET /api/v2/version` so the mount is testable end-to-end.
|
||||
- **Out of scope:** any real endpoint, any auth change, any controller edit. Those are PR 2+.
|
||||
|
||||
## File tree to create
|
||||
|
||||
```
|
||||
website/server/src/router/v2/
|
||||
v2.router.js # mounts the domain sub-routers; adds GET /version
|
||||
admin/
|
||||
index.js # mounts the admin capability routers under /admin
|
||||
dashboard.router.js users.router.js moderation.router.js
|
||||
content.router.js wiki.router.js shard.router.js
|
||||
settings.router.js invites.router.js bot-activity.router.js
|
||||
auth/
|
||||
index.js login.router.js sso.router.js totp.router.js session.router.js
|
||||
public/
|
||||
index.js news.router.js wiki.router.js page.router.js shard.router.js
|
||||
player/
|
||||
index.js profile.router.js appeals.router.js shard.router.js
|
||||
```
|
||||
|
||||
`internal/` is **not** part of v2's public tree — the internal routes stay on the separate,
|
||||
unpublished port (`internalApp.js`), exactly as in v1. See `API_V2_PLAN.md` § Phase 2.
|
||||
|
||||
## Wiring
|
||||
|
||||
`api.router.js` gains the v2 mount next to v1:
|
||||
|
||||
```js
|
||||
const v1Router = require('./v1/v1.router')
|
||||
const v2Router = require('./v2/v2.router')
|
||||
|
||||
apiRouter.use('/v1', v1Router)
|
||||
apiRouter.use('/v2', v2Router) // NEW — parallel version, migrate off v1 route-by-route
|
||||
```
|
||||
|
||||
`v2.router.js` mounts each domain group and exposes the version ping:
|
||||
|
||||
```js
|
||||
const express = require('express')
|
||||
const v2Router = express.Router()
|
||||
|
||||
const adminRouter = require('./admin')
|
||||
const authRouter = require('./auth')
|
||||
const publicRouter = require('./public')
|
||||
const playerRouter = require('./player')
|
||||
|
||||
// Cheap liveness/mount check so the parallel version is testable before any
|
||||
// real endpoint exists. Returns the API major version, nothing sensitive.
|
||||
v2Router.get('/version', (req, res) => res.json({ version: 2 }))
|
||||
|
||||
v2Router.use('/auth', authRouter)
|
||||
v2Router.use('/public', publicRouter)
|
||||
v2Router.use('/admin', adminRouter)
|
||||
v2Router.use('/player', playerRouter)
|
||||
// NOTE: /internal is intentionally NOT mounted here — same reason as v1.
|
||||
|
||||
module.exports = v2Router
|
||||
```
|
||||
|
||||
Each capability router is an empty stub at this stage — a router that mounts cleanly and adds no
|
||||
routes yet, so PR 2+ only has to add handlers, never re-wire:
|
||||
|
||||
```js
|
||||
// router/v2/admin/dashboard.router.js
|
||||
const express = require('express')
|
||||
const router = express.Router()
|
||||
|
||||
// Routes added in the admin domain-split PR (see API_V2_PLAN.md § Phase 2).
|
||||
|
||||
module.exports = router
|
||||
```
|
||||
|
||||
Each `index.js` mounts its group's capability routers under the URL that names them, e.g.:
|
||||
|
||||
```js
|
||||
// router/v2/admin/index.js
|
||||
const express = require('express')
|
||||
const admin = express.Router()
|
||||
|
||||
admin.use('/dashboard', require('./dashboard.router'))
|
||||
admin.use('/users', require('./users.router'))
|
||||
admin.use('/moderation', require('./moderation.router'))
|
||||
admin.use('/content', require('./content.router'))
|
||||
admin.use('/wiki', require('./wiki.router'))
|
||||
admin.use('/shard', require('./shard.router'))
|
||||
admin.use('/settings', require('./settings.router'))
|
||||
admin.use('/invites', require('./invites.router'))
|
||||
admin.use('/bot-activity', require('./bot-activity.router'))
|
||||
|
||||
module.exports = admin
|
||||
```
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Server boots with no error; every sub-router mounts.
|
||||
- `GET /api/v2/version` → `200 { "version": 2 }`.
|
||||
- `GET /api/v1/**` behavior is **byte-for-byte unchanged** — v1 is untouched.
|
||||
- Existing server tests stay green (`cd website/server && npm test`).
|
||||
- A new test asserts the `/api/v2/version` mount (smallest possible coverage of the wiring).
|
||||
|
||||
## Docs / spec
|
||||
|
||||
- Swagger regeneration is deferred until v2 has real routes (PR 2) — a lone `/version` ping doesn't
|
||||
need an annotation. When PR 2 lands, add `#swagger.*` to the new routes and run `npm run swagger`.
|
||||
- No `BACKEND_DESIGN.md` change here beyond noting the parallel `/api/v2` mount exists; the
|
||||
route-map/security-contract edits land with the PRs that add real endpoints.
|
||||
|
||||
## Next
|
||||
|
||||
PR 2 fills the `auth/` routers with the bearer access + refresh flow and drops the session cookie —
|
||||
see [API_V2_PLAN.md](./API_V2_PLAN.md) § Phase 1.
|
||||
@@ -1,120 +0,0 @@
|
||||
# Website — Architecture
|
||||
|
||||
How the pieces of `RunicGateway/website` fit together. The React SPA and the native mobile app
|
||||
talk to one Express backend (`router → controller → model → db`), which persists to MariaDB and
|
||||
bridges to the live game world **only** through the **uo-link** sidecar. The ServUO shard itself is
|
||||
never internet-facing — it dials out to the sidecar over loopback, and only the sidecar is exposed.
|
||||
|
||||
This is the canonical copy of the diagram; the same diagram is embedded in the website's
|
||||
[`README.md`](https://gitea.whitlocktech.com/RunicGateway/website/src/branch/main/README.md#architecture).
|
||||
See [BACKEND_DESIGN.md](BACKEND_DESIGN.md) for the full API / schema / security contract, and
|
||||
[`docs/link/`](../link/) for the wire protocol between the sidecar and the shard.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
%% ---------- Clients ----------
|
||||
subgraph clients["Clients"]
|
||||
browser["Browser<br/>React + Vite SPA<br/>(public · wiki · admin)"]
|
||||
mobile["Native mobile app<br/>(bearer tokens)"]
|
||||
end
|
||||
|
||||
idp["SSO providers<br/>Google · Discord · custom OIDC"]
|
||||
discord["Discord"]
|
||||
|
||||
%% ---------- Website (one repo) ----------
|
||||
subgraph website["website/ — Node app (one repo)"]
|
||||
direction TB
|
||||
|
||||
subgraph backend["server/ — Express backend"]
|
||||
direction TB
|
||||
mw["Middleware<br/>helmet · siteMode · noindex<br/>rateLimit · loginProtection · botScore · validate"]
|
||||
router["Router /api/v1<br/>auth (web · mobile · sso) · public · admin"]
|
||||
ctrl["Controllers"]
|
||||
auth["Session layer (auth/)<br/>sessionService · JWT/cookie · bearer · SSO+PKCE"]
|
||||
model["Models (.model + .db)<br/>raw parameterized SQL — no ORM"]
|
||||
sse["SSE fan-out<br/>public stream (allowlist) · admin stream (sensitive)"]
|
||||
|
||||
subgraph shardutil["Shard integration (utils/)"]
|
||||
ingest["shardIngest.js<br/>WS ingest dispatcher"]
|
||||
restcli["uoLinkClient.js<br/>REST client (never throws)"]
|
||||
end
|
||||
|
||||
secret["secretBox.js<br/>AES-256-GCM secrets at rest"]
|
||||
end
|
||||
|
||||
bot["bot/<br/>Discord bot"]
|
||||
end
|
||||
|
||||
db[("MariaDB<br/>users · posts · wiki · settings · activity<br/>mobileSessions · authProviders · userIdentities<br/>uoLinkConfig · shard_online/economy/houses/events")]
|
||||
|
||||
%% ---------- Shard side ----------
|
||||
subgraph shardside["Game shard (never internet-facing)"]
|
||||
direction TB
|
||||
sidecar["uo-link sidecar<br/>(Rust) — the only bridge exposed"]
|
||||
servuo["ServUO shard<br/>(C# plugin)"]
|
||||
end
|
||||
|
||||
%% ---------- Edges ----------
|
||||
browser <-->|"same-origin JSON + SSE (cookie)"| mw
|
||||
mobile -->|"REST (bearer access/refresh)"| mw
|
||||
browser -.->|"OAuth redirect + PKCE"| idp
|
||||
auth -.->|"token exchange"| idp
|
||||
|
||||
mw --> router --> ctrl
|
||||
ctrl --> auth
|
||||
ctrl --> model
|
||||
ctrl --> restcli
|
||||
ctrl --> sse
|
||||
auth --> model
|
||||
model <--> db
|
||||
auth -. reads/writes secrets .-> secret
|
||||
restcli -. reads config/token .-> secret
|
||||
ingest --> model
|
||||
ingest --> sse
|
||||
sse -->|"live events"| browser
|
||||
bot -->|"messages"| discord
|
||||
bot <--> db
|
||||
|
||||
restcli -->|"REST: /char /roster /economy /history · /link/confirm · /towncrier"| sidecar
|
||||
sidecar -->|"WebSocket live event feed (bearer + X-UOLink-Version)"| ingest
|
||||
servuo -->|"loopback TCP 127.0.0.1:7788<br/>newline-delimited JSON (shard dials out)"| sidecar
|
||||
|
||||
%% ---------- Styling ----------
|
||||
classDef ext fill:#2d2233,stroke:#7a5c94,color:#e8dff0;
|
||||
classDef store fill:#1f2d2a,stroke:#4c8c7d,color:#dff0ea;
|
||||
classDef bridge fill:#2d2620,stroke:#94764c,color:#f0e6d8;
|
||||
class idp,discord ext;
|
||||
class db store;
|
||||
class sidecar,servuo bridge;
|
||||
```
|
||||
|
||||
## Notes on the diagram
|
||||
|
||||
- **One backend, layered.** Every request flows `middleware → router → controller → model → db`.
|
||||
Web browsers authenticate with an httpOnly JWT cookie; the native app uses short-lived bearer
|
||||
access tokens plus rotated, hashed, revocable refresh tokens; SSO (Google/Discord/OIDC) is
|
||||
link-only and PKCE-guarded. All three surfaces resolve to the *same* session model via the session
|
||||
layer, and admin routes are re-validated against the DB on every request.
|
||||
- **The shard is never reachable.** The ServUO shard *dials out* over loopback TCP `127.0.0.1:7788`
|
||||
(newline-delimited JSON) to the uo-link sidecar; only the sidecar is exposed, and only the backend
|
||||
talks to it. Every backend→sidecar call carries `Authorization: Bearer <token>` and an
|
||||
`X-UOLink-Version` header (a protocol mismatch fails fast with `409`). The REST client
|
||||
(`uoLinkClient.js`) never throws — every call returns `{ ok, data, status }` — so the site degrades
|
||||
gracefully when the shard is down. That guarantee covers **reading the config too**: resolving the
|
||||
admin-managed config decrypts the stored auth token, which throws when the ciphertext can't be
|
||||
authenticated (`SECRET_ENC_KEY` rotated, or a DB dump restored under a different key). This is
|
||||
handled inside the client and reported as `{ ok: false, status: 0, error: 'uo-link config
|
||||
unreadable' }` plus a distinct `ERROR`-level log, so a wrong key degrades the shard surface to
|
||||
"unavailable" instead of 500ing it — and `GET /admin/uo-link/config` keeps working, which is the
|
||||
screen an admin needs to re-enter the token and recover.
|
||||
- **Two ways in from the sidecar.** Live game events arrive over an outbound **WebSocket** and are
|
||||
routed by the `shardIngest.js` dispatcher (state-changing kinds update `shard_*` tables, notable
|
||||
kinds append to `shard_events`, high-frequency kinds only update state). Point-in-time reads and
|
||||
commands go over **REST** through `uoLinkClient.js`.
|
||||
- **Sensitive events stay private.** Ingested events fan out to browsers over two **SSE** channels —
|
||||
a public allowlist stream and an admin-only stream that additionally carries staff audit, cheat
|
||||
detection, and login-attempt events. The allowlist split is a security boundary; sensitive kinds
|
||||
can never leak onto the public channel.
|
||||
- **Secrets at rest.** OAuth client secrets, the uo-link token, and the Gmail refresh token are
|
||||
AES-256-GCM encrypted via `secretBox.js` (keyed by `SECRET_ENC_KEY`). The uo-link token is
|
||||
write-only in the API — never returned to any client.
|
||||
@@ -29,24 +29,6 @@ Public contact email: **UOMysticmoon@gmail.com**
|
||||
|
||||
Skeleton from the spec, with a small number of justified additions marked **(+)**.
|
||||
|
||||
> **Complete.** The monolithic route files (`admin.routes.js` especially, originally 1552 lines /
|
||||
> 110 routes) have been split into one router file per business capability — **in place, with every
|
||||
> URL unchanged**. See [API_V2_PLAN.md](./API_V2_PLAN.md) § Phase 2.
|
||||
>
|
||||
> `users`, `account`, `invites`, `auth/providers` (PR 1, 28 routes), `moderation`, `bot-activity`,
|
||||
> `activity` (PR 2, 18 routes), `posts`, `uploads`, `wiki`, `pages` (PR 3, 31 routes) and `shard`,
|
||||
> `uo-link`, `email`, `discord-bot`, `settings`, `dashboard`/`site-mode` (PR 4, 33 routes) each live
|
||||
> in their own router under `admin/`, behind `admin/index.js`. PR 5 did the same for `public/` (24),
|
||||
> `player/` (20) and the residual `auth/` (10). **`admin.routes.js`, `public.routes.js`,
|
||||
> `player.routes.js` and `auth.routes.js` are all deleted**; each group is now a directory whose
|
||||
> `index.js` owns the group gate and the mount table and declares no routes of its own.
|
||||
>
|
||||
> "Every URL unchanged" is enforced mechanically, not by review: `server/scripts/routeManifest.js`
|
||||
> (`npm run routes:manifest`) walks the live Express stack and writes the sorted
|
||||
> `{ method, path }` freeze to `server/routes.manifest.json`, mirrored here as
|
||||
> [api-route-inventory.json](./api-route-inventory.json). PR checks regenerate it and fail on any
|
||||
> diff, so a split PR that moves a URL cannot merge silently. See § 4.0.
|
||||
|
||||
```
|
||||
server/
|
||||
.env.example
|
||||
@@ -60,108 +42,10 @@ server/
|
||||
router/
|
||||
api.router.js mounts /v1
|
||||
v1/
|
||||
v1.router.js mounts /auth /public /admin /player
|
||||
auth/ index.js mounts the routers below; no group gate — /auth
|
||||
is where an anonymous caller becomes
|
||||
authenticated, so the authenticated parts gate
|
||||
themselves. Mount order is load-bearing (see
|
||||
session.router.js)
|
||||
login.router.js (2) /auth/login + /login/totp — shared
|
||||
loginGuards stack
|
||||
register.router.js (1) /auth/register — honours the
|
||||
player_registration setting
|
||||
invite.router.js (2) /auth/invite/:token[/accept] — the
|
||||
token is its own authority, so it
|
||||
bypasses player_registration
|
||||
password.router.js (3) /auth/password/forgot + reset/:token
|
||||
session.router.js (2) POST /logout and GET /me — the two
|
||||
singletons owning no path segment, so
|
||||
mounted at the group root, LAST: the
|
||||
/me sub-routers below also match the
|
||||
bare /me and supply its noindex header
|
||||
me.routes.js (23) /auth/me/account*, sessions, trusted
|
||||
devices — router-level requireAuth
|
||||
notifications.routes.js (3) /auth/me/devices*, notifications/*
|
||||
mobile.routes.js + /auth/mobile/* — native bearer login
|
||||
mobileSso.routes.js (5)
|
||||
sso.routes.js (4) mounted PATHLESS: owns two prefixes,
|
||||
/auth/providers and /auth/sso/*
|
||||
loginGuards.js shared backoff/slow/limiter stack for
|
||||
every credential-guessing surface
|
||||
(not a router)
|
||||
auth.controller.js + invite/passwordReset/sso/mobile controllers
|
||||
public/ index.js mounts the routers below; **no group gate** —
|
||||
this surface is anonymous by design (SPA
|
||||
logged-out, Discord bot, Android ShardStream)
|
||||
posts.router.js (2) /public/posts/:category[/:idOrSlug]
|
||||
wiki.router.js (4) /public/wiki — /categories and /tags
|
||||
MUST precede /:slug
|
||||
pages.router.js (2) /public/pages — the draft-preview
|
||||
route precedes /:slug and is
|
||||
deliberately not site-mode gated
|
||||
shard.router.js (14) /public/shard/* incl. the anonymous
|
||||
SSE stream; never site-mode gated
|
||||
atlas.router.js (6) /public/atlas/* — the spawn atlas.
|
||||
NOT under /shard: nothing here
|
||||
touches the sidecar, and unlike
|
||||
/shard it IS site-mode gated
|
||||
site.router.js (4) /settings /status /version /contact —
|
||||
the group-root singletons; declares no
|
||||
router-level middleware
|
||||
public.controller.js + shard.controller.js
|
||||
player/ index.js owns the shared `noindex, requireAuth` gate
|
||||
(authenticated, ANY role — staff are a superset
|
||||
of players) and the mount table
|
||||
account.router.js (8) /player/account — credentials, TOTP,
|
||||
linked identities; handlers shared
|
||||
with /admin/account and /auth/me
|
||||
shard.router.js (8) /player/shard — linking + own roster,
|
||||
vendors, chars, sales, houses
|
||||
appeals.router.js (4) /player/appeals
|
||||
shard.controller.js + appeals.controller.js
|
||||
admin/ index.js mounts the capability routers below at their
|
||||
own prefixes; owns the shared
|
||||
`noindex, isLoggedIn, staffOnly` gate and
|
||||
declares no routes itself
|
||||
account.router.js (6) /admin/account — self-service, no adminOnly
|
||||
users.router.js (15) /admin/users — adminOnly
|
||||
invites.router.js (3) /admin/invites — adminOnly
|
||||
authProviders.router.js (4) /admin/auth — adminOnly
|
||||
moderation.router.js (15) /admin/moderation — modAccess
|
||||
(admin+moderator) at router level
|
||||
botActivity.router.js (2) /admin/bot-activity — adminOnly
|
||||
activity.router.js (1) /admin/activity — staff-wide
|
||||
audit log, no extra gate
|
||||
posts.router.js (9) /admin/posts — editor tier, no
|
||||
gate beyond staffOnly
|
||||
uploads.router.js (1) /admin/uploads — rich-text editor
|
||||
image upload
|
||||
wiki.router.js (14) /admin/wiki — pages, revisions,
|
||||
categories, tags
|
||||
pages.router.js (7) /admin/pages — CMS page builder
|
||||
imageUpload.js shared multer config for the two
|
||||
upload routes above (not a router)
|
||||
shard.router.js (16) /admin/shard — 7 self-service
|
||||
account-linking routes (no extra
|
||||
gate, handlers shared with
|
||||
/player/shard) + 9 in-game staff
|
||||
ops on modAccess, per route
|
||||
uoLink.router.js (5) /admin/uo-link — sidecar config,
|
||||
town crier, admin SSE — adminOnly
|
||||
email.router.js (6) /admin/email — Gmail OAuth2
|
||||
delivery — adminOnly
|
||||
discordBot.router.js (2) /admin/discord-bot — adminOnly
|
||||
settings.router.js (2) /admin/settings — adminOnly
|
||||
dashboard.router.js (2) GET /dashboard (staff-wide) and
|
||||
PUT /site-mode (adminOnly) — the
|
||||
two singletons owning no path
|
||||
segment, so mounted at the group
|
||||
root; declares no router-level
|
||||
middleware, which is what makes a
|
||||
root mount safe
|
||||
admin.controller.js + the per-capability controllers
|
||||
(already domain-split; the split PRs re-wire
|
||||
routes, not logic)
|
||||
v1.router.js mounts /auth /public /admin
|
||||
auth/ auth.routes.js + auth.controller.js
|
||||
public/ public.routes.js + public.controller.js
|
||||
admin/ admin.routes.js + admin.controller.js
|
||||
model/
|
||||
users/ users.model.js + users.db.js
|
||||
posts/ posts.model.js + posts.db.js (news/five-on-friday/newsletter/screenshots)
|
||||
@@ -272,322 +156,6 @@ Seeded keys: `site_mode` (default `maintenance`), `site_mode_changed_at`,
|
||||
Same "store only the hash of an opaque token" pattern as `user_invites` / `mobile_refresh_tokens`.
|
||||
A DB read never yields a usable reset link. See §4 `/auth/password/*`.
|
||||
|
||||
### push_devices — opt-in push endpoints (M7)
|
||||
| col | type | notes |
|
||||
|---|---|---|
|
||||
| id | INT PK AUTO_INCREMENT | |
|
||||
| user_id | INT NOT NULL FK→users(id) ON DELETE CASCADE | owner |
|
||||
| transport | ENUM('unifiedpush','fcm') DEFAULT 'unifiedpush' | UnifiedPush for the sideloaded APK; FCM reserved for a later Play flavor |
|
||||
| endpoint | VARCHAR(512) NOT NULL | the distributor URL the app's ntfy topic was handed (or an FCM token). Unguessable but **not a secret** — stored in the clear (unlike refresh tokens), because pushes are content-free tickles |
|
||||
| platform | VARCHAR(40) NULL | free-form label, e.g. `android` |
|
||||
| created_at / last_seen_at | DATETIME | |
|
||||
|
||||
`UNIQUE(user_id, endpoint)` — re-registering the same endpoint is an idempotent upsert.
|
||||
|
||||
### notification_subscriptions — which streams a user opted into (M7)
|
||||
| col | type | notes |
|
||||
|---|---|---|
|
||||
| user_id | INT NOT NULL FK→users(id) ON DELETE CASCADE | |
|
||||
| stream_id | VARCHAR(64) NOT NULL | an id from the catalog (`config/notificationStreams.js`), validated on write |
|
||||
| created_at | DATETIME | |
|
||||
|
||||
`PRIMARY KEY(user_id, stream_id)`. Subscriptions are per-user (applied to every device); a PUT
|
||||
replaces the whole set. Nothing is pushed unless the user subscribed.
|
||||
|
||||
### mobile_auth_sessions / mobile_auth_codes — mobile SSO bridge (M9)
|
||||
|
||||
Two short-lived, self-pruning tables that bridge a browser SSO redirect flow to a native client. They
|
||||
carry the **app ↔ website** PKCE + CSRF state (a *second* PKCE layer, distinct from the website ↔ IdP
|
||||
PKCE the `sso_tx` cookie already carries) and the one-time authorization code the app exchanges for
|
||||
bearer tokens. Neither holds a secret in the clear — the PKCE `code_challenge` is a hash by
|
||||
construction, and the authorization code is stored as a **sha256 hash only** (same pattern as
|
||||
`user_invites` / `password_resets` / `mobile_refresh_tokens`).
|
||||
|
||||
`mobile_auth_sessions` — one row per `/auth/mobile/sso/start`:
|
||||
|
||||
| col | type | notes |
|
||||
|---|---|---|
|
||||
| id | INT PK AUTO_INCREMENT | |
|
||||
| session_id | CHAR(36) UNIQUE | opaque uuid; carried inside the signed `sso_tx` (mode `mobile`) so the callback can find this row |
|
||||
| provider | VARCHAR(40) NOT NULL | provider id validated enabled at `/start` |
|
||||
| code_challenge | VARCHAR(255) NOT NULL | app-supplied PKCE S256 challenge (base64url); verified at `/exchange` |
|
||||
| redirect_uri | VARCHAR(255) NOT NULL | the requested app callback — **exact-match** against the allowlist (never prefix) |
|
||||
| state | VARCHAR(255) NOT NULL | app-generated opaque CSRF value, echoed on the callback for the app to verify |
|
||||
| status | ENUM('pending','completed','consumed') DEFAULT 'pending' | `pending`→`completed` when the code is minted; `consumed` after a successful exchange |
|
||||
| user_id | INT NULL FK→users(id) ON DELETE CASCADE | set once SSO resolves the account |
|
||||
| trust_device | TINYINT(1) NOT NULL DEFAULT 0 | user ticked "trust this device" on the Custom Tab TOTP form. A **boolean only** — it tells `/exchange` to mint the app's own trust token; the token never rests here (only its sha256 reaches `trusted_devices`) |
|
||||
| expires_at | DATETIME NOT NULL | short (~10 min — one redirect round-trip incl. TOTP) |
|
||||
| created_at / used_at | DATETIME | `used_at` stamped at exchange |
|
||||
|
||||
`mobile_auth_codes` — one row per completed SSO callback (the code the app redeems):
|
||||
|
||||
| col | type | notes |
|
||||
|---|---|---|
|
||||
| id | INT PK AUTO_INCREMENT | |
|
||||
| code_hash | CHAR(64) UNIQUE | sha256 hex of the opaque ≥128-bit code; the raw code never touches the DB |
|
||||
| user_id | INT NOT NULL FK→users(id) ON DELETE CASCADE | the authenticated account |
|
||||
| session_id | CHAR(36) NOT NULL | the owning `mobile_auth_sessions.session_id` (ties the code to its PKCE challenge) |
|
||||
| expires_at | DATETIME NOT NULL | very short (~5 min) |
|
||||
| used_at | DATETIME NULL | set on first successful exchange — **single use** (a reused code fails) |
|
||||
| created_at | DATETIME | |
|
||||
|
||||
Both self-prune (indexed `expires_at`): a best-effort sweep runs at boot beside the existing
|
||||
`revoked_sessions` prune, and each bridge write opportunistically deletes expired rows — so no cron
|
||||
infra is added (same approach as `revoked_sessions`).
|
||||
|
||||
**`mobile_refresh_tokens` additions (M9).** Two nullable columns are added to support the device
|
||||
list/revoke surface: `device_name VARCHAR(100) NULL` (a friendly label) and `last_used_at DATETIME
|
||||
NULL` (bumped on each refresh). Existing rows get them via the schema's ALTER section; the token model
|
||||
is otherwise unchanged.
|
||||
|
||||
### trusted_devices — MFA "Trust this device"
|
||||
|
||||
Lets a browser/app **skip the TOTP step** at login (never the password) for 30 days. Pattern-identical
|
||||
to `mobile_refresh_tokens`: the opaque trust token lives client-side (the `rg_trust` httpOnly cookie on
|
||||
web, `X-Trust-Token` / EncryptedSharedPreferences on native) and only its **sha256** hash is stored
|
||||
(`token_hash CHAR(64) UNIQUE`) — sha256, not bcrypt, because a 256-bit random token is looked up **by
|
||||
its hash** via the unique index (a per-row salt would break that). Columns mirror the mobile table
|
||||
(`platform`, `device_name`, `device_hash`, `user_agent`, `created_at`, `last_used_at`, `expires_at`,
|
||||
`revoked_at`). Capped at 10 rows/user **in application code — no silent pruning** (an over-cap trust is
|
||||
refused so the client can prompt the user to revoke one first). Consulted only at the login/password
|
||||
step, never at token refresh, and revoked wholesale on untrust / password change / password reset /
|
||||
TOTP disable. See `docs/website/TRUSTED_DEVICES_MFA.md`.
|
||||
|
||||
### recovery_codes — single-use MFA backup codes
|
||||
|
||||
Generated at TOTP enrollment (10 at a time, shown to the user **once**) so a user who loses their
|
||||
authenticator can complete login without an admin reset. `code_hash VARCHAR(72)` is a **bcrypt** hash
|
||||
(not sha256): a recovery code is a human-typed, lower-entropy fallback credential — the closest
|
||||
analogue to a password — and there is no hash-lookup constraint (verification fetches the user's ≤10
|
||||
unused rows and `bcrypt.compare`s each, like password verification). `used_at` is the single-use
|
||||
marker. Cleared wholesale on TOTP disable / password change / password reset.
|
||||
|
||||
### shard_ruleset — the shard's published ruleset (Protocol 3.0)
|
||||
|
||||
Singleton row (`id = 1`, CHECK-constrained) holding the latest `world.ruleset` frame: `rev`,
|
||||
`expansion`, `payload` JSON (the whole frame), `t`, `updated_at`. The shard re-emits the complete
|
||||
ruleset on every sidecar connect, so this is an **overwrite, not an append** — and the kind is
|
||||
deliberately **not** in `LOGGED_KINDS`, since logging it would put a duplicate row in `shard_events`
|
||||
on every reconnect while `server.hello` already marks each of those.
|
||||
|
||||
The frame is stored whole rather than normalized into columns: it is a flat description of server
|
||||
config that is read as one page, so splitting it up would mean a schema change every time the shard
|
||||
grows a new block. `rev` (the shard's FNV-1a of the body) and `expansion` are hoisted only because
|
||||
they are cheap to display — the same payload-plus-hoisted-columns shape `shard_champs` uses.
|
||||
|
||||
**No row means the shard has never published one** (an older plugin, or `Bridge.RulesetEnabled=false`),
|
||||
served as `null` rather than `{}`: "not published yet" and "published, everything off" are different
|
||||
answers and the page renders them differently.
|
||||
|
||||
### shard_points_boards — points / loyalty leaderboards (Protocol 3.0)
|
||||
|
||||
One row per point system, keyed by the shard's own `PointsType` name (`QueensLoyalty`,
|
||||
`CleanUpBritannia`, …). The shard carries ~25 of these, each a standing players build over months.
|
||||
Columns: `system` (PK), `name`, `name_cliloc`, `max_points`, `players`, `show_on_gump`, `payload` JSON
|
||||
(the whole `points.board` frame), `t`, `updated_at`.
|
||||
|
||||
**The top-N list stays inside `payload`** rather than being normalized into a `shard_points_entries`
|
||||
table. It is a fixed-size list (10 by default) that is only ever read whole — exactly like
|
||||
`shard_governors.candidates` — so normalizing buys nothing until something needs a per-character
|
||||
reverse lookup, and a character's own standings already ride inside `char.profile` instead.
|
||||
|
||||
Board state, not events: `points.board` is **not** in `LOGGED_KINDS`, for the same reason
|
||||
`guild.update` isn't. The shard emits a frame every time anyone's score moves a top ten, so logging
|
||||
would grow `shard_events` without bound for something whose only interesting value is its latest
|
||||
version. There is also **no delete path** — the shard's set of systems is fixed at startup, so there is
|
||||
no `points.remove` to mirror.
|
||||
|
||||
Two values carry non-obvious meanings, both set by the plugin and both documented in
|
||||
[`link/INTEGRATION.md`](../link/INTEGRATION.md) §4:
|
||||
|
||||
- **`max_points = 0` means uncapped**, and on a real shard that is the *common* case (ServUO's
|
||||
uncapped idiom is `double.MaxValue`, which the plugin normalises to 0). Anything rendering
|
||||
`points / max_points` must special-case it.
|
||||
- **`name` is usually NULL**, with `name_cliloc` set instead — most systems name themselves with a
|
||||
cliloc rather than a literal. Listing therefore orders by `COALESCE(name, system)`, so boards
|
||||
awaiting cliloc resolution sort by their own key rather than clumping together under NULL.
|
||||
|
||||
### shard_vendors / shard_vendor_items — the player-vendor marketplace (Protocol 3.0)
|
||||
|
||||
The shard-wide shop index, fed by `vendor.listing` / `vendor.listing.remove`. One row per player
|
||||
vendor and one per priced listing. Full operator detail in [`MARKETPLACE.md`](MARKETPLACE.md); the
|
||||
design is `docs/link/v3.md` §8.
|
||||
|
||||
| Table | Shape |
|
||||
|---|---|
|
||||
| `shard_vendors` | `serial` (PK), `shop_name`, `owner_serial`, `owner_name`, `map`/`x`/`y`/`z`, `region`, `house`, `item_count`, `item_total`, `truncated`, `t`, `updated_at`. Indexes on owner, map, region and `updated_at`. |
|
||||
| `shard_vendor_items` | `id` (PK), `vendor_serial`, `serial`, `item_id`, `hue`, `amount`, `price`, `name`, `cliloc`, `display_name`, `child`. Indexes on `vendor_serial`, `price`, `item_id`, `display_name`, and `(display_name, price)`. |
|
||||
|
||||
**Ingest is per-vendor and authoritative**: the frame is the whole shop, so ingest is
|
||||
delete-then-insert of that vendor's listings inside one transaction. All-or-nothing matters
|
||||
specifically because the two writes are "the shop" and "what is in it" — a failure between them
|
||||
leaves a shop advertising an inventory it no longer has, which is visibly wrong and indistinguishable
|
||||
from a genuinely empty shop. No foreign keys, consistent with every other `shard_*` table.
|
||||
|
||||
**There is deliberately no `payload` column**, unlike `shard_points_boards` directly above. The
|
||||
board's top-N is a fixed-size list read whole, so it lives in JSON; here the items *are* the
|
||||
searchable rows, so they are normalized and nothing is left worth duplicating. The sidecar keeps the
|
||||
whole blob — outage resilience is its job, search is ours.
|
||||
|
||||
Market state, not events: neither kind is in `LOGGED_KINDS`, and this is the strongest case of the
|
||||
three v3 kinds. One frame carries up to 250 listings and the sweep re-emits a shop on any price
|
||||
change, so logging would turn `shard_events` into a price history nobody reads.
|
||||
|
||||
Two columns carry non-obvious meanings:
|
||||
|
||||
- **`item_count` vs `item_total`.** `item_count` is what the frame published; `item_total` is what
|
||||
the shop actually holds. They differ when `truncated` — the shard caps listings per frame
|
||||
(`Bridge.MarketMaxListings`, 250 by default), and a commodity reseller with thousands of stacks
|
||||
genuinely exceeds it. Any UI must show both or it presents a partial shop as complete.
|
||||
- **`display_name` is denormalized at ingest**, resolved from the item's literal `name` (preferred —
|
||||
a player set it, so it is more specific) else its `cliloc` against `shard_clilocs`. Resolving at
|
||||
query time would put the cliloc table on the hot path and make search-by-name impossible. Because
|
||||
the shard's diff sweep will not re-send an unchanged shop just because the site learned what its
|
||||
items are called, **a cliloc import triggers a bulk re-resolution** of this column (after a boot
|
||||
import and after an admin import; ~50 ms per thousand rows, never throws).
|
||||
|
||||
`updated_at` is written explicitly on every upsert rather than left to `ON UPDATE CURRENT_TIMESTAMP`,
|
||||
which MariaDB does not fire when every column is written back unchanged. A shop re-published
|
||||
identically is still *freshly confirmed*, and without this the staleness banner would age a perfectly
|
||||
current shop forever.
|
||||
|
||||
### shard_feature_visibility — per-feature audience config (Protocol 3.0)
|
||||
|
||||
One row per shard feature: `feature` (PK), `enabled`, `audience` (a rung on the ladder in §6.5),
|
||||
`stream` (whether the feature's kinds fan out over SSE at all), `field_rules` JSON (`{field: rung}`
|
||||
for the sensitive fields only), `updated_by`, `updated_at`.
|
||||
|
||||
**An absent row means "use the compiled default", and the compiled defaults reproduce pre-3.0
|
||||
behavior — so an empty table is a no-op and there is nothing to seed.** Stored rows are merged over
|
||||
the defaults on read, which is also where the invariants are re-applied: a row naming an unknown
|
||||
feature is ignored (a stale row must not resurrect a removed feature), an invalid rung falls back to
|
||||
the default rather than failing open, and a rule touching a locked field (`acct` / `webId`) is
|
||||
discarded. See §6.5.
|
||||
|
||||
### shard_spawn_* / shard_regions / shard_landmarks / shard_champion_spawns / shard_atlas_meta — the spawn atlas (Protocol 3.0)
|
||||
|
||||
Static shard **content**, not live shard state. Nothing here comes from the sidecar: the atlas is
|
||||
derived from the shard's own ServUO tree, re-read on **every server boot** and hash-gated so an
|
||||
unchanged tree costs one read pass and no write. Nothing is precomputed and committed — a shard's
|
||||
maps change over its life, and a snapshot in the repo would silently drift from the world players
|
||||
actually see. These tables stay populated whether the shard is up or not. Full operator detail in
|
||||
[`SPAWN_ATLAS.md`](SPAWN_ATLAS.md); the design is `docs/link/v3.md` §6.
|
||||
|
||||
**No facet name appears anywhere in the code.** A shard may add facets, replace them, or rename them
|
||||
when its maps are updated; the facet set is discovered from the tree, and the loose spellings in
|
||||
`Data/Locations` are matched against it rather than looked up in a table.
|
||||
|
||||
| Table | Key columns |
|
||||
|---|---|
|
||||
| `shard_spawn_creatures` | `slug` PK, `name`, `total`, `points`, `facets` JSON, `art` NULL |
|
||||
| `shard_spawn_points` | `id` PK, `facet`, `name`, `x`, `y`, `width`, `height`, `spawn_range`, `max_count`, `min_delay`, `max_delay`, `tod_start/end/mode`, `region`, `landmark`, `label` |
|
||||
| `shard_spawn_point_types` | `(point_id, slug)` PK, `max_count` |
|
||||
| `shard_regions` | `facet`, `name`, `type`, `priority`, `parent`, `rects` JSON |
|
||||
| `shard_landmarks` | `facet`, `name`, `grp`, `x`, `y`, `z` |
|
||||
| `shard_champion_spawns` | `slug` PK, `name`, `grp`, `type`, `random_type`, `facet`, `x`, `y`, `z`, `radius`, `label` |
|
||||
| `shard_atlas_meta` | Singleton (`id = 1`), `payload` JSON (counts, a sha256 per source file, `parserVersion`), `imported_at` |
|
||||
| `shard_atlas_pending` | Singleton (`id = 1`), `status` (`pending`/`rejected`), `payload` JSON, `detected_at` |
|
||||
|
||||
The first seven are **import-owned**: a refresh empties and reloads every one inside a single
|
||||
transaction, so a failed reload leaves the previous atlas intact rather than a half-loaded world.
|
||||
Nothing else writes to them, and nothing holds a foreign key to them — no FKs at all, consistent with
|
||||
every other `shard_*` table.
|
||||
|
||||
**`shard_atlas_pending` is the security-relevant one.** A refresh that would REMOVE a facet is never
|
||||
applied automatically: facet loss is indistinguishable at boot from a half-copied or mid-update tree,
|
||||
so it is staged here for an admin to approve or reject, and **startup is never blocked by it**. Only
|
||||
the decision is stored — source hashes plus the facet diff, a few KB — and approving re-parses the
|
||||
tree, so a multi-megabyte blob never lands in the database and what gets applied matches the tree at
|
||||
approval time. A rejection is remembered against those exact hashes so a declined refresh does not
|
||||
re-prompt on every restart. Everything else (new facets, renamed regions, changed spawns) applies
|
||||
immediately, since none of it can destroy data an operator would miss.
|
||||
|
||||
The boot refresh is **best-effort by contract**: no configured path, an unreadable mount, a malformed
|
||||
file or a database error is caught and logged, and the site comes up serving whatever atlas it had.
|
||||
The tree path comes from the `spawn_atlas_servuo_path` setting, falling back to `SERVUO_PATH`.
|
||||
|
||||
**A refresh re-derives when the tree changed OR the parser did.** `spawnAtlasSource.PARSER_VERSION`
|
||||
is stored in `shard_atlas_meta` beside the source hashes and bumped whenever the parser produces
|
||||
different data from identical files. Hashing the tree alone would strand an install whose maps never
|
||||
change on whatever an older build derived — a corrected parse would ship and never reach the data.
|
||||
|
||||
Four column choices worth stating, because each one is a trap:
|
||||
|
||||
- **`spawn_range`, not `range`**, and **`grp`, not `group`** — both are reserved words.
|
||||
- **`DELETE`, not `TRUNCATE`.** `TRUNCATE` is DDL in MariaDB and implicitly commits, which would
|
||||
defeat the all-or-nothing reload. At ~7k rows the difference does not matter.
|
||||
- **Point ids are assigned explicitly**, not left to `AUTO_INCREMENT`: the `shard_spawn_point_types`
|
||||
rows need to know them, and `conn.batch()` reports no usable `insertId` for a multi-row insert.
|
||||
- **Plain `INDEX` on `name`, deliberately not `FULLTEXT`.** ~800 creature rows makes a `LIKE` scan
|
||||
free, and FULLTEXT's minimum token length would break searches for names like "orc".
|
||||
|
||||
`shard_champion_spawns` is the *configured* altar roster ("there is an Unholy Terror altar in
|
||||
Deceit"). The live `champ.update` feed in `shard_champs` is the separate answer to "it is on level 3
|
||||
right now". Both exist; they are not the same data.
|
||||
|
||||
**`shard_spawn_creatures.art` is always NULL on a fresh import.** The project ships no creature
|
||||
artwork: sprites live in the operator's own client `.mul`/`.uop` files and are theirs, not ours to
|
||||
redistribute. An operator supplies art via a gitignored map plus images under the (already
|
||||
gitignored) `server/uploads/atlas/`. Text-only is the normal, supported state.
|
||||
|
||||
### shard_clilocs / shard_cliloc_meta — UO's localization table (Protocol 3.0)
|
||||
|
||||
Items on the wire carry a `LabelNumber`, not a name. The bridge has always sent it —
|
||||
`char.profile.equipment.cliloc`, reward titles as a cliloc number in string form, and one per
|
||||
marketplace listing — but with no table to resolve it against, the character sheet could only render
|
||||
`id 1023721` where the game renders "quarter staff".
|
||||
|
||||
| Table | Shape |
|
||||
|---|---|
|
||||
| `shard_clilocs` | `number` INT PK, `flag`, `text` TEXT |
|
||||
| `shard_cliloc_meta` | Singleton (`id = 1`), `payload` JSON (source file, sha256, count, `parserVersion`), `imported_at` |
|
||||
|
||||
Import-owned and all-or-nothing in one transaction, same contract as the atlas — including **`DELETE`,
|
||||
not `TRUNCATE`**, for the same reason.
|
||||
|
||||
**Sourced from files the operator supplies**, at a path from the `cliloc_client_path` setting falling
|
||||
back to `UO_CLIENT_PATH`. Nothing client-derived is committed: UO's strings are EA's, exactly as the
|
||||
creature sprites are. A shard with nothing configured is fully supported — names render as ids. Full
|
||||
design and operator guide: [`CLILOCS.md`](CLILOCS.md).
|
||||
|
||||
**It reads a SET of sources, not one file**, because shards edit items and add new ones and those
|
||||
carry cliloc ids no stock client table has. A base (the converted client table) plus every overlay
|
||||
under `custom/` are re-read on every boot and hash-gated **together**, exactly as the atlas re-reads
|
||||
`Regions.xml` + `Locations/*.xml` + `Spawns/*.xml` + `ChampionSpawns.xml`. Later sources win, so an
|
||||
overlay both adds ids and overrides stock ones, and adding one custom item never means re-exporting a
|
||||
5 MB client file. Scale, measured on the live shard: its script tree references 16,434 cliloc ids and
|
||||
only 37 are absent from stock — tens of entries against a 67k base, which is why this is an overlay
|
||||
and not a second table.
|
||||
|
||||
The conversion step is not avoidable: **every current client ships its cliloc files compressed**
|
||||
(first DWORD's high byte `0x8E`), and ServUO's own bundled `Ultima.StringList` cannot read that
|
||||
either — so the shard cannot supply names on our behalf. The plain layout and a delimited text export
|
||||
are both accepted, sniffed by header rather than extension.
|
||||
|
||||
Three decisions worth stating:
|
||||
|
||||
- **`text` is TEXT, not VARCHAR.** Long property descriptions reach 12 KB. The index that matters for
|
||||
marketplace search is the denormalized `shard_vendor_items.display_name`, not this table.
|
||||
- **Blank entries are dropped at import** — 123,490 parsed → **67,496** stored. Roughly half a cliloc
|
||||
table is empty strings for ids the client reserves and never uses; a row that resolves to no name is
|
||||
indistinguishable from no row at all, and dropping them makes the binary and text imports converge
|
||||
on identical content.
|
||||
- **Two refusals, one of them the atlas's.** A corrupt source fails the parse on a truncated record,
|
||||
so it is caught outright and leaves the previous table serving. But a source that has **vanished**
|
||||
parses perfectly and imports a table quietly missing everything it contributed — an unmounted volume
|
||||
and a deliberate deletion are indistinguishable from here, which is precisely the ambiguity the
|
||||
atlas stages a facet removal for. So it is escalated: `status: 'needsReview'`, nothing applied,
|
||||
`missingSources` reported by both the import and `status()`, and an admin accepts it with
|
||||
`{approve:true}`. That is a flag rather than the atlas's approve/reject pair because the atlas
|
||||
stores a pending decision so that approving **re-parses** the tree; here nothing is stored, so
|
||||
re-reading at approval time is automatic.
|
||||
|
||||
**Resolution is server-side and there is no public route.** The table is never served *as* a table:
|
||||
67k rows would dwarf any page using them, and the Android client consumes the same already-resolved
|
||||
JSON. `resolveMany()` returns only ids that resolved to something displayable — placeholders like
|
||||
`~1_val~` are stripped, since the bridge sends the id and never the property packet that carries the
|
||||
arguments — and it never throws, because a cliloc lookup is decoration on a character sheet.
|
||||
|
||||
---
|
||||
|
||||
## 4. API contract
|
||||
@@ -595,50 +163,11 @@ arguments — and it never throws, because a cliloc lookup is decoration on a ch
|
||||
Base path `/api/v1`. JSON in/out. Auth via httpOnly cookie (`isLoggedIn` reads it; also
|
||||
accepts `Authorization: Bearer` for API testing).
|
||||
|
||||
### 4.0 The authoritative route list
|
||||
|
||||
The prose tables below are **orientation for a human reader** and can drift. Two generated artifacts
|
||||
are authoritative, and they answer different questions:
|
||||
|
||||
| Artifact | Source of truth for | Generated by |
|
||||
|---|---|---|
|
||||
| `server/routes.manifest.json` — mirrored as [api-route-inventory.json](./api-route-inventory.json) | **What URLs exist.** 215 public routes + 2 on the internal listener, sorted, method + path only. | `npm run routes:manifest`, by walking the live Express stack |
|
||||
| `server/swagger/swagger-output.json` — served at `/api/docs` | **What each route means.** Parameters, bodies, response codes, security. | `npm run swagger`, from `#swagger.*` annotations |
|
||||
|
||||
The split is deliberate: Swagger is annotation-derived, so an unannotated route is invisible in it and
|
||||
it churns whenever a description is reworded — it documents *intent*. The manifest is introspection-
|
||||
derived and records *reality*, which is why it, not Swagger, is the thing PR checks freeze
|
||||
(`npm run routes:manifest -- --check`).
|
||||
|
||||
Both artifacts are emitted with **sorted** keys, so a diff in either is proportional to the change
|
||||
rather than to how the routers happen to be traversed. `swagger.js` additionally strips trailing
|
||||
slashes from generated path keys — see *Regenerating the spec* in the website README for why the
|
||||
domain split makes that necessary.
|
||||
|
||||
Scope: the manifest keeps `/api/**` and `/.well-known/**` from the public app plus everything on the
|
||||
internal listener. The SPA catch-all, `/uploads` and `/brand` are filesystem-conditional static
|
||||
mounts — not API contract, and including them would make the output depend on whether CI had built
|
||||
the client.
|
||||
|
||||
A third generated file, `server/routes.guards.json`, is a **review aid and not a contract**: per route,
|
||||
the middleware handler count plus the *named* middleware on its mount chain. It exists because a
|
||||
router-level `router.use(noindex, isLoggedIn, staffOnly)` gate never appears in an individual route's
|
||||
own stack, so a capability router extracted without re-applying the gate would otherwise publish
|
||||
authenticated endpoints silently. Names are a hint only — `requireRole(...)` returns an anonymous
|
||||
arrow and cannot be observed — but a *missing* `requireAuth` is unambiguous, and the server test suite
|
||||
asserts every `/admin/**` and `/player/**` route still carries it.
|
||||
|
||||
### /auth (auth/index.js → the capability routers in §2)
|
||||
|
||||
No group gate — `/auth` is where an anonymous caller becomes authenticated. The authenticated parts
|
||||
gate themselves: `me.routes.js` and `notifications.routes.js` each apply `noindex, requireAuth` at
|
||||
their own router level, and `/sso/:provider/link` carries `requireAuth` per route.
|
||||
|
||||
### /auth (auth.routes.js → auth.controller.js)
|
||||
| Method | Path | Auth | Body | Purpose |
|
||||
|---|---|---|---|---|
|
||||
| POST | `/login` | — (rate-limited) | `{username,password}` | verify, set cookie, log `auth.login`, update `last_login_at`. If the account has TOTP **and this browser is a trusted device** (a valid `rg_trust` cookie bound to the user), the TOTP step is **skipped** and a session is issued directly (logs `auth.login.trusted_device`). Otherwise a 2FA account returns `{totpRequired, challenge}`. |
|
||||
| POST | `/login/totp` | — (rate-limited) | `{challenge, code? \| recoveryCode?, trustDevice?, deviceName?}` | complete 2FA with a TOTP **or** single-use recovery code. `trustDevice` sets the `rg_trust` cookie so future logins skip TOTP; at the device cap the session is still issued and the body carries `{trustLimitReached, devices}`. |
|
||||
| POST | `/logout` | cookie | — | clear cookie (the `rg_trust` trust cookie deliberately **survives** logout) |
|
||||
| POST | `/login` | — (rate-limited) | `{username,password}` | verify, set cookie, log `auth.login`, update `last_login_at` |
|
||||
| POST | `/logout` | cookie | — | clear cookie |
|
||||
| GET | `/me` | cookie / bearer | — | current user (no hash) or 401 — client bootstraps auth state |
|
||||
| POST | `/password/forgot` | — (rate-limited) | `{email}` | email a single-use, ~1h reset link to **every active account** on the address; **always** returns the same generic 200 (no account enumeration). Email is non-unique, so several accounts may each get a link naming their username. Logs `account.password.reset.request`. |
|
||||
| GET | `/password/reset/:token` | — | — | validate a link → `{username}` for the form, else 404 (never distinguishes expired/used/never-existed) |
|
||||
@@ -646,17 +175,8 @@ their own router level, and `/sso/:provider/link` carries `requireAuth` per rout
|
||||
| GET | `/me/account` | cookie / bearer | — | full self account (`id, username, role, email, status, totp_enabled, has_password`) |
|
||||
| PATCH | `/me/account/username` | cookie / bearer (rate-limited) | `{username}` | change own username; re-issues the caller's session |
|
||||
| PATCH | `/me/account/password` | cookie / bearer (rate-limited) | `{newPassword, currentPassword?}` | change/set own password (current required unless the account has none); revokes other sessions, keeps the caller's |
|
||||
| POST | `/me/account/totp/setup` · `…/enable` · `…/disable` | cookie / bearer | `{code}` on enable/disable | self 2FA enrollment (disable needs a valid current code, not a password). **enable** returns the one-time `recoveryCodes`; **disable** clears the user's trusted devices + recovery codes |
|
||||
| POST | `/me/account/totp/setup` · `…/enable` · `…/disable` | cookie / bearer | `{code}` on enable/disable | self 2FA enrollment (disable needs a valid current code, not a password) |
|
||||
| GET | `/me/account/identities` · DELETE `…/:provider` | cookie / bearer | — | list / unlink own SSO identities |
|
||||
| GET | `/me/trusted-devices` | cookie / bearer | — | list own active trusted devices (never tokens) |
|
||||
| POST | `/me/trusted-devices` | cookie / bearer (rate-limited) | `{deviceName?}` | trust the current device; web gets an httpOnly `rg_trust` cookie, native gets `{trustToken}`. **409 `{error:'trusted_device_limit', devices}`** at the cap |
|
||||
| DELETE | `/me/trusted-devices` · `…/:id` | cookie / bearer | — | untrust all / one (ownership-scoped) |
|
||||
| GET | `/me/account/recovery-codes/status` | cookie / bearer | — | remaining unused code count (never the codes) |
|
||||
| POST | `/me/account/recovery-codes/generate` | cookie / bearer (rate-limited, **password step-up**) | `{currentPassword?}` | regenerate the one-time recovery codes (returned once); refused when 2FA is off |
|
||||
| POST | `/me/devices` | cookie / bearer | `{endpoint, transport?, platform?}` | register a push endpoint; **rejects a disallowed endpoint 400** (SSRF guard). Idempotent per (user, endpoint) |
|
||||
| GET | `/me/devices` · DELETE `…/:id` | cookie / bearer | — | list / unregister own push devices |
|
||||
| GET | `/me/notifications/streams` | cookie / bearer | — | the subscribable catalog (`personal`/`requiresLinkedAccount` flags) |
|
||||
| GET · PUT | `/me/notifications/subscriptions` | cookie / bearer | `{streams:[id]}` on PUT | get / replace own opted-in streams (unknown ids dropped) |
|
||||
|
||||
**Role-agnostic self-service (`/auth/me/*`).** The canonical "me" surface for **every** authenticated
|
||||
role. It reuses the exact `account.controller` handlers as `/player/account/*` and `/admin/account/*`
|
||||
@@ -665,16 +185,6 @@ lets a client (the Android app) manage its own account through one surface witho
|
||||
`/admin` (docs/android/PLAN.md §6.4). The older `/player/account/*` + `/admin/account/*` routes stay
|
||||
for web back-compat.
|
||||
|
||||
**The `/player/*` group is self-service, not player-only.** Staff are a **superset** of players — every
|
||||
player ability plus their staff tools on top — so the whole group (`account.router.js`,
|
||||
`shard.router.js`, `appeals.router.js`, mounted by `player/index.js`) sits behind the shared
|
||||
`noindex, requireAuth` gate **only**, never `requireRole('player')`. Every handler is self-scoped to the caller by `req.user.id`, so an admin/editor/
|
||||
moderator using it sees only their **own** linked accounts and characters (with the pre-existing
|
||||
`isAdmin` bypass still letting a genuine admin read *any* character). Staff also reach the identical
|
||||
self-scoped handlers under `/admin/shard/*` (same controller) for the web admin surface; the two are
|
||||
interchangeable. This is why a staff account with linked game characters gets its "My characters" and
|
||||
personal notification streams on the mobile client — the group no longer 403s a non-`player` role.
|
||||
|
||||
**Password reset.** Uses the same audited pattern as `user_invites`: an opaque 32-byte token
|
||||
whose **sha256 hash only** is stored in `password_resets`, single-use and short-lived (~1h). It
|
||||
also serves SSO-only accounts (null `password_hash`) as their "set an initial password" path. The
|
||||
@@ -682,103 +192,10 @@ reset link points at the web front end (`/account/reset/:token`); the Android ap
|
||||
rather than shipping its own reset screen (docs/android/PLAN.md §4.2). First admin is bootstrapped
|
||||
by `seed.js` from env (see §6); further staff are created under `/admin/users` or via email invites.
|
||||
|
||||
**Push notifications (M7, opt-in).** The app subscribes per stream (`/auth/me/notifications/*`) and
|
||||
registers device endpoints (`/auth/me/devices`); nothing is pushed unless subscribed. Delivery is a
|
||||
**content-free tickle** — `{ stream, ref }`, no sensitive data — POSTed to each subscribed device's
|
||||
self-hosted **ntfy** endpoint (`utils/pushDispatch`); the app wakes and pulls the real, ownership-
|
||||
checked content over the authenticated API. Two producers fan out through the one publisher: the shard
|
||||
ingest dispatcher (`utils/shardIngest`, beside the SSE broadcast) for shard-derived streams, and the
|
||||
create/publish-post path for `news.post`. The stream catalog + event→stream mapping is
|
||||
`config/notificationStreams.js`. Security invariants:
|
||||
- **Same public/admin split as the SSE feed.** Public streams are drawn *only* from the SSE
|
||||
`PUBLIC_KINDS` allowlist; a sensitive kind (audit/cheat/IP/login-attempt) can never produce a public
|
||||
push.
|
||||
- **Personal streams are owner-keyed.** `vendor.sale` / `house.idoc` / `account.login` are delivered
|
||||
only to the *owning* user's devices, resolved via `shardLinks` (the same ownership check as
|
||||
`/player/shard/*`).
|
||||
- **SSRF guard.** A device `endpoint` is a client-supplied URL the server POSTs to, so registration and
|
||||
every publish validate it is HTTPS, non-private/loopback, and (when configured) on the shard's ntfy
|
||||
allow-set (`NTFY_BASE_URL` / `NTFY_ALLOWED_ORIGINS`).
|
||||
- ntfy is treated as an **untrusted relay** — no per-user accounts, unguessable topics; an optional
|
||||
`NTFY_PUBLISH_TOKEN` hardens backend→ntfy publishes but is not required. See docs/android/PLAN.md §11.
|
||||
|
||||
### Mobile SSO Authorization Bridge (`/auth/mobile/sso/*`, M9)
|
||||
|
||||
Native "Sign in with Google/Discord" for the Android app **without shipping any OAuth secret in the
|
||||
app**. The website stays the identity authority: each shard owner's provider credentials live in
|
||||
`auth_providers` (encrypted at rest) and are only ever used server-side. The bridge is a **new
|
||||
consumer of the existing SSO + mobile-bearer machinery**, not a parallel auth path — it reuses the
|
||||
`/auth/sso/:provider/*` redirect flow, the link-only + opt-in-provisioning policy, the TOTP gate, and
|
||||
issues the **same** token pair as `/auth/mobile/login`.
|
||||
|
||||
The TOTP gate it reuses includes the **trusted-device skip** (see
|
||||
`TRUSTED_DEVICES_MFA.md` §6). Because the app opens this flow in a Custom Tab, which shares the
|
||||
system browser's cookie jar, the `rg_trust` cookie set on the TOTP form is presented back on the next
|
||||
app sign-in — so "don't ask me again" works for native SSO without the app injecting a header into a
|
||||
tab it does not control, and without a trust token ever appearing in a start URL.
|
||||
|
||||
| Method | Path | Auth | Body / Query | Purpose |
|
||||
|---|---|---|---|---|
|
||||
| GET | `/auth/providers` | — | — | **reused** discovery; the app renders provider buttons from this (never exposes secrets) |
|
||||
| GET | `/auth/mobile/sso/start` | — (rate-limited per-IP + per-provider) | `?provider&code_challenge&state&redirect_uri` | validate provider enabled + `redirect_uri` **exact-match** allowlist; insert a `mobile_auth_sessions` row; create the existing `sso_tx` tagged `mode:'mobile'` carrying `session_id`; **302 to the IdP** (existing authorize URL) |
|
||||
| GET | `/auth/sso/:provider/callback` | — (signed `sso_tx`) | `?code&state` | **existing** endpoint; a new branch when `tx.mode==='mobile'`: resolve the account (same policy as web login incl. TOTP), mint a single-use hashed authorization code into `mobile_auth_codes`, mark the session `completed`, and **302 to `redirect_uri?code=…&state=…`** (the app's original `state`) — **no cookie is set** |
|
||||
| POST | `/auth/mobile/sso/exchange` | — (rate-limited per-IP) | `{code, code_verifier}` | validate the code exists / unexpired / unused (mark used) and `sha256(code_verifier)` matches the stored challenge → issue the existing mobile access + refresh pair (`createMobileSession`) → `{accessToken, refreshToken, expiresIn, user}`. When the session carries `trust_device`, also mint a `platform:'mobile'` trusted device and add `trustToken` — minted here, on an authenticated app→server call, so it never travels in the deep link. Best-effort: at the trusted-device cap the response simply omits it rather than failing the sign-in |
|
||||
| POST | `/auth/mobile/refresh` | — | `{refreshToken}` | **reused** unchanged — rotate the pair |
|
||||
| POST | `/auth/mobile/logout` | bearer | `{refreshToken?, all?}` | **reused** unchanged — revoke this (or all) refresh token(s) |
|
||||
| GET | `/auth/me/sessions` · DELETE `…/:id` | cookie / bearer | — | list / revoke own **mobile sessions** (device_name, last_used_at, created_at) — the "Active Devices" surface (distinct from `/auth/me/devices`, which is push endpoints) |
|
||||
|
||||
**Two PKCE layers (do not conflate).**
|
||||
- *Layer A (existing):* website ↔ IdP. The `code_verifier` is generated at `/start`, kept only in the
|
||||
httpOnly `sso_tx` cookie, sent to the IdP token endpoint at the callback. Unchanged.
|
||||
- *Layer B (new):* app ↔ website. The **app** generates `code_verifier`/`code_challenge`; the
|
||||
challenge is stored in `mobile_auth_sessions` at `/start`; the verifier is presented at `/exchange`.
|
||||
This is what stops an intercepted callback code from being redeemed by anyone but the real app.
|
||||
|
||||
**State / CSRF.** The app-generated `state` is stored at `/start`, echoed on the callback redirect,
|
||||
and **verified by the app** before it calls `/exchange` — a CSRF guard independent of both PKCE
|
||||
layers (a different app instance triggering `/start` cannot complete someone else's flow).
|
||||
|
||||
**Redirect-URI allowlist.** `/start` and the callback validate `redirect_uri` by **exact match**
|
||||
against a configured allowlist (`MOBILE_AUTH_REDIRECT_URIS`, default the one fixed application-owned
|
||||
callback `runicgateway://auth/callback`) — **never prefix match** (prefix matching on custom schemes
|
||||
is a known open-redirect vector). Tokens are **never** placed in the callback URL — only the
|
||||
short-lived authorization code.
|
||||
|
||||
*App Links (implemented).* When the admin toggle `mobile_app_links_enabled` is **on**, `/start` also
|
||||
accepts the self-origin HTTPS callback `https://<request-host>/mobile/callback` — one *additive*
|
||||
exact-match entry, derived from the request/`APP_BASE_URL` and never from client input; the
|
||||
custom-scheme allowlist is never narrowed. The shard then auto-serves `GET
|
||||
/.well-known/assetlinks.json` (fixed package `com.runicgateway.app` + `MOBILE_APP_CERT_SHA256`
|
||||
fingerprints; 404 when the toggle is off or no fingerprint is configured), and
|
||||
`settings.getPublic()` advertises `mobileAppLinks: <bool>`. These two things — one static file route
|
||||
and one more allowlist entry — are the *entire* server surface App Links require. See
|
||||
docs/android/APP_LINKS.md.
|
||||
|
||||
**TOTP through the bridge.** A 2FA account keeps full parity: the callback stages the existing
|
||||
pending-TOTP cookie (now also carrying the bridge `session_id`) and bounces the Custom Tab through the
|
||||
web TOTP form; on a correct code the completion mints the authorization code and deep-links back to
|
||||
the app — it never mints a session cookie for a mobile flow.
|
||||
|
||||
**Revocation latency (documented tradeoff).** Revoking a refresh token (device revoke / logout) stops
|
||||
future renewals but does **not** invalidate an already-issued access token until it expires — up to
|
||||
the access-token lifetime (`MOBILE_ACCESS_TTL`, default 15 min) of continued access. This is an
|
||||
accepted tradeoff given the short lifetime. If instant revocation is ever required, add an
|
||||
access-token (jti) blocklist check on the `requireAuth` path — the same `revoked_sessions` mechanism
|
||||
web sessions already use.
|
||||
|
||||
**Authorization code.** Cryptographically random, ≥128 bits, stored **hash-only**, single-use, short
|
||||
expiry (~5 min); `/exchange` is rate-limited per-IP. The bridge tables self-prune (§3).
|
||||
|
||||
### /public (public/index.js → the capability routers in §2) — all GET except `/contact`, no auth
|
||||
|
||||
**No group gate, deliberately.** This surface is anonymous by design: the SPA renders it logged-out,
|
||||
the Discord bot reads it with no credentials, and the Android `ShardStreamClient` consumes
|
||||
`/public/shard/stream` without an `Authorization` header. Content visibility during maintenance comes
|
||||
from the per-route **siteMode** middleware (§5), never from an auth gate.
|
||||
|
||||
### /public (public.routes.js → public.controller.js) — all GET, no auth
|
||||
| Method | Path | Notes |
|
||||
|---|---|---|
|
||||
| GET | `/settings` | whitelisted public keys, derived `registration`/`gameAccountSignup` flags, the per-shard **`brand`** block (name, `accent` color, logo/hero/favicon) a client themes itself from — one image runs as any shard, asset fields may be site-relative paths (resolve against the base URL) — and a **`push`** block `{ ntfyUrl }` (M7): the client-facing ntfy relay URL the app's embedded distributor registers its device topic against, from `NTFY_PUBLIC_URL` / first `NTFY_ALLOWED_ORIGINS` (never the internal `NTFY_BASE_URL`); `null` when push isn't configured for the shard. |
|
||||
| GET | `/settings` | whitelisted public keys, derived `registration`/`gameAccountSignup` flags, and the per-shard **`brand`** block (name, `accent` color, logo/hero/favicon) a client themes itself from — one image runs as any shard. Asset fields may be site-relative paths (resolve against the base URL). |
|
||||
| GET | `/status` | status message + current mode, **plus a `version` block** (`{ service:'runic-gateway', api, server }`) so a client first-run probe recognizes the backend and can run a version-mismatch guard |
|
||||
| GET | `/version` | lightweight, **DB-free** backend identity/version (`{ service, api, server }`) — the canonical target for the version guard and a cheap liveness check |
|
||||
| GET | `/posts/:category` | published only; `category` ∈ news\|five-on-friday\|newsletter\|screenshots |
|
||||
@@ -786,43 +203,10 @@ from the per-route **siteMode** middleware (§5), never from an auth gate.
|
||||
| GET | `/wiki` | list of pages (slug + title) |
|
||||
| GET | `/wiki/:slug` | single page |
|
||||
| POST | `/contact` | (rate-limited) send mail via SMTP; if unconfigured, respond `{fallback:"mailto", email}` |
|
||||
| GET | `/shard/ruleset` | the shard's own published ruleset (Protocol 3.0 `world.ruleset`): expansion, which optional systems are on, skill/stat caps, account and house limits, champion scroll rules, the save/restart schedule. Served from `shard_ruleset`, so it renders while the shard is down; live via `world.ruleset` on `/shard/stream`. Behind `requireFeature('ruleset')`. **`null`** means the shard has never published one — a real answer, distinct from a published ruleset. `caps.skill` / `caps.totalSkill` are in **tenths** (1000 = 100.0). |
|
||||
| GET | `/shard/points` | every points/loyalty leaderboard the shard publishes (Protocol 3.0 `points.board`) — Queen's Loyalty, Void Pool, the nine city loyalties, Clean Up Britannia, … Served from `shard_points_boards`, so it renders while the shard is down; live via `points.board` on `/shard/stream`. Behind `requireFeature('leaderboards')`, ordered by display name. **`maxPoints: 0` means uncapped** (the common case), and `nameString` is usually `null` with `nameNumber` holding a cliloc — resolve client-side or humanise the `system` key. |
|
||||
| GET | `/shard/points/:system` | one board by the shard's `PointsType` name (e.g. `QueensLoyalty`); `:system` must match `/^[A-Za-z][A-Za-z0-9_]{0,47}$/` or **400** before any query runs. **404** = the shard has never published that system, which is distinct from a published board nobody has scored in yet (**200** with an empty `top`). |
|
||||
| GET | `/shard/market?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=` | search the player-vendor marketplace (Protocol 3.0 `vendor.listing`). Returns **listings**, not vendors — "who sells X and for how much" is the question, and a vendor-shaped result would make every caller flatten the shops back out. Served from `shard_vendors` + `shard_vendor_items`, so it renders while the shard is down. Behind `requireFeature('market')` **and rate-limited** — the first genuinely expensive public read on the site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*` table, reachable with no session). `sort ∈ {price_asc, price_desc, recent}`. `q` matches the resolved display name **or** the item's literal name, with `%`/`_` escaped: they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let `?q=%` match every listing on the shard. Every response repeats `staleAt` (the oldest vendor row) because the shard sweeps round-robin — a banner that ages with the results it labels, not one fetched once. |
|
||||
| GET | `/shard/market/meta` | index size, staleness (`staleAt`/`freshAt`) and which facets and regions actually hold vendors, so a client builds its filters without running a search it will discard. |
|
||||
| GET | `/shard/market/vendors/:serial` | one shop and its listings; `:serial` must match `/^0x[0-9A-Fa-f]{1,16}$/` or **400** before any query runs. **404** = a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer, and distinguishing them would leak that a hidden vendor exists. `truncated` (with `total` exceeding `count`) means the shop holds more than the shard publishes per frame. |
|
||||
| GET | `/shard/features` | the shard features **this caller** may reach plus the audience rung they resolved to (§6.5), so a client hides nav it can't follow. Reports only what the caller can see — the list itself never discloses a gated feature. Consumed by the SPA header and (pending) the Android nav. |
|
||||
| GET | `/atlas/creatures?q=&facet=&limit=&offset=` | the bestiary, most numerous first, with an unpaginated `total`. Static content parsed from the shard's ServUO tree — **not** sidecar-backed, which is why the atlas sits outside `/shard`, and unlike `/shard/*` it **is** site-mode gated. Behind `requireFeature('atlas')`. `?facet=` is matched exactly and never validated against a list (no facet name exists in the code); the filter is an `EXISTS` over the points rather than a JSON path or `JSON_SEARCH` built from caller input, whose `%`/`_` wildcards would make `?facet=%` match everything. |
|
||||
| GET | `/atlas/creatures/:slug` | one creature: `places` (the point-in-rect aggregate — "lizardman → Shrines, Isamu-Jima, Yew"), `spawners` (the bounded raw list, with `spawnersTruncated`), `alsoHere`. **`points` is a COUNT and `spawners` is the LIST** — named apart so one key never means a number on one route and an array on another. `minDelay`/`maxDelay` are in **seconds**, normalised at parse time from the source's per-record minutes-or-seconds. 404 = no such creature in this atlas. |
|
||||
| GET | `/atlas/regions?facet=&q=` | named regions and the rectangles that placed each spawner |
|
||||
| GET | `/atlas/landmarks?facet=&q=` | points of interest, labelled by `group` ("Covetous", not "Level 1") |
|
||||
| GET | `/atlas/champions?facet=` | the **configured** altar roster. Not `/shard/champs`, which is the live board. |
|
||||
| GET | `/atlas/meta` | facets, counts and when the atlas was parsed. Game-world facts only — the ServUO path, source hashes and any pending refresh are operator detail and live on the admin route. |
|
||||
|
||||
Public content GETs pass through the **siteMode** gate (§5).
|
||||
|
||||
### /admin (admin/index.js → the capability routers in §2) — all behind `isLoggedIn` + `noindex` + `staffOnly`
|
||||
|
||||
`admin/index.js` applies the shared gate and mounts each capability router at the prefix it owns;
|
||||
`users`, `invites`, `auth/providers` and `bot-activity` add `adminOnly` on top, and `moderation` adds
|
||||
`modAccess` (admin + moderator, so editors are excluded). The content capabilities — `posts`,
|
||||
`uploads`, `wiki`, `pages` — add nothing: managing content is the editor tier's job, so `staffOnly` is
|
||||
the whole gate. The ops/config capabilities — `uo-link`, `email`, `discord-bot`, `settings`, and
|
||||
`PUT /site-mode` — are `adminOnly`; `shard` is the one mixed prefix, where self-service account
|
||||
linking carries no extra gate and the in-game staff operations carry `modAccess`. There is no residual
|
||||
file: every admin route is declared in a capability router.
|
||||
|
||||
`GET`/`PUT /admin/shard/visibility` are the third tier on that mixed prefix: **`adminOnly`**, because
|
||||
they decide what *anonymous* visitors can see (§6.5). They sit above `modAccess` deliberately — a
|
||||
moderator can ban a player but cannot decide what the public internet reads.
|
||||
|
||||
`GET /dashboard` and `PUT /site-mode` are the one place where a **single screen spans two tiers**: the
|
||||
dashboard is staff-wide, but the site-mode toggle on it is `adminOnly`. The client must therefore gate
|
||||
that control on its own (`Dashboard.jsx` renders it only for `role === 'admin'`) rather than relying on
|
||||
the route gate that admitted them to the page — the same rule the sidebar follows, so a non-admin is
|
||||
never shown a control that would 403. The URLs below are unaffected by which
|
||||
file a route sits in — that is the property the route manifest freezes.
|
||||
### /admin (admin.routes.js → admin.controller.js) — all behind `isLoggedIn` + `noindex`
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| GET | `/dashboard` | current mode, last change time + who, content counts, recent activity |
|
||||
@@ -839,16 +223,6 @@ file a route sits in — that is the property the route manifest freezes.
|
||||
| GET | `/settings` · PUT `/settings` | read all / update `{key:value,...}` |
|
||||
| GET | `/activity?limit=&offset=` | paginated activity log |
|
||||
| GET | `/users` · POST `/users` · PUT `/users/:id` · DELETE `/users/:id` | user mgmt (can't delete self / last admin; password hashed on write) |
|
||||
| GET | `/users/:id/trusted-devices` | list a user's active trusted devices (never tokens) |
|
||||
| DELETE | `/users/:id/trusted-devices` · `…/:deviceId` | revoke all / one of a user's trusted devices (logs `admin.trusted_device.revoke[_all]`) |
|
||||
| POST | `/users/:id/mfa/reset` | recover a locked-out user: disable TOTP + revoke all trusted devices + clear recovery codes (logs `admin.user.totp.reset`) |
|
||||
| GET | `/shard/atlas` | spawn-atlas status (`adminOnly`): the ServUO path, whether the tree is readable, whether it has drifted from what is loaded, counts, facets, and any refresh staged for review. The public `/atlas/meta` reports the game world only; the filesystem detail is here. |
|
||||
| POST | `/shard/atlas/import` | re-import without restarting; `{force}` ignores the hash gate. **An unreadable tree answers 200 with `status:"unavailable"`, not 500** — `refresh()` reports outcomes rather than throwing (the boot path must never be blocked by a bad tree) and that contract is preserved at the API. |
|
||||
| POST | `/shard/atlas/approve` · `/shard/atlas/reject` | answer a refresh staged because it would REMOVE a facet. Approving **re-parses** the tree, so what lands matches it at approval time; rejecting is remembered against those source hashes so it does not re-prompt every restart. 404 when nothing is staged. |
|
||||
| PUT | `/shard/atlas/path` | point the atlas at a different tree (persisted as `spawn_atlas_servuo_path`, which wins over `SERVUO_PATH`). Blank clears it. Deliberately **does not import** — moving the mount and reloading the world are separate decisions — and returns fresh status so the panel can offer the import next. |
|
||||
| GET | `/shard/clilocs` | cliloc-table status (`adminOnly`): every source found now (base first, then `custom/` overlays in merge order), what each contributed at the last import, readability, drift across the set, the entry count, and `missingSources`. `configured:false` is a supported state — item names then render as ids. No public counterpart: the table is never served *as* a table. |
|
||||
| POST | `/shard/clilocs/import` | reload after a client patch or an overlay edit; `{force}` ignores the hash gate, `{approve}` accepts a **vanished** source (refused by default — see the table notes above). **A missing path — or the likely mistake of pointing at the client's own COMPRESSED `Cliloc.enu` — answers 200 with `status:"unavailable"` and a `code`, not 500.** `COMPRESSED` is called out by name: a 500 would say only "something broke", and the operator needs to be told which file to convert. |
|
||||
| PUT | `/shard/clilocs/path` | point the site at a different cliloc base file or directory (persisted as `cliloc_client_path`, which wins over `UO_CLIENT_PATH`). Overlays are read from `custom/` beside it either way. Blank clears it. Deliberately **does not import**, same reasoning as the atlas path. |
|
||||
|
||||
Every admin write logs to `activity_log`.
|
||||
|
||||
@@ -879,128 +253,17 @@ who"; `activity_log` provides the history feed.
|
||||
## 6. Auth & security
|
||||
|
||||
- **JWT** signed with `JWT_SECRET`, `expiresIn=JWT_EXPIRES_IN` (default `1d`); payload `{id,username,role}`.
|
||||
- **Cookie**: `httpOnly`, `sameSite=Lax`, `path=/`, and **`secure` decided per-request** (`COOKIE_SECURE=auto` → `secure: req.secure`).
|
||||
- **Trusted-device MFA.** A second, separate httpOnly cookie (`rg_trust`, default 30d) — opaque, sha256-hashed server-side in `trusted_devices` — lets a browser/app **skip the TOTP step** (never the password) on future logins. It is a server-side, per-row-revocable record (never a JWT claim), so the stateless session JWT is unchanged and trust stays revocable. It only ever gates the **second factor**; it deliberately outlives logout, and is cleared on untrust / password change / password reset / TOTP disable. **Recovery codes** (bcrypt, single-use) are the 2FA-lockout fallback. All admin trusted-device/MFA actions and the self actions (`auth.login.trusted_device`, `account.trusted_device.*`, `account.recovery_code*`, `admin.trusted_device.*`, `admin.user.totp.reset`) are audit-logged. See `docs/website/TRUSTED_DEVICES_MFA.md`. This is the key to dual access: the cookie is `Secure` when reached through Pangolin (HTTPS, `X-Forwarded-Proto: https`) but **not** `Secure` when reached directly over the LAN IP on plain HTTP — so login works in both. `COOKIE_SECURE=true|false` can force it. Requires `trust proxy` (below). `localhost:5173` (Vite) and `localhost:3000` are same-site, so the cookie flows in dev too.
|
||||
- **Cookie**: `httpOnly`, `sameSite=Lax`, `path=/`, and **`secure` decided per-request** (`COOKIE_SECURE=auto` → `secure: req.secure`). This is the key to dual access: the cookie is `Secure` when reached through Pangolin (HTTPS, `X-Forwarded-Proto: https`) but **not** `Secure` when reached directly over the LAN IP on plain HTTP — so login works in both. `COOKIE_SECURE=true|false` can force it. Requires `trust proxy` (below). `localhost:5173` (Vite) and `localhost:3000` are same-site, so the cookie flows in dev too.
|
||||
- **bcrypt** hashing (cost 10+); plaintext passwords never stored, logged, or returned.
|
||||
- **Rate limiting** (`express-rate-limit`) on `/auth/login`, `/public/contact`, and — the only limited
|
||||
*read* — `/public/shard/market` and `/public/shard/market/vendors/:serial` (60/min/IP). Every other
|
||||
public read is an indexed lookup of bounded size; the marketplace search is a `LIKE` scan plus a
|
||||
`COUNT` over the largest `shard_*` table, anonymous by default, so it is the one public GET that is
|
||||
worth money to serve.
|
||||
- **Rate limiting** (`express-rate-limit`) on `/auth/login` and `/public/contact`.
|
||||
- **Validation** (`express-validator`) on all writes; centralized error handler.
|
||||
- **helmet** with a Content-Security-Policy tuned for the built React SPA. The policies now live in
|
||||
**`server/src/config/csp.js`** (`app.js` only wires them up):
|
||||
`default-src 'self'`; `script-src 'self'` (the Vite build emits only external module chunks — the
|
||||
inline module-preload polyfill is disabled in `client/vite.config.js` to keep this valid);
|
||||
`style-src 'self' 'unsafe-inline' https://fonts.googleapis.com` (React's pervasive inline
|
||||
`style={{…}}` attributes can't be nonce'd, plus the Google Fonts stylesheet); `font-src 'self'
|
||||
https://fonts.gstatic.com` (Cinzel); `img-src 'self' data: https:` (same-origin uploads, plus
|
||||
external https images embedded in wiki/news bodies or `BRAND_*` logo/hero/favicon); `connect-src
|
||||
'self'` (REST + SSE are same-origin); `frame-ancestors 'self'`; `object-src 'none'`; `base-uri
|
||||
'self'`; `form-action 'self'` (blocks an injected `<form action="https://evil">` from POSTing
|
||||
credentials off-origin — an exfil path `connect-src` does not cover; it was always emitted via
|
||||
helmet's `useDefaults` and is now pinned explicitly so it cannot vanish under a helmet upgrade).
|
||||
`upgrade-insecure-requests` is intentionally **not** set (TLS terminates at the proxy, there
|
||||
are no mixed-content subresources, and it would break a local `npm start` over plain http). The
|
||||
`/api/docs` Swagger UI route gets a **looser** policy that additionally allows inline script/style,
|
||||
since swagger-ui-express injects an inline bootstrap. helmet also strips `X-Powered-By`; the two
|
||||
internal-only listeners (`internalApp.js`, `bot/src/app.js`) disable it explicitly too.
|
||||
- **A second, tightened policy ships alongside on `Content-Security-Policy-Report-Only`** for one
|
||||
release before it replaces the enforced one (`docs/website/API_V2_PLAN.md` § Phase 1). It is derived
|
||||
from the enforced policy so the two cannot drift, and differs by exactly one directive:
|
||||
`frame-ancestors 'self'` → **`'none'`**. Serving both headers at once means the live policy keeps
|
||||
protecting users while anything the tightened version would break arrives as a report rather than as
|
||||
a broken page — and for `frame-ancestors` specifically, a report from the browser of whoever framed
|
||||
the site is the only way to learn that something does.
|
||||
- **`POST /api/csp-report`** is the same-origin violation sink that `report-to` / `report-uri` point at
|
||||
(`report-to` additionally requires the `Reporting-Endpoints` response header, which is set alongside).
|
||||
Same-origin on purpose: reports describe attacks against this site and are not handed to a
|
||||
third-party collector. It parses **both** wire formats (`application/csp-report` from Firefox/Safari,
|
||||
`application/reports+json` from Chrome's Reporting API — handling one drops half the browsers),
|
||||
writes to the `csp` log tag and **stores nothing**. Necessarily unauthenticated (browsers send
|
||||
reports with no session), so it is bounded on every axis: 16 KB body cap, per-IP rate limit, fixed
|
||||
field allowlist, every logged field truncated, and **always 204 — even for malformed input**, since a
|
||||
4xx would make the global error handler log the attacker-supplied body and turn an open endpoint into
|
||||
a log-flood primitive. Mounted outside `/api/v1` next to `/api/health`: the browser learns the path
|
||||
from the policy header, never from a client build, so it is not part of the versioned client
|
||||
contract.
|
||||
- **helmet** with a CSP suited to the SPA (self + inline styles as needed; image sources for uploads/hero).
|
||||
- **Admin not indexed**: `X-Robots-Tag: noindex, nofollow` on `/api/v1/admin` and the admin SPA routes; `robots.txt` disallows `/admin`.
|
||||
- **No directory browsing** (express.static doesn't list; no `serve-index`).
|
||||
- **No hardcoded credentials**: first admin via `seed.js` reading `ADMIN_USERNAME`/`ADMIN_PASSWORD` from env (created only if no users exist); `.env` git-ignored, `.env.example` committed.
|
||||
- **`app.set('trust proxy', 1)`** so secure cookies, `req.ip`, and rate-limiting work behind Pangolin.
|
||||
- **CORS**: same-origin in prod (SPA served by Express). Dev only: allow `CLIENT_ORIGIN` (Vite, `http://localhost:5173`) with `credentials:true`.
|
||||
|
||||
### 6.5 Shard visibility — the audience boundary (Protocol 3.0)
|
||||
|
||||
Every shard-derived surface is gated by an **admin-configurable, per-feature and per-field** audience
|
||||
setting. This **replaces** the static `PUBLIC_KINDS` allowlist that used to be the whole boundary.
|
||||
Policy lives in `utils/shardVisibility.js`; rows live in `shard_feature_visibility`; the admin surface
|
||||
is `GET`/`PUT /admin/shard/visibility` (`adminOnly`). Admin-facing guide:
|
||||
[`SHARD_VISIBILITY.md`](SHARD_VISIBILITY.md). Design: [`../link/v3.md`](../link/v3.md) §3.
|
||||
|
||||
**The ladder.** `anonymous < logged_in < player < staff < admin`, each rung implying the ones below.
|
||||
`viewerLevel(req)` resolves it: no session ⇒ `anonymous`; authenticated ⇒ `logged_in`; authenticated
|
||||
with a linked game account ⇒ `player`; moderator ⇒ `staff`; admin ⇒ `admin`. **Staff satisfy the
|
||||
`player` rung without a linked account** (consistent with `/player/*` being role-agnostic).
|
||||
**`editor` gets no shard privilege** — it is a content role, and mapping it to `staff` would silently
|
||||
widen what editors see.
|
||||
|
||||
**Two invariants that are code, not configuration.** Both are enforced server-side and both reject
|
||||
rather than silently ignore:
|
||||
|
||||
1. **`acct` and `webId` are admin-only, always.** They are not exposed as configurable fields, and a
|
||||
stored row attempting to loosen them is discarded on read as well as rejected on write. A character
|
||||
name is visible in game; the account behind it and the website user it links to are not.
|
||||
The lock is on the field's **meaning, not one spelling**: `isLockedField(key)` matches a key that
|
||||
*is* or *ends in* `acct`/`webId`, case-insensitively, so the flattened forms the read models emit
|
||||
(`shapeHouse` → `ownerAcct`, `shapeGuild` → `leaderWebId`) are covered too. An exact-key check was
|
||||
the original implementation and it let `GET /public/shard/idoc` serve `ownerAcct` anonymously.
|
||||
2. **A kind absent from `KIND_FEATURE` is never broadcast below `admin`.** Fail closed. This is what
|
||||
keeps the kind map a security boundary rather than a convenience filter, and it means a shard that
|
||||
starts emitting an unknown event degrades to staff-only, never to public.
|
||||
|
||||
**Fail-closed everywhere else too.** An unreadable visibility config withholds every public frame; a
|
||||
DB failure falls back to the compiled defaults (pre-3.0 behavior), not to open; an unresolvable viewer
|
||||
subscribes as `anonymous`. The ladder comparison uses **asymmetric** fallbacks by design — an unknown
|
||||
*viewer* level floors to the bottom rung and an unknown *requirement* ceils to admin, so an
|
||||
unrecognised value loses on both sides. (A single shared fallback cannot do that: whichever direction
|
||||
it picks, it fails open on one side.)
|
||||
|
||||
**Three enforcement points, one config:**
|
||||
|
||||
| Where | Mechanism |
|
||||
|---|---|
|
||||
| Routes | `requireFeature(name)` — **404** when the feature is disabled (don't leak that it exists), **403** when the caller is below its audience. `projectFeature` then strips out-of-rung fields from the body. |
|
||||
| SSE (`utils/shardBroadcast.js`) | Per-connection filtering. A subscriber's rung is resolved **once at subscribe time and frozen** for that connection, so a long-lived stream can't gain privilege; each frame is then mapped kind→feature, gated, and field-projected per viewer. Two subscribers can legitimately receive different versions of one event, or one of them nothing. |
|
||||
| Nav | `GET /public/shard/features` returns only what the caller may reach, so the SPA never renders a link that would 403. Presentation only. |
|
||||
|
||||
Config reads are cached ~5s, so admin changes take effect within seconds **including on already-open
|
||||
streams**. `PUBLIC_KINDS` still exists and is still exported (`notificationStreams.js`) but is now
|
||||
**derived** from the kind map rather than hand-maintained, so the two cannot drift.
|
||||
|
||||
**`PUBLIC_KINDS` is a module-load constant and must not be used to answer "may this caller read this
|
||||
kind?"** — it is computed from the compiled *defaults*, so it cannot see an admin's changes. Use
|
||||
`visibleKinds(level, config)`, which resolves against the live config. `/feed` uses it; it originally
|
||||
used `PUBLIC_KINDS` and consequently kept serving `guild.join` to anonymous callers after an admin had
|
||||
moved `guilds` to `staff`. `visibleKinds` deliberately ignores the `stream` flag: that governs SSE
|
||||
fan-out only, so a feature whose live firehose ships off (market) stays readable from stored history.
|
||||
|
||||
**Every read path that returns shard data must call `projectFeature`.** The stored-history endpoints
|
||||
are not exempt — `/feed` returns the same events the stream does, and returning them unprojected
|
||||
reopens on the REST side exactly what the stream closes. Relatedly, `shardEvents.db.list` treats an
|
||||
**empty** `kinds` array as "serve nothing", never "no filter"; the fall-through it used to take would
|
||||
have turned a fully-gated config into a dump of the entire event log.
|
||||
|
||||
`projectFeature` walks **arrays and plain objects only**. A `Date`, `Buffer` or other class instance
|
||||
is passed through as a value — rebuilding one key-by-key yields `{}`, which is the difference between
|
||||
the pure-JSON wire frames and the DB-backed read models whose rows carry real `Date` columns.
|
||||
|
||||
**Defaults reproduce pre-3.0 behavior exactly**, so installing the framework is a no-op until an admin
|
||||
changes something — with deliberate exceptions, which are the leaks it was written to close.
|
||||
`/public/shard/guilds`, `/public/shard/governors` and `/public/shard/feed` previously returned the raw
|
||||
stored payload, whose actors carry `acct` and `webId`; `/public/shard/idoc` returned the flattened
|
||||
`ownerAcct`. All are now stripped for every caller below admin.
|
||||
|
||||
---
|
||||
|
||||
## 7. Email
|
||||
@@ -1019,11 +282,7 @@ instead. Errors never leak credentials.
|
||||
|
||||
`utils/logger.js` — a small dependency-free logger with **two transports, console + file**,
|
||||
and four levels (`error`/`warn`/`info`/`debug`). Each line is timestamped and tagged by
|
||||
subsystem (`[server]`, `[http]`, `[db]`, `[auth]`, `[admin]`, `[ratelimit]`, `[csp]`, …).
|
||||
|
||||
> During the CSP report-only soak, `[csp]` is the tag to watch: a `csp violation` warn line with
|
||||
> `directive: frame-ancestors` means something really does frame the site and the enforce PR would
|
||||
> break it. Silence across one release is the green light to flip.
|
||||
subsystem (`[server]`, `[http]`, `[db]`, `[auth]`, `[admin]`, `[ratelimit]`, …).
|
||||
|
||||
- **Console**: color on a TTY, plain in Docker; verbosity = `LOG_LEVEL` (default `info`).
|
||||
- **File**: plain text appended to `LOG_DIR/LOG_FILE` (default `<server>/logs/app.log`,
|
||||
@@ -1045,15 +304,7 @@ subsystem (`[server]`, `[http]`, `[db]`, `[auth]`, `[admin]`, `[ratelimit]`, `[c
|
||||
- `app`: builds the Dockerfile (installs client+server, builds Vite, serves via Express),
|
||||
`env_file: .env`, `DB_HOST=db`, `depends_on: db (healthy)`, volume `uploads:/app/uploads`,
|
||||
`ports: "3000:3000"` — **binds 0.0.0.0** (no `127.0.0.1:` prefix) so Pangolin reaches it.
|
||||
- `ntfy` (M7): pinned upstream `binwiederhier/ntfy` image, declarative config only
|
||||
(`./ntfy/server.yml` mounted `:ro` + `NTFY_BASE_URL`), volume `ntfydata:/var/lib/ntfy`,
|
||||
**publishes `:80` on a host port** (`${NTFY_HOST_PORT:-2586}:80`, binds 0.0.0.0) so Pangolin — which
|
||||
runs outside the compose network — can forward the notification subdomain to it, the same reason
|
||||
`app` publishes `3000`. Both devices (SSE subscribe) and the backend publisher (POSTing tickles to
|
||||
registered device endpoints) reach ntfy on that public origin. Anonymous read-write to unguessable
|
||||
topics (no accounts to provision) — safe because pushes are content-free tickles. Bringing the stack
|
||||
up provisions a working push relay with **zero interactive setup**.
|
||||
- Volumes: `dbdata`, `uploads`, `ntfydata`.
|
||||
- Volumes: `dbdata`, `uploads`.
|
||||
|
||||
Express listens on `0.0.0.0:${PORT||3000}`. Pangolin terminates TLS and proxies to `app`.
|
||||
|
||||
@@ -1075,19 +326,6 @@ ADMIN_USERNAME=
|
||||
ADMIN_PASSWORD=
|
||||
# Email: configured in Admin → Settings → Email (Gmail OAuth2), not via env
|
||||
CLIENT_ORIGIN=http://localhost:5173
|
||||
# Push (M7): the ntfy relay URL — also the backend's SSRF allow-set for device
|
||||
# endpoints. NTFY_ALLOWED_ORIGINS / NTFY_PUBLISH_TOKEN are optional.
|
||||
NTFY_BASE_URL=https://ntfy.example.com
|
||||
# The client-facing ntfy URL surfaced to the app via /public/settings.push.ntfyUrl
|
||||
# (the app registers its topic endpoint here). Defaults to the first
|
||||
# NTFY_ALLOWED_ORIGINS entry; set explicitly when the public URL differs from the
|
||||
# internal NTFY_BASE_URL. Without it (and without NTFY_ALLOWED_ORIGINS) the app
|
||||
# shows push as unavailable for the shard.
|
||||
NTFY_PUBLIC_URL=https://ntfy.example.com
|
||||
NTFY_ALLOWED_ORIGINS=https://ntfy.example.com
|
||||
# Host port the ntfy container publishes :80 on (default 2586); the reverse proxy
|
||||
# forwards the notification subdomain to host:NTFY_HOST_PORT. Change on a conflict.
|
||||
NTFY_HOST_PORT=2586
|
||||
```
|
||||
|
||||
`.gitignore`: `node_modules/`, `.env`, `_reference/`, `client/dist/`, `uploads/`.
|
||||
|
||||
@@ -1,296 +0,0 @@
|
||||
# Cliloc table (item and title names)
|
||||
|
||||
**Status:** Complete on `edge` — website [#115](https://gitea.whitlocktech.com/RunicGateway/website/pulls/115), docs [#70](https://gitea.whitlocktech.com/RunicGateway/docs/pulls/70).
|
||||
**Design:** [`docs/link/v3.md` §8.6](../link/v3.md) — Protocol 3.0, the dependency Part B/3 was sequenced behind.
|
||||
|
||||
A "cliloc" is UO's localization table: an integer id mapped to a display string.
|
||||
**Items on the wire carry a `LabelNumber`, not a name.** The bridge has always
|
||||
sent that number — `char.profile.equipment` has a `cliloc` field, reward titles
|
||||
arrive as a cliloc number in string form, and every marketplace listing carries
|
||||
one — but the site had no table to look it up in, so a character sheet could only
|
||||
render `id 1023721` where the game renders **"quarter staff"**.
|
||||
|
||||
The number was never the missing piece. The table was.
|
||||
|
||||
## Why the operator has to convert the file
|
||||
|
||||
This is the awkward part, and it is not avoidable:
|
||||
|
||||
**Every current UO client ships its cliloc files compressed.** All eight
|
||||
`Cliloc.*` files in a modern client (`chs`, `cht`, `deu`, `enu`, `esp`, `fra`,
|
||||
`jpn`, `kor`) begin with a DWORD whose high byte is `0x8E` — the "Mythic"
|
||||
compressed container. The plain layout this site parses is what those files
|
||||
looked like *before* that change.
|
||||
|
||||
Decompressing it means an inverse-BWT coder with a frequency header — a few
|
||||
hundred lines of bit-level work whose failure mode is plausible-looking garbage
|
||||
rather than an error. The site has no business carrying that at runtime.
|
||||
|
||||
Two facts make the alternatives worse, not better:
|
||||
|
||||
- **ServUO cannot read it either.** Its bundled `Ultima.StringList` implements
|
||||
only the plain layout, so on a modern client `VendorSearch.StringList` is null
|
||||
and `VendorSearch.GetItemName` returns `item.Name` — usually nothing. The
|
||||
shard cannot supply names on our behalf; the in-game Vendor Search gump has the
|
||||
same gap.
|
||||
- **Nothing client-derived may be committed.** UO's strings are EA's. The repo
|
||||
ships no string table for the same reason it ships no artwork and no map
|
||||
snapshot — see [`SPAWN_ATLAS.md`](SPAWN_ATLAS.md).
|
||||
|
||||
So the conversion happens **once, on the operator's machine, against their own
|
||||
client**, and the site reads the result from a path it is given. A shard that
|
||||
never does this is in a fully supported state: names render as ids, exactly as
|
||||
they did before the table existed.
|
||||
|
||||
## Converting
|
||||
|
||||
Either format below is accepted; the site sniffs which one it was handed.
|
||||
|
||||
| Format | Fidelity | Notes |
|
||||
|---|---|---|
|
||||
| **Plain binary** (recommended) | Exact | 6-byte header, then `{int32 number, byte flag, uint16 length, UTF-8}` records |
|
||||
| Delimited text | Loses leading/trailing whitespace | `number<TAB\|,\|;>text` per line; a header row, blank lines and `#` comments are ignored |
|
||||
|
||||
The whitespace caveat is real but cosmetic: ~1,300 of the 123,490 entries in a
|
||||
stock `Cliloc.enu` are label prefixes like `"max = "` whose trailing space is
|
||||
meaningful when the client concatenates a value onto them. Nothing on this site
|
||||
concatenates, and every consumer passes through `displayText()`, which trims.
|
||||
|
||||
### Using the bundled tool
|
||||
|
||||
`server/tools/cliloc-export/` is a small .NET console app that drives
|
||||
[UOFiddler](https://github.com/polserver/UOFiddler)'s `Ultima.dll` — the
|
||||
decompressor that already exists and is already maintained — and writes the plain
|
||||
format. It loads that DLL **reflectively** so it compiles against any SDK, and it
|
||||
writes the records by hand because UOFiddler's own `SaveStringList` *re-compresses*
|
||||
on save (its purpose is round-tripping a file back into the client, so its output
|
||||
is byte-identical to its input — a trap worth knowing about).
|
||||
|
||||
```bash
|
||||
cd website/server/tools/cliloc-export
|
||||
dotnet build -c Release
|
||||
|
||||
# binary (recommended)
|
||||
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.plain
|
||||
|
||||
# or tab-delimited
|
||||
dotnet run -- "<UOFiddler>/Ultima.dll" "<UO client>/Cliloc.enu" /srv/uo-data/clilocs.tsv --tsv
|
||||
```
|
||||
|
||||
A UOFiddler GUI export works equally well — anything producing one of the two
|
||||
shapes above is fine.
|
||||
|
||||
## Shard-added and shard-edited items
|
||||
|
||||
**Shards edit items and add new ones**, and those carry cliloc ids no stock
|
||||
client table has. The table is therefore built from a **set** of sources, all
|
||||
re-read on every boot and hash-gated together — the same shape as the spawn
|
||||
atlas, which reads `Regions.xml` + `Locations/*.xml` + `Spawns/*.xml` +
|
||||
`ChampionSpawns.xml` and merges them:
|
||||
|
||||
```
|
||||
<cliloc path>/
|
||||
clilocs.plain ← base: the converted client table
|
||||
custom/
|
||||
01-uomysticmoon.tsv ← overlays: shard additions and overrides
|
||||
02-events.tsv
|
||||
```
|
||||
|
||||
Overlays use the same delimited-text format, are read in **sorted order**, and
|
||||
**later sources win** — so an overlay both *adds* ids the client never had and
|
||||
*overrides* stock ones the shard has re-purposed. Any `.tsv`, `.csv`, `.txt`,
|
||||
`.enu` or `.plain` file in `custom/` is picked up; anything else (a `README.md`,
|
||||
say) is ignored.
|
||||
|
||||
Adding, editing or removing any overlay counts as drift, so a new custom item
|
||||
needs only a file edit and a restart — or the admin panel's Import button.
|
||||
**Adding one item never means re-exporting a 5 MB client file.**
|
||||
|
||||
The import result reports what each source contributed, which is how you confirm
|
||||
an overlay took effect — `overrode: 0` on a file meant to re-label stock items
|
||||
says it did not:
|
||||
|
||||
```json
|
||||
"sources": [
|
||||
{ "label": "clilocs.plain", "kind": "base", "entries": 123490, "added": 123490, "overrode": 0 },
|
||||
{ "label": "custom/uomysticmoon.tsv", "kind": "custom", "entries": 2, "added": 1, "overrode": 1 }
|
||||
]
|
||||
```
|
||||
|
||||
**Why a convention rather than discovery.** Everywhere else this pipeline follows
|
||||
the shard's own files, but **ServUO has no server-side notion of a custom
|
||||
cliloc** — they live in the patched client a shard distributes to its players,
|
||||
and nothing in the tree declares them. There is nothing to discover, so `custom/`
|
||||
is the one thing here that is our convention rather than the shard's. (An
|
||||
operator who *does* patch their client cliloc needs no overlay at all: convert
|
||||
the patched file and their edits are simply in the base.)
|
||||
|
||||
Measured on the live shard for scale: its script tree references **16,434** cliloc
|
||||
ids and only **37** are absent from the stock client table — tens of entries
|
||||
against a 67k base, which is what makes an overlay the right shape rather than a
|
||||
second full table.
|
||||
|
||||
## Configuring the path
|
||||
|
||||
Two ways to point at the sources, the setting winning over the environment:
|
||||
|
||||
| Source | Notes |
|
||||
|---|---|
|
||||
| `cliloc_client_path` setting | Admin-editable (Admin → Shard); takes effect on the next refresh without a redeploy |
|
||||
| `UO_CLIENT_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |
|
||||
|
||||
The value may be **the base file itself or a directory to search**, because both
|
||||
are natural answers to "where is it". Overlays are read from a `custom/`
|
||||
directory beside the base **either way** — pointing at a file does not forfeit
|
||||
them.
|
||||
|
||||
A directory is searched case-insensitively (the client writes `Cliloc.enu` on
|
||||
Windows; the site usually runs on Linux) for, in order: `clilocs.tsv`,
|
||||
`clilocs.csv`, `clilocs.plain`, `cliloc.plain`, `cliloc.plain.enu`,
|
||||
`cliloc.enu.plain`, `clilocs.txt`, `cliloc.enu`.
|
||||
|
||||
That ordering puts explicitly-converted names first on purpose. Pointing the
|
||||
setting straight at an unconverted client directory finds `cliloc.enu`, which is
|
||||
compressed — and the site says so by name rather than failing obscurely:
|
||||
|
||||
```
|
||||
status: unavailable
|
||||
code: COMPRESSED
|
||||
reason: This is a compressed (Mythic-format) cliloc file, which the site cannot
|
||||
read. Convert it to the plain format first — see docs/website/CLILOCS.md.
|
||||
```
|
||||
|
||||
## Refresh contract
|
||||
|
||||
Identical in shape to the spawn atlas, and for the same reasons:
|
||||
|
||||
- **It never blocks startup.** No path, an unreadable file, a wrong-format file,
|
||||
a database error — all caught and logged. The site comes up either way.
|
||||
- **Hash-gated.** The boot path hashes the file and skips the parse entirely when
|
||||
it matches what is loaded, which is every restart that did not follow a client
|
||||
patch. Measured on a stock table: **14 ms** for the no-op, **663 ms** for a full
|
||||
parse and replace.
|
||||
- **A `PARSER_VERSION` bump also counts as drift**, so a corrected parse reaches
|
||||
an install whose client never patches.
|
||||
|
||||
### Two ways a refresh is refused
|
||||
|
||||
**A corrupt file** — the realistic failure for any single source — makes the
|
||||
parser fail on a truncated record rather than yield a plausible-but-short table,
|
||||
so it is caught outright. Verified: a file truncated to half its length reports
|
||||
|
||||
```
|
||||
code: TRUNCATED
|
||||
reason: Truncated record header at byte 2486759 (74909 entries read)
|
||||
```
|
||||
|
||||
and the rows already loaded are untouched. A malformed overlay names the file it
|
||||
came from (`custom/broken.tsv: No cliloc entries found…`), because "which of my
|
||||
six overlay files is broken" is otherwise a guessing game.
|
||||
|
||||
**A source that has VANISHED** is the hazard a single file did not have. It
|
||||
parses perfectly and imports a table quietly missing everything that file
|
||||
contributed — and an unmounted volume looks exactly like a deliberate deletion
|
||||
from here. This is the same ambiguity the atlas stages a facet removal for, so it
|
||||
is escalated rather than applied:
|
||||
|
||||
```
|
||||
status: needsReview
|
||||
reason: 1 previously-loaded cliloc source(s) are missing;
|
||||
the existing table is unchanged
|
||||
missingSources: ["custom/uomysticmoon.tsv"]
|
||||
```
|
||||
|
||||
`status()` reports `missingSources` too, so the panel can show it before anyone
|
||||
clicks Import. An admin accepts it by re-running the import with
|
||||
`{ "approve": true }`.
|
||||
|
||||
**Why that is a flag and not the atlas's approve/reject pair.** The atlas stores
|
||||
a pending decision in its own table so that approving *re-parses the tree*, which
|
||||
is what keeps a multi-megabyte blob out of the database and makes the applied
|
||||
result match the tree at approval time. Here nothing is stored, so re-reading at
|
||||
approval time is automatic — the decision is a single boolean on the import an
|
||||
admin was already going to run.
|
||||
|
||||
## What gets stored
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Parsed from a stock `Cliloc.enu` | **123,490** entries |
|
||||
| Of those, empty strings | **55,994** (ids the client reserves and never uses) |
|
||||
| Stored in `shard_clilocs` | **67,496** |
|
||||
|
||||
Blank entries are dropped at import. A row resolving to no name is
|
||||
indistinguishable from no row at all to every caller, and dropping them makes the
|
||||
binary and text imports converge on **identical** content — the binary format
|
||||
carries the blanks explicitly and a text export may or may not, depending on the
|
||||
tool. Verified: both formats import to the same 67,496 rows with the same keys.
|
||||
|
||||
`text` is `TEXT`, not `VARCHAR`: the long property descriptions reach 12 KB, and
|
||||
silently truncating them would be worse than storing them. The index that matters
|
||||
for marketplace search is on the denormalized `shard_vendor_items.display_name`,
|
||||
not here.
|
||||
|
||||
## How names are applied
|
||||
|
||||
**Resolution happens server-side.** The table is never served *as* a table and
|
||||
there is no public route for it. Two reasons: 67k rows would dwarf any page that
|
||||
used them, and the Android client consumes the same JSON and would otherwise need
|
||||
its own copy.
|
||||
|
||||
`resolveMany()` takes a batch of ids and returns a `Map` holding only those that
|
||||
resolved to something displayable, so "no such id" and "id with no usable name"
|
||||
collapse into one branch at the call site. It never throws — a cliloc lookup is
|
||||
decoration on someone's character sheet, and a database blip must not fail the
|
||||
sheet. A capped in-process cache fronts it; measured cold **4.2 ms**, warm
|
||||
**0.015 ms**.
|
||||
|
||||
### `displayText()`
|
||||
|
||||
Cliloc strings interpolate arguments the client pulls from an item's property
|
||||
list — `~1_val~`, `~2_NAME~`. **We never have those**: the bridge sends the id,
|
||||
not the packet. So a name carrying them is reduced to what is actually knowable.
|
||||
|
||||
| Raw | Displayed |
|
||||
|---|---|
|
||||
| `quarter staff` | `quarter staff` |
|
||||
| `cold damage ~1_val~%` | `cold damage` |
|
||||
| `[~1_stuff~]` | *(nothing — the whole string was the argument)* |
|
||||
| `50%` | `50%` |
|
||||
| `Runic Gateway Sigil (v2)` | `Runic Gateway Sigil (v2)` |
|
||||
|
||||
**Punctuation is only tidied when a placeholder was actually removed.** The
|
||||
trailing `%` in row two is the unit belonging to the number we never had, and the
|
||||
brackets in row three only ever wrapped the argument — but a string with no
|
||||
placeholder has no such debris, and trimming it anyway corrupts real names. Rows
|
||||
four and five are the ones that caught it: a shard's custom
|
||||
`"Runic Gateway Sigil (v2)"` rendered as `"(v2"` while the bracket trim was
|
||||
unconditional.
|
||||
|
||||
### Consumers
|
||||
|
||||
- **Character sheet equipment.** `enrichCharProfile` attaches `clilocName` to each
|
||||
item. A player-given `name` always wins — "Bob's lucky axe" must not be
|
||||
relabelled "hatchet" — and the client re-states that precedence.
|
||||
- **Reward titles.** `titles.rewardResolved` is a parallel array with the numeric
|
||||
entries turned into words (`null` where nothing resolved). The sheet used to
|
||||
*skip* numeric reward titles entirely, having no way to render them.
|
||||
- **Marketplace listings** (Protocol 3.0 §8) denormalize the resolved name into
|
||||
`shard_vendor_items.display_name` so search can index it.
|
||||
|
||||
## Admin surface
|
||||
|
||||
All admin-only, alongside the atlas under Admin → Shard:
|
||||
|
||||
| Route | Purpose |
|
||||
|---|---|
|
||||
| `GET /api/v1/admin/shard/clilocs` | Sources found, what each contributed at the last import, readability, drift, entry count, `missingSources` |
|
||||
| `POST /api/v1/admin/shard/clilocs/import` | Reload after a client patch or an overlay edit; `{ "force": true }` reimports an unchanged set, `{ "approve": true }` accepts a vanished source |
|
||||
| `PUT /api/v1/admin/shard/clilocs/path` | Set the path; blank disables resolution |
|
||||
|
||||
A refresh **result is not an exception**: a missing file, or the likely mistake of
|
||||
pointing at the client's own compressed `Cliloc.enu`, answers `200` with
|
||||
`status: "unavailable"` and a reason. A `500` would say only "something broke";
|
||||
the operator needs to be told which file to convert. Setting the path
|
||||
deliberately does **not** import as a side effect — the response carries the
|
||||
refreshed status so the panel can offer that as the next step.
|
||||
@@ -1,167 +0,0 @@
|
||||
# Marketplace — the player-vendor index
|
||||
|
||||
**Status:** On `edge` — servuo-plugins [#5](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/5), link [#19](https://gitea.whitlocktech.com/RunicGateway/link/pulls/19), website [#116](https://gitea.whitlocktech.com/RunicGateway/website/pulls/116).
|
||||
**Design:** [`docs/link/v3.md` §8](../link/v3.md) — Protocol 3.0 Part B/3.
|
||||
**Depends on:** [`CLILOCS.md`](CLILOCS.md) — without a cliloc table, listings render as item ids.
|
||||
|
||||
The marketplace is a searchable index of every player vendor on the shard: what
|
||||
each shop is selling, for how much, and where it is standing. It is the same set
|
||||
the in-game **Vendor Search** gump reads, offered from outside the game — so a
|
||||
player can find the vanquishing kryss they want before logging in, and someone
|
||||
who does not play at all can see that the economy exists.
|
||||
|
||||
Page: `/site/market`, plus `/site/market/vendors/:serial` for one shop.
|
||||
|
||||
## Three things the pages must say out loud
|
||||
|
||||
Everything below follows from how the data is gathered, and each has a visible
|
||||
consequence the UI is required to surface.
|
||||
|
||||
**1. The prices are not live.** The shard sweeps vendors **round-robin** — at
|
||||
most `Bridge.MarketSweepBatch` shops per tick — so a given shop can be a full
|
||||
cycle behind. The page carries a *"prices last refreshed N minutes ago"* banner
|
||||
driven by the **oldest** vendor row, not the newest: the one stale shop is the
|
||||
one that wastes somebody's trip.
|
||||
|
||||
**2. A shop can be truncated.** `Bridge.MarketMaxListings` (250 by default) caps
|
||||
how many listings one frame carries. A commodity reseller with thousands of
|
||||
stacked resources is a real thing, and an uncapped frame for one is measured in
|
||||
megabytes. Over the cap the shop reports `truncated`, and the vendor page says
|
||||
*"showing 250 of 3,104 — this shop holds more than the shard publishes"* rather
|
||||
than presenting a partial shop as complete.
|
||||
|
||||
**3. An item may have no name.** Items on the wire carry a cliloc id, not a name.
|
||||
On a shard whose operator has not converted a cliloc table
|
||||
([`CLILOCS.md`](CLILOCS.md)) the honest render is the item id — never an invented
|
||||
label, which would be indistinguishable from a real one.
|
||||
|
||||
## Privacy: the player's own toggle wins
|
||||
|
||||
Only vendors whose owner left the in-game **Vendor Search** flag ON are ever sent
|
||||
to the site. A player who hides their shop in game is hidden here too, and no
|
||||
admin setting overrides that. When they hide one that was already indexed, the
|
||||
shard emits `vendor.listing.remove` and the row is deleted — so revoking consent
|
||||
takes effect, it does not merely stop refreshing.
|
||||
|
||||
Shop name, owner character name and location default to **Everyone**, because the
|
||||
stock Vendor Search gump already shows exactly that set to any player in game.
|
||||
They remain admin-configurable; see [`SHARD_VISIBILITY.md`](SHARD_VISIBILITY.md).
|
||||
Account names and website user ids never cross the wire at all.
|
||||
|
||||
## How it is put together
|
||||
|
||||
```
|
||||
ServUO uo-link sidecar website
|
||||
────── ─────────────── ───────
|
||||
BridgeMarket.cs vendors table shard_vendors
|
||||
round-robin sweep ──────► (whole frame blob) ──────► shard_vendor_items
|
||||
per-vendor diff GET /market (paged) + display_name
|
||||
vendor.listing resolved at ingest
|
||||
vendor.listing.remove
|
||||
```
|
||||
|
||||
**The shard side** walks at most `MarketSweepBatch` vendors per tick from a
|
||||
persistent cursor, diffs each against what it last published, and emits a whole
|
||||
frame for any shop that moved. Per-tick cost is therefore bounded by the batch,
|
||||
not by how many vendors the world holds — full coverage takes
|
||||
`ceil(vendors / batch) × MarketSweepSeconds`.
|
||||
|
||||
**The sidecar** stores each frame whole and serves `GET /market`, its only paged
|
||||
read. It normalizes nothing and defines no audiences: it is a dumb forwarder, and
|
||||
search is the website's job.
|
||||
|
||||
**The website** splits each frame into a vendor row and its listings, replacing
|
||||
that vendor's whole listing set inside one transaction (the frame is
|
||||
authoritative for that vendor, never a delta). Item names are resolved against
|
||||
the cliloc table **on the way in** and stored denormalized, which is what makes
|
||||
search-by-name possible and keeps the cliloc table off the hot path.
|
||||
|
||||
## Operating it
|
||||
|
||||
Everything is in `Config/Bridge.cfg` on the shard. There is nothing to configure
|
||||
on the website.
|
||||
|
||||
| Setting | Default | What it does |
|
||||
|---|---|---|
|
||||
| `MarketEnabled` | `true` | Master switch. Off publishes nothing; the page shows an empty index. |
|
||||
| `MarketSweepSeconds` | `60` | Tick interval. |
|
||||
| `MarketSweepBatch` | `25` | Vendors inventoried per tick. Clamped 1..500. |
|
||||
| `MarketMaxListings` | `250` | Per-shop listing cap, after which `truncated`. Clamped 1..5000. |
|
||||
|
||||
**Faster coverage vs. per-tick cost.** Lowering `MarketSweepSeconds` or raising
|
||||
`MarketSweepBatch` both refresh the index sooner and both cost more per tick.
|
||||
The expensive part is the item walk, which recurses into every container a vendor
|
||||
is selling — so a shard of big shops should raise the interval rather than the
|
||||
batch.
|
||||
|
||||
`[bridge status` reports the sweep, including `lastMs` and `maxMs`:
|
||||
|
||||
```
|
||||
market(enabled=True sweeps=42 scanned=108 emitted=27 removed=0 skipped=0
|
||||
truncated=0 tracked=27 vendors=27 cursor=2 batch=25 lastMs=0.31 maxMs=15.40)
|
||||
```
|
||||
|
||||
A tick over **50 ms** prints a rate-limited warning naming the knob:
|
||||
|
||||
```
|
||||
[Bridge] market sweep took 82.4 ms (budget 50 ms) - lower Bridge.MarketSweepBatch (now 25) if this persists
|
||||
```
|
||||
|
||||
Measured on a shard with 27 vendors × 40 listings (209k items, 43k mobiles):
|
||||
**15.4 ms** for the first cold tick of 25 vendors, **0.3 ms** in steady state —
|
||||
the diff is what makes an unchanged world nearly free. Note the arithmetic: 25
|
||||
*full* shops at the 250-listing cap is 6,250 items ≈ 95 ms, over budget. Real
|
||||
shops hold tens, which is why 25 is the default and why the warning exists.
|
||||
|
||||
`[bridge sweepnow` runs one tick immediately; `[bridge reload` re-reads the
|
||||
settings above without a restart.
|
||||
|
||||
## Names arriving late
|
||||
|
||||
Item names come from the cliloc table, and the market sweep will **not** re-send
|
||||
an unchanged shop just because the site learned what its items are called. So a
|
||||
cliloc import triggers a bulk re-resolution of every stored listing — otherwise
|
||||
an operator who configures clilocs after the first sweep would see item ids until
|
||||
every shop happened to change on its own. It runs after a boot import and after
|
||||
an admin import, takes ~50 ms per thousand listings, and never throws: a failure
|
||||
leaves names exactly as they were.
|
||||
|
||||
## API
|
||||
|
||||
All under `/api/v1/public/shard`, gated by the `market` feature and
|
||||
**rate-limited** — these are the first genuinely expensive public reads on the
|
||||
site (a `LIKE` scan plus a `COUNT` over what is typically the largest `shard_*`
|
||||
table, reachable with no session).
|
||||
|
||||
| Route | What |
|
||||
|---|---|
|
||||
| `GET /market` | Search. Returns **listings**, not vendors — "who sells X and for how much" is the question. `?q=&minPrice=&maxPrice=&itemId=&map=®ion=&sort=&limit=&offset=`, `sort ∈ {price_asc, price_desc, recent}`. |
|
||||
| `GET /market/meta` | Index size, staleness, and which facets and regions actually hold vendors — so a client builds its filters without running a search it will discard. |
|
||||
| `GET /market/vendors/:serial` | One shop and its listings. **404** for a serial the index has never seen, which also covers a vendor since dismissed or hidden — to an anonymous caller those are the same answer. |
|
||||
|
||||
`q` matches the resolved display name **or** the item's own literal name, because
|
||||
an item with a player-set name (most of what is worth searching for on a
|
||||
player-run shard) may carry a generic cliloc. `%` and `_` in a query are escaped:
|
||||
they are `LIKE` metacharacters, not SQL ones, so parameterization alone would let
|
||||
a search for `%` match every listing on the shard.
|
||||
|
||||
Full schemas are in the OpenAPI spec (`ShardMarketPage`, `ShardMarketVendor`,
|
||||
`ShardMarketMeta`, `ShardMarketListing`, `ShardMarketLocation`).
|
||||
|
||||
## Tables
|
||||
|
||||
`shard_vendors` (one row per shop) and `shard_vendor_items` (one row per priced
|
||||
listing). Both are ingest-owned; nothing else writes to them. No foreign keys,
|
||||
in keeping with every other `shard_*` table — the ingest transaction is what
|
||||
keeps them consistent, and an FK would turn a malformed frame into a failed write
|
||||
rather than a dropped row.
|
||||
|
||||
There is deliberately **no `payload` column** on `shard_vendors`, unlike the
|
||||
points board next door. The board's top-N is a fixed-size list read whole, so it
|
||||
lives in JSON; here the items *are* the searchable rows, so they are normalized
|
||||
and there is nothing left worth duplicating. The sidecar keeps the whole blob,
|
||||
because outage resilience is its job.
|
||||
|
||||
`shard_vendor_items.display_name` is denormalized and indexed (alone, and
|
||||
composite with `price` for "cheapest matching X"). See "Names arriving late"
|
||||
above for how it is kept current.
|
||||
@@ -1,555 +0,0 @@
|
||||
# Website — Project Tree
|
||||
|
||||
> **Auto-generated.** This file is maintained by the `sync-project-tree` CI workflow in
|
||||
> the [`RunicGateway/website`](https://gitea.whitlocktech.com/RunicGateway/website) repository, which
|
||||
> opens a pull request here whenever the tracked file layout on `main` changes. Do not edit
|
||||
> by hand — changes will be overwritten by the next sync.
|
||||
|
||||
A snapshot of the tracked files in the repository (build output, dependencies, and other
|
||||
git-ignored paths are excluded).
|
||||
|
||||
```text
|
||||
website/
|
||||
├── .claude/
|
||||
│ └── launch.json
|
||||
├── .gitea/
|
||||
│ ├── ISSUE_TEMPLATE/
|
||||
│ │ ├── bug_report.md
|
||||
│ │ ├── config.yaml
|
||||
│ │ └── feature_request.md
|
||||
│ ├── scripts/
|
||||
│ │ └── gen_tree.py
|
||||
│ ├── workflows/
|
||||
│ │ ├── build-images.yml
|
||||
│ │ ├── pr-checks.yml
|
||||
│ │ ├── sonarqube.yml
|
||||
│ │ └── sync-project-tree.yml
|
||||
│ └── PULL_REQUEST_TEMPLATE.md
|
||||
├── bot/
|
||||
│ ├── src/
|
||||
│ │ ├── discord/
|
||||
│ │ │ ├── commands/
|
||||
│ │ │ │ ├── announce.command.js
|
||||
│ │ │ │ ├── autorole.command.js
|
||||
│ │ │ │ ├── ban.command.js
|
||||
│ │ │ │ ├── filter.command.js
|
||||
│ │ │ │ ├── filterallow.command.js
|
||||
│ │ │ │ ├── index.js
|
||||
│ │ │ │ ├── invite.command.js
|
||||
│ │ │ │ ├── kick.command.js
|
||||
│ │ │ │ ├── modlog.command.js
|
||||
│ │ │ │ ├── mute.command.js
|
||||
│ │ │ │ ├── news.command.js
|
||||
│ │ │ │ ├── ping.command.js
|
||||
│ │ │ │ ├── role.command.js
|
||||
│ │ │ │ ├── rolemenu.command.js
|
||||
│ │ │ │ ├── roles.command.js
|
||||
│ │ │ │ ├── schedule.command.js
|
||||
│ │ │ │ ├── warn.command.js
|
||||
│ │ │ │ ├── warnings.command.js
|
||||
│ │ │ │ └── wiki.command.js
|
||||
│ │ │ ├── discordManager.js
|
||||
│ │ │ ├── guildMemberAdd.js
|
||||
│ │ │ ├── guildMemberRemove.js
|
||||
│ │ │ ├── inviteTracker.js
|
||||
│ │ │ ├── messageFilter.js
|
||||
│ │ │ ├── modLog.js
|
||||
│ │ │ ├── newsAnnounce.js
|
||||
│ │ │ └── roleMenuHandler.js
|
||||
│ │ ├── filter/
|
||||
│ │ │ ├── filterCache.js
|
||||
│ │ │ ├── inviteFilter.js
|
||||
│ │ │ ├── normalize.js
|
||||
│ │ │ └── spamFilter.js
|
||||
│ │ ├── internal/
|
||||
│ │ │ ├── internal.controller.js
|
||||
│ │ │ ├── internal.routes.js
|
||||
│ │ │ └── requireInternalKey.js
|
||||
│ │ ├── invites/
|
||||
│ │ │ ├── inviteRotator.js
|
||||
│ │ │ └── inviteScheduler.js
|
||||
│ │ ├── model/
|
||||
│ │ │ ├── filterAllowlist.js
|
||||
│ │ │ ├── filterHits.js
|
||||
│ │ │ ├── filterWords.js
|
||||
│ │ │ ├── guildConfig.js
|
||||
│ │ │ ├── inviteLog.js
|
||||
│ │ │ ├── memberEvents.js
|
||||
│ │ │ ├── roleMenus.js
|
||||
│ │ │ ├── scheduledMessages.js
|
||||
│ │ │ ├── spamHits.js
|
||||
│ │ │ ├── tempRoles.js
|
||||
│ │ │ └── warnings.js
|
||||
│ │ ├── roles/
|
||||
│ │ │ └── tempRoleSweeper.js
|
||||
│ │ ├── scheduler/
|
||||
│ │ │ └── scheduler.js
|
||||
│ │ ├── site/
|
||||
│ │ │ └── siteApiClient.js
|
||||
│ │ ├── utils/
|
||||
│ │ │ ├── duration.js
|
||||
│ │ │ └── logger.js
|
||||
│ │ ├── app.js
|
||||
│ │ ├── bootstrap.js
|
||||
│ │ ├── brand.js
|
||||
│ │ ├── db.js
|
||||
│ │ └── server.js
|
||||
│ ├── .env.example
|
||||
│ ├── .gitignore
|
||||
│ ├── Dockerfile
|
||||
│ ├── package-lock.json
|
||||
│ └── package.json
|
||||
├── brand/
|
||||
│ └── README.md
|
||||
├── client/
|
||||
│ ├── public/
|
||||
│ │ ├── assets/
|
||||
│ │ │ └── img/
|
||||
│ │ │ ├── favicon.ico
|
||||
│ │ │ ├── hero-moon.png
|
||||
│ │ │ ├── runic-emblem.png
|
||||
│ │ │ └── uomysticmoon-main-hero.png
|
||||
│ │ └── robots.txt
|
||||
│ ├── src/
|
||||
│ │ ├── api/
|
||||
│ │ │ └── client.js
|
||||
│ │ ├── blocks/
|
||||
│ │ │ ├── types/
|
||||
│ │ │ │ ├── cta.jsx
|
||||
│ │ │ │ ├── divider.jsx
|
||||
│ │ │ │ ├── heading.jsx
|
||||
│ │ │ │ ├── image.jsx
|
||||
│ │ │ │ ├── quote.jsx
|
||||
│ │ │ │ ├── richText.jsx
|
||||
│ │ │ │ └── twoColumn.jsx
|
||||
│ │ │ ├── BlockRenderer.jsx
|
||||
│ │ │ ├── editorKit.jsx
|
||||
│ │ │ ├── index.js
|
||||
│ │ │ └── registry.js
|
||||
│ │ ├── components/
|
||||
│ │ │ ├── security/
|
||||
│ │ │ │ ├── RecoveryCodesDisplay.jsx
|
||||
│ │ │ │ ├── RecoveryCodesPanel.jsx
|
||||
│ │ │ │ ├── TrustedDevicesPanel.jsx
|
||||
│ │ │ │ └── TrustLimitModal.jsx
|
||||
│ │ │ ├── CharacterSheet.jsx
|
||||
│ │ │ ├── CharacterStats.jsx
|
||||
│ │ │ ├── CreateGameAccountForm.jsx
|
||||
│ │ │ ├── GameAccounts.jsx
|
||||
│ │ │ ├── HeroElement.jsx
|
||||
│ │ │ ├── MaintenanceGate.jsx
|
||||
│ │ │ ├── Modal.jsx
|
||||
│ │ │ ├── MoonDot.jsx
|
||||
│ │ │ ├── PageHeader.jsx
|
||||
│ │ │ ├── PageState.jsx
|
||||
│ │ │ ├── PlayersOnline.jsx
|
||||
│ │ │ ├── ProviderIcon.jsx
|
||||
│ │ │ ├── PublicLayout.jsx
|
||||
│ │ │ ├── RequireAuth.jsx
|
||||
│ │ │ ├── RequirePlayer.jsx
|
||||
│ │ │ ├── RichTextEditor.jsx
|
||||
│ │ │ ├── RoleGate.jsx
|
||||
│ │ │ ├── ShardAccountActions.jsx
|
||||
│ │ │ ├── SiteFooter.jsx
|
||||
│ │ │ ├── SiteHeader.jsx
|
||||
│ │ │ └── VendorSales.jsx
|
||||
│ │ ├── contexts/
|
||||
│ │ │ ├── AuthContext.jsx
|
||||
│ │ │ └── SiteContext.jsx
|
||||
│ │ ├── data/
|
||||
│ │ │ ├── cityCrests.js
|
||||
│ │ │ └── regionBuckets.js
|
||||
│ │ ├── lib/
|
||||
│ │ │ ├── format.js
|
||||
│ │ │ ├── heroLayout.js
|
||||
│ │ │ ├── shardEvents.js
|
||||
│ │ │ ├── useAsync.js
|
||||
│ │ │ └── useShardFeed.js
|
||||
│ │ ├── routes/
|
||||
│ │ │ ├── admin/
|
||||
│ │ │ │ ├── views/
|
||||
│ │ │ │ │ ├── AccountAdmin.jsx
|
||||
│ │ │ │ │ ├── ActivityAdmin.jsx
|
||||
│ │ │ │ │ ├── AdminCharacter.jsx
|
||||
│ │ │ │ │ ├── AdminCharacters.jsx
|
||||
│ │ │ │ │ ├── Appeals.jsx
|
||||
│ │ │ │ │ ├── AuthProvidersAdmin.jsx
|
||||
│ │ │ │ │ ├── BotActivityAdmin.jsx
|
||||
│ │ │ │ │ ├── Dashboard.jsx
|
||||
│ │ │ │ │ ├── DiscordBotAdmin.jsx
|
||||
│ │ │ │ │ ├── EmailDelivery.jsx
|
||||
│ │ │ │ │ ├── HeroEditor.jsx
|
||||
│ │ │ │ │ ├── HousesAdmin.jsx
|
||||
│ │ │ │ │ ├── InvitesAdmin.jsx
|
||||
│ │ │ │ │ ├── Moderation.jsx
|
||||
│ │ │ │ │ ├── ModerationUser.jsx
|
||||
│ │ │ │ │ ├── PageBuilder.jsx
|
||||
│ │ │ │ │ ├── PagesAdmin.jsx
|
||||
│ │ │ │ │ ├── PostEditor.jsx
|
||||
│ │ │ │ │ ├── PostsAdmin.jsx
|
||||
│ │ │ │ │ ├── SettingsAdmin.jsx
|
||||
│ │ │ │ │ ├── ShardAdmin.jsx
|
||||
│ │ │ │ │ ├── ShardOps.jsx
|
||||
│ │ │ │ │ ├── UserDetail.jsx
|
||||
│ │ │ │ │ ├── UserEditor.jsx
|
||||
│ │ │ │ │ ├── UsersAdmin.jsx
|
||||
│ │ │ │ │ ├── WikiAdmin.jsx
|
||||
│ │ │ │ │ ├── WikiCategories.jsx
|
||||
│ │ │ │ │ ├── WikiEditor.jsx
|
||||
│ │ │ │ │ └── WikiHistory.jsx
|
||||
│ │ │ │ ├── AdminLayout.jsx
|
||||
│ │ │ │ └── AdminLogin.jsx
|
||||
│ │ │ ├── player/
|
||||
│ │ │ │ ├── AcceptInvite.jsx
|
||||
│ │ │ │ ├── ForgotPassword.jsx
|
||||
│ │ │ │ ├── PlayerAccount.jsx
|
||||
│ │ │ │ ├── PlayerAppeals.jsx
|
||||
│ │ │ │ ├── PlayerCharacter.jsx
|
||||
│ │ │ │ ├── PlayerCharacters.jsx
|
||||
│ │ │ │ ├── PlayerLogin.jsx
|
||||
│ │ │ │ ├── PlayerPortalLayout.jsx
|
||||
│ │ │ │ ├── PlayerRegister.jsx
|
||||
│ │ │ │ ├── PlayerShell.jsx
|
||||
│ │ │ │ └── ResetPassword.jsx
|
||||
│ │ │ ├── public/
|
||||
│ │ │ │ ├── About.jsx
|
||||
│ │ │ │ ├── ChampSpawns.jsx
|
||||
│ │ │ │ ├── CmsPage.jsx
|
||||
│ │ │ │ ├── FiveOnFriday.jsx
|
||||
│ │ │ │ ├── Governors.jsx
|
||||
│ │ │ │ ├── Guilds.jsx
|
||||
│ │ │ │ ├── Houses.jsx
|
||||
│ │ │ │ ├── Maintenance.jsx
|
||||
│ │ │ │ ├── News.jsx
|
||||
│ │ │ │ ├── Newsletter.jsx
|
||||
│ │ │ │ ├── NewsletterIssue.jsx
|
||||
│ │ │ │ ├── Portal.jsx
|
||||
│ │ │ │ ├── Screenshots.jsx
|
||||
│ │ │ │ ├── Shard.jsx
|
||||
│ │ │ │ ├── ShardActivity.jsx
|
||||
│ │ │ │ ├── Status.jsx
|
||||
│ │ │ │ └── Website.jsx
|
||||
│ │ │ └── wiki/
|
||||
│ │ │ ├── Wiki.jsx
|
||||
│ │ │ └── WikiArticle.jsx
|
||||
│ │ ├── styles/
|
||||
│ │ │ └── theme.css
|
||||
│ │ ├── App.jsx
|
||||
│ │ └── main.jsx
|
||||
│ ├── test/
|
||||
│ │ ├── apiClient.test.js
|
||||
│ │ ├── format.test.js
|
||||
│ │ ├── heroLayout.test.js
|
||||
│ │ ├── regionBuckets.test.js
|
||||
│ │ └── shardEvents.test.js
|
||||
│ ├── index.html
|
||||
│ ├── package-lock.json
|
||||
│ ├── package.json
|
||||
│ └── vite.config.js
|
||||
├── ntfy/
|
||||
│ └── server.yml
|
||||
├── scripts/
|
||||
│ ├── dev/
|
||||
│ │ ├── README.md
|
||||
│ │ ├── seed-sso-provider.js
|
||||
│ │ ├── sso-bridge-smoketest.js
|
||||
│ │ └── stub-idp.js
|
||||
│ └── sonar-test-reporter.mjs
|
||||
├── server/
|
||||
│ ├── db/
|
||||
│ │ ├── schema.sql
|
||||
│ │ └── seed.js
|
||||
│ ├── scripts/
|
||||
│ │ └── routeManifest.js
|
||||
│ ├── src/
|
||||
│ │ ├── auth/
|
||||
│ │ │ ├── providers/
|
||||
│ │ │ │ ├── base.provider.js
|
||||
│ │ │ │ ├── discord.provider.js
|
||||
│ │ │ │ ├── genericOidc.provider.js
|
||||
│ │ │ │ ├── google.provider.js
|
||||
│ │ │ │ ├── local.provider.js
|
||||
│ │ │ │ ├── oauth2.provider.js
|
||||
│ │ │ │ └── registry.js
|
||||
│ │ │ ├── session.middleware.js
|
||||
│ │ │ ├── session.service.js
|
||||
│ │ │ ├── ssoState.js
|
||||
│ │ │ ├── token.js
|
||||
│ │ │ └── usernamePolicy.js
|
||||
│ │ ├── blocks/
|
||||
│ │ │ ├── types/
|
||||
│ │ │ │ ├── cta.js
|
||||
│ │ │ │ ├── divider.js
|
||||
│ │ │ │ ├── heading.js
|
||||
│ │ │ │ ├── image.js
|
||||
│ │ │ │ ├── quote.js
|
||||
│ │ │ │ ├── richText.js
|
||||
│ │ │ │ └── twoColumn.js
|
||||
│ │ │ ├── index.js
|
||||
│ │ │ ├── propHelpers.js
|
||||
│ │ │ ├── registry.js
|
||||
│ │ │ ├── sanitizeBlocks.js
|
||||
│ │ │ └── validateBlocks.js
|
||||
│ │ ├── config/
|
||||
│ │ │ ├── brand.js
|
||||
│ │ │ ├── csp.js
|
||||
│ │ │ ├── notificationStreams.js
|
||||
│ │ │ └── version.js
|
||||
│ │ ├── middleware/
|
||||
│ │ │ ├── botScore.js
|
||||
│ │ │ ├── loginProtection.js
|
||||
│ │ │ ├── noindex.js
|
||||
│ │ │ ├── rateLimit.js
|
||||
│ │ │ ├── requireInternalKey.js
|
||||
│ │ │ ├── siteMode.js
|
||||
│ │ │ └── validate.js
|
||||
│ │ ├── model/
|
||||
│ │ │ ├── activity/
|
||||
│ │ │ │ ├── activity.db.js
|
||||
│ │ │ │ └── activity.model.js
|
||||
│ │ │ ├── announceJobs/
|
||||
│ │ │ │ ├── announceJobs.db.js
|
||||
│ │ │ │ ├── announceJobs.logic.js
|
||||
│ │ │ │ └── announceJobs.model.js
|
||||
│ │ │ ├── appeals/
|
||||
│ │ │ │ ├── appeals.db.js
|
||||
│ │ │ │ ├── appeals.model.js
|
||||
│ │ │ │ └── appeals.pure.js
|
||||
│ │ │ ├── authProviders/
|
||||
│ │ │ │ ├── authProviders.db.js
|
||||
│ │ │ │ └── authProviders.model.js
|
||||
│ │ │ ├── botConfig/
|
||||
│ │ │ │ ├── botConfig.db.js
|
||||
│ │ │ │ └── botConfig.model.js
|
||||
│ │ │ ├── emailConfig/
|
||||
│ │ │ │ ├── emailConfig.db.js
|
||||
│ │ │ │ └── emailConfig.model.js
|
||||
│ │ │ ├── invites/
|
||||
│ │ │ │ ├── invites.db.js
|
||||
│ │ │ │ └── invites.model.js
|
||||
│ │ │ ├── mobileAuthBridge/
|
||||
│ │ │ │ ├── mobileAuthBridge.db.js
|
||||
│ │ │ │ └── mobileAuthBridge.model.js
|
||||
│ │ │ ├── mobileSessions/
|
||||
│ │ │ │ ├── mobileSessions.db.js
|
||||
│ │ │ │ └── mobileSessions.model.js
|
||||
│ │ │ ├── moderation/
|
||||
│ │ │ │ ├── moderation.db.js
|
||||
│ │ │ │ ├── moderation.model.js
|
||||
│ │ │ │ └── moderation.pure.js
|
||||
│ │ │ ├── modNotes/
|
||||
│ │ │ │ ├── modNotes.db.js
|
||||
│ │ │ │ └── modNotes.model.js
|
||||
│ │ │ ├── notificationSubs/
|
||||
│ │ │ │ ├── notificationSubs.db.js
|
||||
│ │ │ │ └── notificationSubs.model.js
|
||||
│ │ │ ├── pages/
|
||||
│ │ │ │ ├── pages.db.js
|
||||
│ │ │ │ ├── pages.model.js
|
||||
│ │ │ │ └── reservedSlugs.js
|
||||
│ │ │ ├── passwordResets/
|
||||
│ │ │ │ ├── passwordResets.db.js
|
||||
│ │ │ │ └── passwordResets.model.js
|
||||
│ │ │ ├── posts/
|
||||
│ │ │ │ ├── posts.db.js
|
||||
│ │ │ │ └── posts.model.js
|
||||
│ │ │ ├── pushDevices/
|
||||
│ │ │ │ ├── pushDevices.db.js
|
||||
│ │ │ │ └── pushDevices.model.js
|
||||
│ │ │ ├── recoveryCodes/
|
||||
│ │ │ │ ├── recoveryCodes.db.js
|
||||
│ │ │ │ └── recoveryCodes.model.js
|
||||
│ │ │ ├── revokedSessions/
|
||||
│ │ │ │ ├── revokedSessions.db.js
|
||||
│ │ │ │ └── revokedSessions.model.js
|
||||
│ │ │ ├── settings/
|
||||
│ │ │ │ ├── settings.db.js
|
||||
│ │ │ │ └── settings.model.js
|
||||
│ │ │ ├── shardEvents/
|
||||
│ │ │ │ ├── shardEvents.db.js
|
||||
│ │ │ │ └── shardEvents.model.js
|
||||
│ │ │ ├── shardLinks/
|
||||
│ │ │ │ ├── shardLinks.db.js
|
||||
│ │ │ │ └── shardLinks.model.js
|
||||
│ │ │ ├── shardState/
|
||||
│ │ │ │ ├── shardState.db.js
|
||||
│ │ │ │ └── shardState.model.js
|
||||
│ │ │ ├── trustedDevices/
|
||||
│ │ │ │ ├── trustedDevices.db.js
|
||||
│ │ │ │ └── trustedDevices.model.js
|
||||
│ │ │ ├── uoLinkConfig/
|
||||
│ │ │ │ ├── uoLinkConfig.db.js
|
||||
│ │ │ │ └── uoLinkConfig.model.js
|
||||
│ │ │ ├── userIdentities/
|
||||
│ │ │ │ ├── userIdentities.db.js
|
||||
│ │ │ │ └── userIdentities.model.js
|
||||
│ │ │ ├── users/
|
||||
│ │ │ │ ├── users.db.js
|
||||
│ │ │ │ └── users.model.js
|
||||
│ │ │ ├── wiki/
|
||||
│ │ │ │ ├── wiki.db.js
|
||||
│ │ │ │ ├── wiki.links.js
|
||||
│ │ │ │ └── wiki.model.js
|
||||
│ │ │ └── singletonConfigDb.js
|
||||
│ │ ├── router/
|
||||
│ │ │ ├── v1/
|
||||
│ │ │ │ ├── admin/
|
||||
│ │ │ │ │ ├── account.controller.js
|
||||
│ │ │ │ │ ├── account.router.js
|
||||
│ │ │ │ │ ├── activity.router.js
|
||||
│ │ │ │ │ ├── admin.controller.js
|
||||
│ │ │ │ │ ├── admin.routes.js
|
||||
│ │ │ │ │ ├── authProviders.controller.js
|
||||
│ │ │ │ │ ├── authProviders.router.js
|
||||
│ │ │ │ │ ├── botActivity.controller.js
|
||||
│ │ │ │ │ ├── botActivity.router.js
|
||||
│ │ │ │ │ ├── discordBot.controller.js
|
||||
│ │ │ │ │ ├── emailConfig.controller.js
|
||||
│ │ │ │ │ ├── imageUpload.js
|
||||
│ │ │ │ │ ├── index.js
|
||||
│ │ │ │ │ ├── invites.controller.js
|
||||
│ │ │ │ │ ├── invites.router.js
|
||||
│ │ │ │ │ ├── moderation.controller.js
|
||||
│ │ │ │ │ ├── moderation.router.js
|
||||
│ │ │ │ │ ├── pages.controller.js
|
||||
│ │ │ │ │ ├── pages.router.js
|
||||
│ │ │ │ │ ├── posts.router.js
|
||||
│ │ │ │ │ ├── shardOps.controller.js
|
||||
│ │ │ │ │ ├── uoLink.controller.js
|
||||
│ │ │ │ │ ├── uploads.router.js
|
||||
│ │ │ │ │ ├── users.router.js
|
||||
│ │ │ │ │ ├── usersShard.controller.js
|
||||
│ │ │ │ │ └── wiki.router.js
|
||||
│ │ │ │ ├── auth/
|
||||
│ │ │ │ │ ├── auth.controller.js
|
||||
│ │ │ │ │ ├── auth.routes.js
|
||||
│ │ │ │ │ ├── invite.controller.js
|
||||
│ │ │ │ │ ├── me.routes.js
|
||||
│ │ │ │ │ ├── mobile.controller.js
|
||||
│ │ │ │ │ ├── mobile.routes.js
|
||||
│ │ │ │ │ ├── mobileSso.controller.js
|
||||
│ │ │ │ │ ├── mobileSso.routes.js
|
||||
│ │ │ │ │ ├── notifications.controller.js
|
||||
│ │ │ │ │ ├── notifications.routes.js
|
||||
│ │ │ │ │ ├── passwordReset.controller.js
|
||||
│ │ │ │ │ ├── sso.controller.js
|
||||
│ │ │ │ │ ├── sso.routes.js
|
||||
│ │ │ │ │ └── trustDevice.helper.js
|
||||
│ │ │ │ ├── internal/
|
||||
│ │ │ │ │ ├── internal.controller.js
|
||||
│ │ │ │ │ └── internal.routes.js
|
||||
│ │ │ │ ├── player/
|
||||
│ │ │ │ │ ├── appeals.controller.js
|
||||
│ │ │ │ │ ├── player.routes.js
|
||||
│ │ │ │ │ └── shard.controller.js
|
||||
│ │ │ │ ├── public/
|
||||
│ │ │ │ │ ├── public.controller.js
|
||||
│ │ │ │ │ ├── public.routes.js
|
||||
│ │ │ │ │ └── shard.controller.js
|
||||
│ │ │ │ └── v1.router.js
|
||||
│ │ │ ├── api.router.js
|
||||
│ │ │ ├── cspReport.controller.js
|
||||
│ │ │ └── wellKnown.controller.js
|
||||
│ │ ├── utils/
|
||||
│ │ │ ├── announceWorker.js
|
||||
│ │ │ ├── auth.js
|
||||
│ │ │ ├── botInternalClient.js
|
||||
│ │ │ ├── botInternalKey.js
|
||||
│ │ │ ├── db.js
|
||||
│ │ │ ├── logger.js
|
||||
│ │ │ ├── mailer.js
|
||||
│ │ │ ├── newsGump.js
|
||||
│ │ │ ├── pushDispatch.js
|
||||
│ │ │ ├── sanitizeHtml.js
|
||||
│ │ │ ├── secretBox.js
|
||||
│ │ │ ├── shardBroadcast.js
|
||||
│ │ │ ├── shardIngest.js
|
||||
│ │ │ ├── shardSales.js
|
||||
│ │ │ ├── totp.js
|
||||
│ │ │ ├── trustProxy.js
|
||||
│ │ │ ├── uoLinkClient.js
|
||||
│ │ │ └── uoLinkSocket.js
|
||||
│ │ ├── app.js
|
||||
│ │ ├── internalApp.js
|
||||
│ │ └── server.js
|
||||
│ ├── swagger/
|
||||
│ │ ├── swagger-output.json
|
||||
│ │ └── swagger.js
|
||||
│ ├── test/
|
||||
│ │ ├── _helper.js
|
||||
│ │ ├── adminTrustedDevices.test.js
|
||||
│ │ ├── adminUserShard.test.js
|
||||
│ │ ├── announceJobs.test.js
|
||||
│ │ ├── appeals.pure.test.js
|
||||
│ │ ├── appeals.test.js
|
||||
│ │ ├── appLinks.test.js
|
||||
│ │ ├── authController.test.js
|
||||
│ │ ├── authMe.test.js
|
||||
│ │ ├── authTrustedDevice.test.js
|
||||
│ │ ├── botInternalKey.test.js
|
||||
│ │ ├── botScore.test.js
|
||||
│ │ ├── csp.test.js
|
||||
│ │ ├── emailConfig.model.test.js
|
||||
│ │ ├── honeypot.test.js
|
||||
│ │ ├── inviteController.test.js
|
||||
│ │ ├── invites.test.js
|
||||
│ │ ├── loginProtection.test.js
|
||||
│ │ ├── mailer.test.js
|
||||
│ │ ├── mobileAuthBridge.model.test.js
|
||||
│ │ ├── mobileDeviceSessions.test.js
|
||||
│ │ ├── mobileSession.test.js
|
||||
│ │ ├── mobileSsoBridge.test.js
|
||||
│ │ ├── moderation.model.test.js
|
||||
│ │ ├── moderation.test.js
|
||||
│ │ ├── newsGump.test.js
|
||||
│ │ ├── notificationsRoutes.test.js
|
||||
│ │ ├── pages.model.test.js
|
||||
│ │ ├── passwordResetController.test.js
|
||||
│ │ ├── passwordResets.test.js
|
||||
│ │ ├── playerAccounts.test.js
|
||||
│ │ ├── playerRouteAccess.test.js
|
||||
│ │ ├── providers.test.js
|
||||
│ │ ├── publicBrand.test.js
|
||||
│ │ ├── publicController.test.js
|
||||
│ │ ├── publicShardOnline.test.js
|
||||
│ │ ├── publicVersion.test.js
|
||||
│ │ ├── pushDispatch.test.js
|
||||
│ │ ├── recoveryCodes.test.js
|
||||
│ │ ├── registry.test.js
|
||||
│ │ ├── requireInternalKey.test.js
|
||||
│ │ ├── routeManifest.test.js
|
||||
│ │ ├── secretBox.test.js
|
||||
│ │ ├── selfTrustedDevices.test.js
|
||||
│ │ ├── session.test.js
|
||||
│ │ ├── shardControllerPublic.test.js
|
||||
│ │ ├── shardIngest.champsPages.test.js
|
||||
│ │ ├── shardIngest.protocol2.test.js
|
||||
│ │ ├── shardState.governorTerms.test.js
|
||||
│ │ ├── shardState.model.test.js
|
||||
│ │ ├── ssoCallback.test.js
|
||||
│ │ ├── ssoState.test.js
|
||||
│ │ ├── totp.test.js
|
||||
│ │ ├── trustedDevices.test.js
|
||||
│ │ ├── trustProxy.test.js
|
||||
│ │ └── usernamePolicy.test.js
|
||||
│ ├── .env.example
|
||||
│ ├── package-lock.json
|
||||
│ ├── package.json
|
||||
│ ├── routes.guards.json
|
||||
│ └── routes.manifest.json
|
||||
├── .dockerignore
|
||||
├── .env.example
|
||||
├── .env.uomysticmoon.example
|
||||
├── .gitignore
|
||||
├── CODE_OF_CONDUCT.md
|
||||
├── CONTRIBUTING.md
|
||||
├── CONTRIBUTORS.md
|
||||
├── docker-compose.dev.yml
|
||||
├── docker-compose.yml
|
||||
├── Dockerfile
|
||||
├── LICENSE.md
|
||||
├── package.json
|
||||
├── README.md
|
||||
├── SECURITY.md
|
||||
└── sonar-project.properties
|
||||
```
|
||||
@@ -1,158 +0,0 @@
|
||||
# Shard visibility — who sees which shard data
|
||||
|
||||
**Status:** Built (Protocol 3.0 Part A). Admin → Shard Visibility.
|
||||
**Audience:** shard owners and admins.
|
||||
**Companion to** [`../link/v3.md`](../link/v3.md) §3 (the design) and
|
||||
[`BACKEND_DESIGN.md`](BACKEND_DESIGN.md) §6 (the security contract).
|
||||
|
||||
The website surfaces a lot of live shard data. What your players, your staff and the anonymous
|
||||
internet may each see is **yours to decide**, per feature, from Admin → Shard Visibility.
|
||||
|
||||
Nothing changes until you change it: every setting ships at the value that reproduces how the site
|
||||
behaved before this panel existed.
|
||||
|
||||
---
|
||||
|
||||
## 1. The audience ladder
|
||||
|
||||
Five rungs. Each one includes everyone below it.
|
||||
|
||||
| Rung | In the UI | Who that is |
|
||||
|---|---|---|
|
||||
| `anonymous` | **Everyone** | Anyone at all, signed in or not. |
|
||||
| `logged_in` | **Signed in** | Any registered account, whether or not they've linked a game account. |
|
||||
| `player` | **Linked players** | Accounts with a linked in-game account. **Staff always qualify**, linked or not. |
|
||||
| `staff` | **Staff** | Admins and moderators. |
|
||||
| `admin` | **Admins only** | Admins. |
|
||||
|
||||
Two notes that surprise people:
|
||||
|
||||
- **`editor` is a content role, not a shard role.** Editors write news and wiki pages; they get no
|
||||
shard privilege from that. An editor is treated by link status like any other member. This matches
|
||||
the rest of the site, where shard staff powers are admin-or-moderator.
|
||||
- **Staff satisfy `player` without linking.** Otherwise an admin would be locked out of surfaces
|
||||
they'd gated to players, which is how the `/player/*` routes already behave.
|
||||
|
||||
## 2. What you can set per feature
|
||||
|
||||
**Enabled.** Off means gone. The feature's pages return “not found”, not “forbidden” — a disabled
|
||||
feature doesn't advertise that it exists.
|
||||
|
||||
**Who can see it.** The minimum rung, from the ladder above.
|
||||
|
||||
**Live updates.** Whether this feature pushes changes to open pages in real time. Turning it off
|
||||
doesn't break the page; it just refreshes on load instead of updating in place.
|
||||
|
||||
**Sensitive fields.** Some features expose a field that deserves its own rung — you can publish the
|
||||
board while holding back one column. See the table in §3.
|
||||
|
||||
## 3. The features, and their defaults
|
||||
|
||||
| Feature | What it exposes | Default | Sensitive fields |
|
||||
|---|---|---|---|
|
||||
| **Shard status** | Connection state, online count, gold-supply series | Everyone | — |
|
||||
| **Activity feed** | Deaths, kills, skill gains, quests, logins | Everyone | — |
|
||||
| **Champion spawns** | The live champion / mini-champ / sea-boss board | Everyone | — |
|
||||
| **Guilds** | Guild rosters, alliances, leaders | Everyone | — |
|
||||
| **Town governors** | City Loyalty governors, elections, term history | Everyone | — |
|
||||
| **Houses / IDOC** | Houses in danger | Everyone | House owner → Staff · House price → Staff |
|
||||
| **Players online** | Population aggregate, staff-online widget | Everyone | In-game location → Staff |
|
||||
| **Shard rules** | Skill/stat caps, house limits, vet rewards, the ruleset | Everyone | Connect address → Everyone |
|
||||
| **Spawn atlas** | Bestiary and spawn locations (static content) | Everyone | — |
|
||||
| **Leaderboards** | Point and loyalty standings | Everyone | Character names → Everyone |
|
||||
| **Marketplace** | The shard-wide player-vendor index | Everyone, **live updates off** | Vendor owner name → Everyone · Vendor owner character id → Everyone · In-game location → Everyone |
|
||||
|
||||
**Why the marketplace ships with live updates off.** A live feed of every vendor's full inventory
|
||||
would be the single largest thing the site sends. No page needs it — the marketplace is a search over
|
||||
stored data with a “prices last refreshed N minutes ago” stamp. Turn it on only if you want it.
|
||||
|
||||
**Why the marketplace's fields default to Everyone.** A vendor's shop name, its owner's character
|
||||
name and where it is standing are *already* visible to every player in game: the stock Vendor Search
|
||||
gump surfaces exactly that set to anyone who opens it. Publishing them on the site is not a new
|
||||
disclosure. They stay configurable because a shard may still prefer to keep its economy behind a
|
||||
login — and because "already public in game" is a judgement about your shard, not ours.
|
||||
|
||||
**Location is one setting covering four things.** Hiding it removes the facet, the coordinates, the
|
||||
region *and* the house name together. That is deliberate: those are four ways of saying the same
|
||||
thing, and a setting that hid the coordinates while publishing the house name would not have hidden
|
||||
anything.
|
||||
|
||||
**Hiding the owner name also hides the owner character id.** They are separate settings so you can
|
||||
be explicit, but leaving the id published while hiding the name achieves nothing — the leaderboards
|
||||
and guild boards resolve that same id back to a character name. Set both.
|
||||
|
||||
**What hiding a vendor cannot do.** Only vendors whose owner left the in-game *Vendor Search* flag ON
|
||||
are ever sent to the site, so a player who hides their shop in game is hidden here too — and no
|
||||
setting on this page can override that. It works the other way as well: these settings control who
|
||||
sees the index, not whether players can find each other's shops in game.
|
||||
|
||||
**Why house owner/price default to Staff.** The public Houses page has always been a "where are the
|
||||
falling houses" board — location only. Owner and price are the staff view. That split is preserved.
|
||||
|
||||
## 4. What you cannot change
|
||||
|
||||
Two rules are enforced in code and are not settings. Attempting to set them returns an error rather
|
||||
than silently ignoring you.
|
||||
|
||||
**1. Game account names and website user ids are admin-only, always.**
|
||||
`acct` and `webId` never appear below the admin rung on any surface. A character *name* is visible in
|
||||
game to anyone standing next to them; the **account** behind it is not, and neither is the website
|
||||
user it's linked to. Publishing those would disclose something the shard itself doesn't, and would
|
||||
tie a player's in-game identity to their forum identity without their consent.
|
||||
|
||||
This rule matches the *meaning* of a field, not one spelling of it. Some responses nest the player
|
||||
who owns a record (`leader.acct`); others flatten it into the row (`ownerAcct`, `leaderWebId`,
|
||||
`governorAcct`). Every one of those is locked, and the admin API refuses to configure any of them —
|
||||
so a new response shape can't quietly reopen the hole by naming the field differently.
|
||||
|
||||
**2. Unknown event kinds are never broadcast below admin.**
|
||||
The live stream maps each event kind to a feature. A kind with no mapping — a new event from a shard
|
||||
plugin the site doesn't know yet, say — goes to admins only. It fails closed. This is what keeps the
|
||||
stream safe by default when the shard starts sending something new: the worst case is that staff see
|
||||
it and players don't, never the reverse.
|
||||
|
||||
## 5. How it's enforced
|
||||
|
||||
Three places, one config:
|
||||
|
||||
- **Page and API requests** are checked before the handler runs, and the response is then stripped of
|
||||
any field above the caller's rung.
|
||||
- **The live stream** resolves a viewer's rung once, when they connect, and freezes it for that
|
||||
connection — a long-open page can't gain privilege because something changed underneath it. Each
|
||||
event is then gated and stripped per viewer, so two people watching the same page can legitimately
|
||||
receive different versions of the same event, or one of them nothing.
|
||||
- **Navigation** hides links a viewer can't follow, so they don't hit a wall. This is presentation
|
||||
only — the gate is server-side either way.
|
||||
|
||||
**Stored history answers the same way the live stream does.** The activity feed reads from the event
|
||||
log rather than the live stream, but it resolves the *same* question against the *same* config: which
|
||||
kinds you may read, and which fields survive. So moving a feature up a rung hides it from the history
|
||||
as well as the stream — there is no back door where yesterday's copy of an event is more revealing
|
||||
than today's.
|
||||
|
||||
One deliberate asymmetry: turning **live updates** off for a feature stops the push, not the reading.
|
||||
The marketplace ships this way — its history and its pages are public, only the firehose is off.
|
||||
|
||||
Changes take effect within about five seconds, **including on streams that are already open**. You
|
||||
don't need to restart anything.
|
||||
|
||||
If the database is briefly unreachable, the site falls back to the built-in defaults — the pre-v3
|
||||
behavior — rather than to "everything is public".
|
||||
|
||||
## 6. Worked examples
|
||||
|
||||
**"I want a private shard — nothing public until people register."**
|
||||
Set every feature to **Signed in**. Anonymous visitors still get the site itself; the shard data
|
||||
disappears from the nav.
|
||||
|
||||
**"Publish the market, but don't tie vendors to players."**
|
||||
Marketplace → Everyone, with **Vendor owner name** → Staff. Prices, items and locations stay public;
|
||||
who owns each vendor doesn't.
|
||||
|
||||
**"Leaderboards for members only."**
|
||||
Leaderboards → **Linked players**. Anyone who's linked a game account sees the standings; drive-by
|
||||
visitors don't.
|
||||
|
||||
**"Let players see house owners."**
|
||||
Houses → Everyone, **House owner** → Linked players. Note this is a real disclosure: house ownership
|
||||
is visible in game, but the website makes it searchable in a way the game doesn't.
|
||||
@@ -1,354 +0,0 @@
|
||||
# Spawn atlas
|
||||
|
||||
**Status:** Complete on `edge` — data pipeline in website [#112](https://gitea.whitlocktech.com/RunicGateway/website/pulls/112), API + pages in website [#113](https://gitea.whitlocktech.com/RunicGateway/website/pulls/113).
|
||||
**Design:** [`docs/link/v3.md` §6](../link/v3.md) — Protocol 3.0 Part C.
|
||||
|
||||
The spawn atlas is a browsable catalogue of what the shard *contains*: which
|
||||
creatures spawn, where, how many, and which champion altars are configured. It
|
||||
answers "where do I find a lizardman?" with **"Shrines, Yew, Isamu-Jima"** rather
|
||||
than with a list of raw coordinates.
|
||||
|
||||
## Two things that shape the whole design
|
||||
|
||||
**The shard's ServUO tree is the single source of truth.** Nothing is
|
||||
precomputed and committed to the repository. A shard's maps change over its
|
||||
lifetime — facets get added, replaced, or renamed — and a snapshot in the repo
|
||||
would silently drift from the world players actually see. The atlas is therefore
|
||||
re-derived from the tree **on every server boot**.
|
||||
|
||||
**Facets are not a fixed list.** Nothing in the codebase names Felucca, Trammel,
|
||||
or any other stock facet. The facet set is whatever the shard's own files
|
||||
declare, discovered at parse time. A shard running entirely custom maps gets
|
||||
exactly the same treatment as a stock one, with no code change.
|
||||
|
||||
## What it is not
|
||||
|
||||
The atlas is **static shard content, not live shard state.**
|
||||
|
||||
- It does **not** come from the sidecar. Nothing here touches the bridge, and
|
||||
there is no event kind, no wire change and no `PROTOCOL_VERSION` bump for it.
|
||||
Part C is website-only.
|
||||
- It stays fully populated while the shard is down.
|
||||
- Its champion table (`shard_champion_spawns`) is the *configured roster* —
|
||||
"there is an Unholy Terror altar in Deceit". The live `champ.update` feed in
|
||||
`shard_champs` is the separate, sidecar-fed answer to "it is on level 3 right
|
||||
now". Both exist; do not conflate them.
|
||||
|
||||
Routes live at `/api/v1/public/atlas`, deliberately **not** under `/shard`,
|
||||
because `/shard/*` means sidecar-dependent.
|
||||
|
||||
## Configuring the tree
|
||||
|
||||
The website needs to be able to *read* the ServUO tree — same host, a bind mount,
|
||||
or a shared volume. Two ways to point at it, the setting winning over the
|
||||
environment:
|
||||
|
||||
| Source | Notes |
|
||||
|---|---|
|
||||
| `spawn_atlas_servuo_path` setting | Admin-editable; changes take effect on the next refresh without a redeploy |
|
||||
| `SERVUO_PATH` env var | The deploy-time default, since the path usually describes a mount the deployment sets up |
|
||||
|
||||
With neither set the atlas is simply skipped — the site runs normally without
|
||||
one.
|
||||
|
||||
## The boot path
|
||||
|
||||
On every start the server hashes the source files and compares them against what
|
||||
is loaded. Unchanged (the normal case on a restart) costs one read pass, ~120 ms,
|
||||
and no database write. A real change costs a ~400 ms parse and a reload.
|
||||
|
||||
Two contracts govern it:
|
||||
|
||||
**1. It never blocks startup.** No configured path, an unreadable mount, a
|
||||
malformed file, a database error — every one is caught and logged, and the site
|
||||
comes up serving whatever atlas it already had.
|
||||
|
||||
**2. A facet disappearing is never applied automatically.** Losing a facet looks
|
||||
exactly like a half-copied or mid-update tree, and boot cannot tell that apart
|
||||
from a real map change. That refresh is *staged* for a human instead. Everything
|
||||
else — new facets, renamed regions, changed spawns — applies immediately, since
|
||||
none of it can destroy something an operator would miss.
|
||||
|
||||
```
|
||||
boot
|
||||
└─ path configured? no ──▶ skip
|
||||
└─ tree readable? no ──▶ warn, carry on
|
||||
└─ hashes changed? no ──▶ done (nothing parsed)
|
||||
└─ parse
|
||||
└─ a facet would be removed?
|
||||
no ──▶ import
|
||||
yes ──▶ stage for admin review; atlas unchanged
|
||||
```
|
||||
|
||||
### Approving or rejecting a staged refresh
|
||||
|
||||
Only the *decision* is stored, never the parsed world — a few KB of source hashes
|
||||
plus the facet diff. Approving **re-parses** the tree, so what lands matches the
|
||||
tree at approval time rather than at boot, and a multi-megabyte blob never sits
|
||||
in the database.
|
||||
|
||||
A rejection is remembered against those exact source hashes, so a declined
|
||||
refresh does not re-prompt on every restart. Change the tree and the hashes
|
||||
differ, which asks again.
|
||||
|
||||
From **Admin → Spawn Atlas**, or from the CLI:
|
||||
|
||||
```bash
|
||||
cd website/server
|
||||
npm run atlas:import -- --status # what is loaded, and what is pending
|
||||
npm run atlas:import -- --approve # apply the staged refresh
|
||||
npm run atlas:import -- --reject # keep the current atlas, dismiss it
|
||||
```
|
||||
|
||||
## The CLI
|
||||
|
||||
The server refreshes itself on boot, so this is for applying a map change
|
||||
*without* a restart, and for the approve/reject flow above.
|
||||
|
||||
```bash
|
||||
npm run atlas:import # import if the tree differs
|
||||
npm run atlas:import -- --servuo <path> # override the path for this run
|
||||
npm run atlas:import -- --force # reimport even if unchanged
|
||||
```
|
||||
|
||||
`--servuo` is a per-run override and deliberately does **not** persist — changing
|
||||
where the atlas permanently reads from is an admin action, not a side effect of a
|
||||
one-off import.
|
||||
|
||||
## Sources
|
||||
|
||||
| File | Count (stock ServUO 57.4) | Used for |
|
||||
|---|---|---|
|
||||
| `Spawns/*.xml` | 13 files, ~10.5 MB | Every spawner: location, size, delays, time-of-day, creature types |
|
||||
| `Data/Regions.xml` | 129 KB, nested | Named regions and their rectangles |
|
||||
| `Data/Locations/*.xml` | 6 files | Landmarks (dungeon levels, town markers) |
|
||||
| `Config/ChampionSpawns.xml` | 4.8 KB | Configured champion altars |
|
||||
|
||||
**A stock tree has 13 spawn files but only 6 facets.** `Eodon.xml`,
|
||||
`GravewaterLake.xml`, `TreasuresOfKotl.xml` and the other named-area files hold
|
||||
TerMur/Trammel points. The facet always comes from each record's own `<Map>`,
|
||||
never from the file name.
|
||||
|
||||
## How a coordinate becomes a place name
|
||||
|
||||
This is the transform the atlas exists for, in `resolveRegion()`:
|
||||
|
||||
1. The highest-`priority` named region whose rectangle contains the point. Ties
|
||||
break toward the **smallest** rect, so a specific room wins over the
|
||||
dungeon-wide rect enclosing it.
|
||||
2. Otherwise the nearest landmark within the landmark radius (200 tiles by
|
||||
default), labelled by its **group** ("Covetous"), not its individual marker
|
||||
("Level 1").
|
||||
3. Otherwise `"Wilderness"`.
|
||||
|
||||
The radius cap in step 2 is what keeps step 3 reachable. Without it the nearest
|
||||
landmark is always *some* landmark however far away, and open countryside gets
|
||||
labelled with a dungeon on the far side of the map.
|
||||
|
||||
Against stock ServUO this resolves **83.2%** of points (5,369 of 6,455): 3,681 by
|
||||
region, 1,688 by landmark, 1,086 Wilderness.
|
||||
|
||||
## Three quirks in the source data
|
||||
|
||||
Each of these is silent if unhandled — the atlas still builds, it is just wrong.
|
||||
|
||||
**Facet names disagree between sources.** `Data/Locations/*.xml` spells them
|
||||
`Ter Mur` and `Tokuno Islands`, while `<Map>` and `<Facet name>` say `TerMur` and
|
||||
`Tokuno`. Unreconciled, the landmark bucket is keyed differently from the points
|
||||
looking it up, so the fallback never fires and every unregioned spawn on those
|
||||
facets reads "Wilderness".
|
||||
|
||||
This is reconciled **by matching, not by a lookup table** — there is no list of
|
||||
facet names anywhere. `facetKey()` collapses spelling differences (lowercase,
|
||||
alphanumerics only), and `resolveFacetName()` matches a loose spelling against
|
||||
the canonical set discovered from the shard's own spawn and region data, by exact
|
||||
key then by prefix in either direction. A name matching nothing keeps its own
|
||||
name: forcing a wrong match would file a real custom facet's landmarks under the
|
||||
wrong facet, which is worse than leaving it alone.
|
||||
|
||||
**Spawn type tokens carry XmlSpawner directives.** The `<Objects2>` type is not
|
||||
always a bare class name:
|
||||
|
||||
```
|
||||
Fairy,{RND,4,8} alchemist/z/-50 Agralem/Name/Agralem
|
||||
GargishRouser,1 greatape,true GargishRefugee/hue/34532
|
||||
```
|
||||
|
||||
Taken literally these invent creatures that do not exist *and* split real ones in
|
||||
two, because `Fairy` and `Fairy,{RND,4,8}` slug apart into separate entries. 71 of
|
||||
845 were affected. Everything from the first `/` or `,` is stripped, leaving 800
|
||||
real creatures.
|
||||
|
||||
**Case is inconsistent across files.** The same creature is `Lizardman` in one
|
||||
file and `lizardman` in another. Slugging collapses them correctly, but the
|
||||
display name is chosen deterministically — most common spelling wins, ties break
|
||||
to the more capitalised form, then alphabetically — because otherwise it would
|
||||
depend on file read order and change on an unrelated restart.
|
||||
|
||||
## Tables
|
||||
|
||||
All are **import-owned**: a refresh empties and reloads them in one transaction,
|
||||
so a failed reload leaves the previous atlas intact rather than a half-loaded
|
||||
world. Nothing else writes to them and nothing holds a foreign key to them — no
|
||||
FKs at all, consistent with every other `shard_*` table. Full column listings in
|
||||
[`BACKEND_DESIGN.md`](BACKEND_DESIGN.md).
|
||||
|
||||
| Table | Rows (stock) | Notes |
|
||||
|---|---|---|
|
||||
| `shard_spawn_creatures` | 800 | `slug` PK; `total` = sum of each type's own max; nullable `art` |
|
||||
| `shard_spawn_points` | 6,455 | `spawn_range`, since `range` is reserved in MariaDB |
|
||||
| `shard_spawn_point_types` | 23,927 | The many-to-many; one spawner commonly carries six types |
|
||||
| `shard_regions` | 387 | Flattened out of the nesting; `rects` JSON |
|
||||
| `shard_landmarks` | 558 | `grp`, since `group` is reserved in SQL |
|
||||
| `shard_champion_spawns` | 25 | Configured altars, not the live feed |
|
||||
| `shard_atlas_meta` | 1 | Singleton; source hashes, for the change check |
|
||||
| `shard_atlas_pending` | 0–1 | Singleton; a staged refresh awaiting admin review |
|
||||
|
||||
`shard_spawn_creatures.name` carries a plain `INDEX`, deliberately **not
|
||||
`FULLTEXT`**: ~800 rows makes a `LIKE` scan free, and FULLTEXT's minimum token
|
||||
length would break searches for names like "orc".
|
||||
|
||||
The reload uses `DELETE`, not `TRUNCATE` — `TRUNCATE` is DDL in MariaDB and would
|
||||
implicitly commit, defeating the all-or-nothing guarantee. Point ids are assigned
|
||||
explicitly rather than left to `AUTO_INCREMENT`, because the join rows need them
|
||||
and `conn.batch()` reports no usable `insertId`.
|
||||
|
||||
## Artwork — operator-supplied, never shipped
|
||||
|
||||
**This project ships no creature art and no extraction tooling, and never will.**
|
||||
UO sprites live in the operator's own client `.mul`/`.uop` files. They are the
|
||||
operator's, not ours to redistribute.
|
||||
|
||||
The atlas is fully functional as text. `shard_spawn_creatures.art` is nullable
|
||||
and is NULL on every fresh import; pages render without images, which is the
|
||||
normal and supported state, not a degraded one.
|
||||
|
||||
An operator who wants art:
|
||||
|
||||
1. Extracts it from **their own** client files (UOFiddler, ClassicUO tooling, or
|
||||
any art extractor).
|
||||
2. Drops the images under `server/uploads/atlas/`.
|
||||
3. Copies `server/db/data/spawnAtlas.art.example.json` to `spawnAtlas.art.json`
|
||||
and maps creature slugs to file names.
|
||||
4. Restarts, or runs `npm run atlas:import -- --force`.
|
||||
|
||||
Both `spawnAtlas.art.json` and `server/uploads/` are gitignored, so neither the
|
||||
map nor the images can be committed by accident.
|
||||
|
||||
## Code layout
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `src/utils/spawnAtlasParse.js` | **Pure and fs-free** parsers, so CI covers them with no ServUO tree. Zero dependencies. |
|
||||
| `src/utils/spawnAtlasSource.js` | The only thing that reads a ServUO tree; shared by the boot path and the CLI |
|
||||
| `src/model/shardAtlas/shardAtlas.db.js` | The one-transaction replace |
|
||||
| `src/model/shardAtlas/shardAtlas.model.js` | The refresh decision, staging, approve/reject |
|
||||
| `scripts/importSpawnAtlas.js` | Thin CLI over the model |
|
||||
|
||||
Parsing notes:
|
||||
|
||||
- `Regions.xml`, `Locations/*.xml` and `ChampionSpawns.xml` genuinely nest, and
|
||||
get a small hand-rolled **subset** tokenizer — elements, attributes,
|
||||
self-closing tags, comments, the XML declaration, CDATA, and the five
|
||||
predefined entities plus numeric refs. It is not a general-purpose XML parser
|
||||
and must not be reused as one.
|
||||
- The ~10.5 MB of `Spawns/*.xml` never touches that tokenizer. Those records are
|
||||
flat, so they get a streaming regex sweep instead; a DOM would allocate a node
|
||||
per element across ~40 fields on every record to keep 14 of them. **Do not put
|
||||
the Points files through a DOM parser.**
|
||||
- `<Objects2>` is `Type:MX=n:SB=…` segments joined by `:OBJ=`. Split on `:OBJ=`
|
||||
*first* — a naive `split(':')` shreds it. A single Trammel point carries six
|
||||
types.
|
||||
- **Respawn delays are stored in two different units, per record.** XmlSpawner
|
||||
writes `MinDelay`/`MaxDelay` in minutes, and switches to seconds only when a
|
||||
spawner's delay does not divide into whole minutes — flagging that with
|
||||
`DelayInSec` on the same record. A `5` therefore means five *minutes* on one
|
||||
spawner and five *seconds* on the next, and both are plausible respawn times,
|
||||
so a reader assuming either unit is silently wrong about the other. Stock
|
||||
ServUO 57.4 has ~170 second-flagged spawners out of 6,455. The parser
|
||||
normalises everything to **seconds**; the API and UI carry seconds throughout.
|
||||
|
||||
### The parser version
|
||||
|
||||
`spawnAtlasSource.js` exports `PARSER_VERSION`, stored in `shard_atlas_meta`
|
||||
alongside the source hashes and bumped whenever the parser derives **different
|
||||
data from identical files** — a fixed misreading, a new field, a changed unit.
|
||||
|
||||
A refresh re-derives when the tree changed **or** the parser did. Hashing the
|
||||
tree alone would be a trap: an install whose maps never change would keep serving
|
||||
whatever an older build derived, indefinitely, and a deploy that corrects the
|
||||
parse would never reach the data. A version mismatch counts as drift, so the
|
||||
correction lands on the next boot without an operator having to know it happened.
|
||||
|
||||
## The API
|
||||
|
||||
Everything is served from MariaDB. Nothing on this path touches the sidecar, so
|
||||
the pages stay complete while the shard is down — which is why the routes sit at
|
||||
`/api/v1/public/atlas` and **not** under `/public/shard`, where a prefix means
|
||||
"sidecar-dependent". Unlike `/shard/*`, they *are* `siteMode`-gated, like
|
||||
`/posts` and `/wiki`: a bestiary is site content and follows site content's rules.
|
||||
|
||||
Every route carries `requireFeature('atlas')` — **404** when an admin has
|
||||
disabled the feature (its pages must not reveal that it exists) and **403** when
|
||||
the caller sits below its configured audience. The default is `anonymous`, so the
|
||||
gates are inert until an admin changes something. Responses are field-projected
|
||||
like every other shard read; `atlas` declares no sensitive fields today, and the
|
||||
projection call is there so the first one that does is covered by construction
|
||||
rather than by a retrofit ([`v3.md` §3.6.1](../link/v3.md)).
|
||||
|
||||
| Route | Answers |
|
||||
|---|---|
|
||||
| `GET /atlas/creatures?q=&facet=&limit=&offset=` | The bestiary, most numerous first, paginated with an unpaginated `total` |
|
||||
| `GET /atlas/creatures/:slug?facet=&points=` | One creature: `places`, `spawners`, `alsoHere` |
|
||||
| `GET /atlas/regions?facet=&q=` | Named regions and their rectangles |
|
||||
| `GET /atlas/landmarks?facet=&q=` | Points of interest, labelled by `group` |
|
||||
| `GET /atlas/champions?facet=` | The configured altar roster |
|
||||
| `GET /atlas/meta` | Facets, counts and when the atlas was parsed |
|
||||
|
||||
Two shapes worth knowing:
|
||||
|
||||
- **`places` is the aggregate the atlas exists for.** "Lizardman → Shrines,
|
||||
Isamu-Jima, Yew", grouped in SQL rather than by summing 6,455 point rows in
|
||||
Node. `spawners` is the raw list underneath it, bounded, with
|
||||
`spawnersTruncated` saying when it was cut.
|
||||
- **`points` is a COUNT, `spawners` is the LIST.** The two are named apart
|
||||
deliberately: the same key meaning a number on the search route and an array on
|
||||
the detail route is the kind of thing a client only discovers in production.
|
||||
|
||||
`GET /atlas/meta` reports the **game world only**. The ServUO path, the per-file
|
||||
hashes and any pending refresh describe the operator's filesystem, and live on
|
||||
the admin route instead.
|
||||
|
||||
A facet is never validated against a list — nothing in the codebase names one.
|
||||
`?facet=` is length-bounded and matched exactly, so an unknown name returns an
|
||||
empty result rather than an error. The filter is an `EXISTS` over the points and
|
||||
deliberately not a JSON path or `JSON_SEARCH` built from caller input: that
|
||||
function treats `%` and `_` as wildcards, which would make `?facet=%` match
|
||||
everything.
|
||||
|
||||
## The admin panel
|
||||
|
||||
**Admin → Spawn Atlas** (`/admin/shard-atlas`, admin-only — it reads a path on
|
||||
the server's filesystem and replaces every atlas table, which is closer to a
|
||||
deploy action than to moderation).
|
||||
|
||||
| Route | Does |
|
||||
|---|---|
|
||||
| `GET /admin/shard/atlas` | Status: path, readable, drift, counts, facets, pending |
|
||||
| `POST /admin/shard/atlas/import` | Import now; `{ force: true }` ignores the hash gate |
|
||||
| `POST /admin/shard/atlas/approve` | Apply a staged refresh, facet loss and all |
|
||||
| `POST /admin/shard/atlas/reject` | Keep the current atlas; remember the decision |
|
||||
| `PUT /admin/shard/atlas/path` | Point the atlas at a different tree |
|
||||
|
||||
Three behaviours that are deliberate:
|
||||
|
||||
- **An unreadable tree is a 200, not a 500.** `refresh()` reports outcomes rather
|
||||
than throwing, because the boot path must never be stopped by a bad tree, and
|
||||
that contract is preserved at the API. The panel says *"The tree could not be
|
||||
read: …"*; a 500 would say only that something broke.
|
||||
- **Setting the path does not import.** Moving the mount and reloading the world
|
||||
are separate decisions, and an operator fixing a typo should not have a
|
||||
multi-thousand-row replace happen under them. The response carries fresh status
|
||||
so the panel can offer the import as the next step.
|
||||
- **Every action is written to the admin activity log** (`shard.atlas.import` /
|
||||
`.approve` / `.reject` / `.path`).
|
||||
@@ -1,259 +0,0 @@
|
||||
# Trusted Devices & MFA Improvements — Design & Implementation Plan
|
||||
|
||||
> Reference plan for the trusted-device + MFA hardening work. Approved 2026-07-21.
|
||||
> This document is the contract the implementation builds against; keep it in sync
|
||||
> with `BACKEND_DESIGN.md` (§3 schema, §4 API, §6 security) as code lands.
|
||||
|
||||
## 1. Goal & scope
|
||||
|
||||
Reduce 2FA friction without weakening the second-factor boundary, and close the
|
||||
2FA-lockout gap. Four deliverables:
|
||||
|
||||
1. **Trusted devices** — an opt-in "Trust this device" that lets a browser or the
|
||||
Android app **skip the TOTP step** (never the password) on future logins for a
|
||||
fixed window.
|
||||
2. **Recovery / backup codes** — single-use codes generated at 2FA enrollment so a
|
||||
user who loses their authenticator can self-recover instead of needing an admin
|
||||
reset.
|
||||
3. **Admin-managed revocation** — staff can view and revoke a user's trusted
|
||||
devices and reset their MFA, with full audit logging (**backend endpoints _and_
|
||||
admin front-end screens**).
|
||||
4. **Step-up (password) for sensitive operations** — reusing the existing
|
||||
`currentPassword`-verification pattern; disabling TOTP keeps its stronger
|
||||
current-TOTP-code requirement.
|
||||
|
||||
Touches `website/` (server + client), `docs/`, and `android-app/` (plan only in
|
||||
this pass). **No** `link/` or `servuo-plugins/` change — no wire-protocol impact.
|
||||
|
||||
### Approved decisions
|
||||
|
||||
| Decision | Value |
|
||||
|---|---|
|
||||
| Trust duration | **30 days** (matches mobile refresh-token lifetime) |
|
||||
| Roles eligible | **All roles** (no staff carve-out) |
|
||||
| Opt-in model | **Explicit "Trust this device" checkbox, default off** |
|
||||
| Recovery codes | **10 codes**, shown **once**, single-use |
|
||||
| Trusted-device cap | **10 per user, no silent pruning** (see §5) |
|
||||
| Trust-token hashing | **sha256** |
|
||||
| Recovery-code hashing | **bcrypt** (cost 10) |
|
||||
|
||||
## 2. Current state (starting point)
|
||||
|
||||
- **One session service** (`server/src/auth/session.service.js`) backs web (JWT
|
||||
`httpOnly` cookie, 1d) and mobile (15m access JWT + 30d opaque refresh token).
|
||||
`requireAuth` accepts either via `token.extractToken()`.
|
||||
- **TOTP** is opt-in per user (`users.totp_secret` / `totp_enabled`), demanded on
|
||||
**every** login. Web uses a staged 5-min `stage:'totp'` challenge; mobile uses a
|
||||
single-request `401 { totpRequired }`. **No recovery codes** exist today.
|
||||
- **Device tracking exists only on mobile** (`mobile_refresh_tokens` rows with
|
||||
`device_name` / `device_hash` / `user_agent` / `last_used_at`). Web JWTs are
|
||||
stateless with no per-session row.
|
||||
- **Revocation is mature:** `revoked_sessions` (jti denylist) + `tokens_valid_after`
|
||||
(per-user cutoff) for web; per-token rows + `revokeAllForUser` for mobile.
|
||||
- **No trusted-device or step-up concept exists anywhere.**
|
||||
|
||||
## 3. Hashing rationale
|
||||
|
||||
The repo already splits hashing by secret entropy, and this plan follows it:
|
||||
|
||||
- **sha256** — every high-entropy machine-generated opaque token
|
||||
(`mobile_refresh_tokens`, `mobile_auth_codes`, `user_invites`, `password_resets`,
|
||||
SSO PKCE). **Trusted-device tokens use sha256:** they are 256-bit random values
|
||||
(nothing to brute-force) looked up **by a `token_hash UNIQUE` index**, which
|
||||
requires a deterministic hash — bcrypt's per-row salt would break the lookup and
|
||||
truncates input at 72 bytes.
|
||||
- **bcrypt (`bcryptjs`, cost 10)** — the repo uses it only for **passwords**, the
|
||||
one human-chosen low-entropy secret. **Recovery codes use bcrypt:** they are a
|
||||
human-typed, lower-entropy fallback credential that grants a login (the closest
|
||||
analogue to a password), and there is no hash-lookup constraint — we fetch the
|
||||
identified user's ≤10 code rows and `bcrypt.compare` each, exactly like password
|
||||
verification.
|
||||
|
||||
## 4. Database (additive, idempotent — matches `schema.sql` style)
|
||||
|
||||
### `trusted_devices`
|
||||
Pattern-identical to `mobile_refresh_tokens`; stores only the token hash.
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| id | INT PK AUTO_INCREMENT | |
|
||||
| user_id | INT NOT NULL | FK → users, `ON DELETE CASCADE` |
|
||||
| token_hash | CHAR(64) NOT NULL UNIQUE | sha256 hex of the opaque trust token |
|
||||
| platform | ENUM('web','mobile') NOT NULL DEFAULT 'web' | |
|
||||
| device_name | VARCHAR(100) NULL | friendly label |
|
||||
| device_hash | VARCHAR(32) NULL | best-effort UA+IP, **display only** |
|
||||
| user_agent | VARCHAR(255) NULL | |
|
||||
| created_at | DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP | |
|
||||
| last_used_at | DATETIME NULL | stamped when trust is honored at login |
|
||||
| expires_at | DATETIME NOT NULL | created_at + 30d |
|
||||
| revoked_at | DATETIME NULL | |
|
||||
|
||||
Indices: `idx_td_user (user_id)`, `idx_td_expires (expires_at)`.
|
||||
|
||||
### `mobile_auth_sessions.trust_device` (SSO bridge)
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| trust_device | TINYINT(1) NOT NULL DEFAULT 0 | user ticked "trust this device" on the Custom Tab TOTP form |
|
||||
|
||||
A **boolean only**. It records the user's choice so `POST /auth/mobile/sso/exchange`
|
||||
knows to mint the app's own trust token over that authenticated app→server call; the
|
||||
token itself is never written here (only its sha256 reaches `trusted_devices`). Set
|
||||
only while the session is still `pending` and unexpired, for the same reason
|
||||
`completeSession` is guarded — a replayed TOTP post must not re-arm a consumed
|
||||
session.
|
||||
|
||||
### `recovery_codes`
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| id | INT PK AUTO_INCREMENT | |
|
||||
| user_id | INT NOT NULL | FK → users, `ON DELETE CASCADE` |
|
||||
| code_hash | VARCHAR(72) NOT NULL | **bcrypt** hash of one code |
|
||||
| used_at | DATETIME NULL | single-use marker |
|
||||
| created_at | DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP | |
|
||||
|
||||
Index: `idx_rc_user (user_id)`.
|
||||
|
||||
No new `users` column: password change/reset and TOTP-disable **bulk-revoke**
|
||||
`trusted_devices` rows and **delete** `recovery_codes` (consistent with
|
||||
`revokeAllForUser`), so no "trust epoch" column is needed.
|
||||
|
||||
## 5. Trusted-device cap — no silent pruning
|
||||
|
||||
Cap = **10**. A shared `assertUnderTrustCap(userId)` guards both entry points (the
|
||||
login/TOTP trust path and the authenticated "trust this device" path). On the 11th
|
||||
attempt the backend **refuses to create the row** and returns
|
||||
`409 { error: 'trusted_device_limit', devices: [...] }`. Login itself still
|
||||
succeeds — only the trust marker is withheld. The web client then renders a modal
|
||||
**in the same visual pattern as the TOTP entry flow** that:
|
||||
|
||||
1. shows the existing trusted devices,
|
||||
2. requires revoking ≥1 before continuing,
|
||||
3. completes via `POST /auth/me/trusted-devices` (trust current device), and
|
||||
4. offers **Cancel**, which returns without creating any trust entry.
|
||||
|
||||
## 6. API additions
|
||||
|
||||
### Auth (login paths)
|
||||
- `POST /auth/login` — after password verify, if a valid unrevoked `rg_trust`
|
||||
cookie matches a live `trusted_devices` row for this user → **skip TOTP**, issue
|
||||
the session, log `auth.login.trusted_device`, stamp `last_used_at`. Otherwise
|
||||
unchanged (`{ totpRequired, challenge }`).
|
||||
- `POST /auth/login/totp` — gains optional `trustDevice` + `deviceName`, and
|
||||
accepts a **recovery code** as an alternative to the TOTP code (single-use). On
|
||||
success with `trustDevice`, mint the opaque trust token, set the `rg_trust`
|
||||
cookie, insert the row (subject to the cap → `409` signal).
|
||||
- `POST /auth/mobile/login` — gains `trustDevice` / `recoveryCode`; returns a
|
||||
`trustToken` the app stores in EncryptedSharedPreferences and replays on a later
|
||||
login to skip TOTP. Same cap behavior.
|
||||
|
||||
#### SSO login paths
|
||||
|
||||
SSO is **not** exempt: an account with TOTP on is asked for a code after a
|
||||
Google/Discord sign-in exactly as it is after a password one, and a trusted device
|
||||
skips that code exactly the same way. (Originally SSO consulted trust nowhere, so a
|
||||
user who signed in with an external identity was asked for a code on *every* sign-in
|
||||
no matter how many times they had ticked "trust this device".)
|
||||
|
||||
- `GET /auth/sso/:provider/callback` — once the account is resolved and before a
|
||||
TOTP challenge is staged, resolve the presented trust (cookie, or `X-Trust-Token`)
|
||||
and, if it belongs to **this** user, skip the code, stamp `last_used_at`, and log
|
||||
`auth.login.trusted_device`. The first factor is the IdP authentication that just
|
||||
succeeded, so this is the same posture as the password path. A store error falls
|
||||
through to the challenge — fail **closed** to asking for the code.
|
||||
- `POST /auth/sso/totp` — gains optional `trustDevice` + `deviceName`, mints the
|
||||
trust and sets the `rg_trust` cookie on success. At the cap the sign-in still
|
||||
completes and the response carries `{ trustLimitReached, devices }`, matching
|
||||
`POST /auth/login/totp`. Recovery codes remain password-login only: this step
|
||||
verifies an authenticator code against the staged challenge.
|
||||
|
||||
**How this reaches the Android app.** The app's SSO runs in a Custom Tab, which
|
||||
shares the system browser's cookie jar, so both halves land in the same place: the
|
||||
`rg_trust` cookie set on the Custom Tab TOTP form is presented back on the *next*
|
||||
app SSO sign-in and skips the code — no app change, and no trust token smuggled
|
||||
through a start URL where it would leak into query strings and logs.
|
||||
|
||||
To cover the app's **native** password login on the same device as well, ticking
|
||||
the box also sets `mobile_auth_sessions.trust_device` (a boolean — never the
|
||||
token), and `POST /auth/mobile/sso/exchange` then mints a `platform: 'mobile'`
|
||||
trust and returns `{ trustToken }` in its JSON body. Minting at exchange time is
|
||||
deliberate: it is an authenticated app→server call, so the raw token never travels
|
||||
in the deep link and never rests in the bridge row. One tick therefore produces two
|
||||
independently-revocable rows (the browser and the app) — which is honest, since they
|
||||
are two distinct credentials on one device. If the user is at the cap, the exchange
|
||||
simply returns no token; it never turns a successful sign-in into an error.
|
||||
|
||||
### Self-service (`/auth/me/*`, `requireAuth`, any role)
|
||||
- `GET /auth/me/trusted-devices` — list active trusted devices (never tokens).
|
||||
- `POST /auth/me/trusted-devices` — trust the current browser/device (cap-checked).
|
||||
- `DELETE /auth/me/trusted-devices/:id` — revoke one (ownership-scoped).
|
||||
- `DELETE /auth/me/trusted-devices` — revoke all ("untrust everywhere").
|
||||
- `POST /auth/me/account/recovery-codes/generate` — **password step-up required**;
|
||||
returns the codes **once**.
|
||||
- `GET /auth/me/account/recovery-codes/status` — remaining count only.
|
||||
|
||||
### Admin (`requireRole('admin')`)
|
||||
- `GET /admin/users/:id/trusted-devices` — list a user's trusted devices.
|
||||
- `DELETE /admin/users/:id/trusted-devices/:deviceId` — revoke one.
|
||||
- `DELETE /admin/users/:id/trusted-devices` — revoke all.
|
||||
- MFA reset control (revoke trust + disable TOTP + clear recovery codes).
|
||||
|
||||
## 7. Cookie / refresh / JWT interaction
|
||||
|
||||
- New **`rg_trust`** cookie: `httpOnly`, `sameSite=Lax`, `secure` per-request
|
||||
(reuse `cookieSecure`), `path=/`, `maxAge` 30d, opaque 256-bit base64url,
|
||||
sha256-hashed server-side. **Separate from the session cookie and deliberately
|
||||
survives logout** (so the next login skips 2FA); only untrust / password-change /
|
||||
TOTP-disable revoke it.
|
||||
- **JWTs stay stateless and unchanged** — trust is a server-side cookie+row, never
|
||||
a JWT claim, so it remains revocable.
|
||||
- **Refresh flow untouched** — trust is consulted only at the login/password step,
|
||||
never at token refresh; the two stores stay independent.
|
||||
|
||||
## 8. Security & invalidation
|
||||
|
||||
- Trust **only ever gates the second factor**; password is always required.
|
||||
- Recovery-code entry reuses the login brute-force stack (backoff + bot scoring +
|
||||
rate limits); recovery codes are single-use.
|
||||
- **Password change/reset and TOTP-disable clear trust and recovery codes.**
|
||||
- **Audit logging** via existing `activity.log` / `activity_log`:
|
||||
`auth.login.trusted_device`, `account.trusted_device.add` / `.revoke` /
|
||||
`.revoke_all`, `account.recovery_codes.generate`, `account.recovery_code.consume`,
|
||||
and admin `admin.trusted_device.revoke` / `.revoke_all`, `admin.user.totp.reset`
|
||||
— each with actor, target user, and device id in `detail`.
|
||||
|
||||
## 9. Backwards compatibility
|
||||
|
||||
Fully additive. With no `rg_trust` cookie the behavior is exactly today's (TOTP
|
||||
every login). Recovery codes exist only for users who generate them. No existing
|
||||
session or login flow changes shape. New tables via `CREATE TABLE IF NOT EXISTS`
|
||||
and columns via `ALTER TABLE … ADD COLUMN IF NOT EXISTS`.
|
||||
|
||||
## 10. Implementation roadmap
|
||||
|
||||
1. **Schema + models** — `trusted_devices` (sha256, cap-checked) + `recovery_codes`
|
||||
(bcrypt); `.db.js` / `.model.js` pairs mirroring `mobileSessions`.
|
||||
2. **Session service** — trust-token mint/sha256/verify + recovery-code
|
||||
generate/bcrypt-verify/consume helpers (pure, DB-free); shared
|
||||
`assertUnderTrustCap()`.
|
||||
3. **Web login** — trust-cookie skip in `/auth/login`; `trustDevice` / recovery
|
||||
handling + cap `409` in `/auth/login/totp`; set/clear `rg_trust`.
|
||||
4. **Mobile login** — `trustDevice` / `trustToken` / `recoveryCode`, same cap.
|
||||
5. **Self-service + admin backend** — `/auth/me/trusted-devices*` + recovery-code
|
||||
endpoints; `/admin/users/:id/trusted-devices*` + MFA reset (all admin-gated).
|
||||
6. **Invalidation wiring** — password change/reset & TOTP-disable revoke trust +
|
||||
delete recovery codes.
|
||||
7. **Web client UI** — "Trust this device" checkbox; cap-reached TOTP-styled modal
|
||||
(revoke-to-continue / cancel); user Trusted Devices + Recovery Codes screens.
|
||||
8. **Admin front-end UI** — admin Trusted Devices management & revocation screens
|
||||
(per-user list, revoke one / revoke all, MFA reset), wired to step 5.
|
||||
9. **Android** — record the app-side trust/recovery flow in
|
||||
`docs/android/PLAN.md`; app implementation sequenced after the backend lands.
|
||||
10. **OpenAPI + docs** — `#swagger.*` on every new/modified route + regenerate
|
||||
`server/swagger/swagger-output.json`; update `BACKEND_DESIGN.md` §3/§4/§6.
|
||||
11. **Automated tests** — trusted-device login skip (valid / missing / expired /
|
||||
revoked), token mint+hash, recovery-code single-use consume + wrong-code
|
||||
backoff, cap `409` behavior, revocation (self + admin), invalidation on
|
||||
password-change / TOTP-disable, and permission checks (admin routes reject
|
||||
non-admins; self routes ownership-scoped); web client pure-logic tests; Android
|
||||
JVM DTO/repository tests.
|
||||
@@ -54,7 +54,7 @@ auth model (admin/editor — no new roles, no public contributions).
|
||||
| Schema | flat `wiki_pages(slug,title,body,updated_by,timestamps)` | [server/db/schema.sql:31](server/db/schema.sql) |
|
||||
| Model | thin CRUD by slug | [server/src/model/wiki/wiki.db.js](server/src/model/wiki/wiki.db.js), [wiki.model.js](server/src/model/wiki/wiki.model.js) |
|
||||
| Public API | `GET /public/wiki`, `GET /public/wiki/:slug` | [public.controller.js:53](server/src/router/v1/public/public.controller.js) |
|
||||
| Admin API | `GET/POST/PUT/DELETE /admin/wiki[...]` | [admin.controller.js:163](server/src/router/v1/admin/admin.controller.js), [wiki.router.js](server/src/router/v1/admin/wiki.router.js) |
|
||||
| Admin API | `GET/POST/PUT/DELETE /admin/wiki[...]` | [admin.controller.js:163](server/src/router/v1/admin/admin.controller.js), [admin.routes.js:68](server/src/router/v1/admin/admin.routes.js) |
|
||||
| Public UI | card grid (hardcoded blurbs + Roman numerals), article w/ auto-TOC | [Wiki.jsx](client/src/routes/wiki/Wiki.jsx), [WikiArticle.jsx](client/src/routes/wiki/WikiArticle.jsx) |
|
||||
| Admin UI | raw-HTML `<textarea>` modal | [WikiAdmin.jsx](client/src/routes/admin/views/WikiAdmin.jsx), [WikiEditor.jsx](client/src/routes/admin/views/WikiEditor.jsx) |
|
||||
| API client | `api.wiki`, `api.admin.*Wiki` | [client/src/api/client.js:52](client/src/api/client.js) |
|
||||
@@ -196,7 +196,7 @@ centralized error handler unchanged. **Every write logs to `activity_log`**
|
||||
|
||||
### 4.4 Image uploads
|
||||
|
||||
Generalize the existing screenshot upload (multer config now in [admin/imageUpload.js](server/src/router/v1/admin/imageUpload.js))
|
||||
Generalize the existing screenshot upload (multer config in [admin.routes.js:17](server/src/router/v1/admin/admin.routes.js))
|
||||
into a shared `POST /admin/uploads` returning `{ url: "/uploads/<file>" }`, reused by both
|
||||
the post editor and the wiki editor. Same size/mime limits. No new storage —
|
||||
served from the existing `uploads/` volume.
|
||||
|
||||
@@ -1,875 +0,0 @@
|
||||
{
|
||||
"$comment": "Generated route inventory - the authoritative freeze of the URL surface. Regenerate with `npm run routes:manifest` in website/server; a domain-split PR must produce a zero-line diff here.",
|
||||
"public": [
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/.well-known/assetlinks.json"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/csp-report"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/docs.json"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/health"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/account"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/account/identities"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/account/identities/:provider"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/account/totp/disable"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/account/totp/enable"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/account/totp/setup"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/activity"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/auth/providers"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/auth/providers"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/auth/providers/:id"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/auth/providers/:id"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/bot-activity"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/bot-activity/unban"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/dashboard"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/discord-bot/config"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/discord-bot/config"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/email/config"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/email/config"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/email/connect/callback"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/email/connect/start"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/email/disconnect"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/email/test"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/invites"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/invites"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/invites/:id"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/appeals"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/appeals/:id"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/moderation/appeals/:id/claim"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/moderation/appeals/:id/resolve"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/filter-hits"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/members"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/recent"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/search"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/spam-hits"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/stats/summary"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/user/:discordId"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/user/:discordId/actions"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/user/:discordId/appeals"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/moderation/user/:discordId/notes"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/moderation/user/:discordId/notes"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/pages"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/pages"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/pages/:id"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/pages/:id"
|
||||
},
|
||||
{
|
||||
"method": "PATCH",
|
||||
"path": "/api/v1/admin/pages/:id"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/pages/:id/preview"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/pages/:id/unprotect"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/posts"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/posts"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/posts/:id"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/posts/:id"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/posts/:id"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/posts/:id/announce"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/posts/:id/announce/retry"
|
||||
},
|
||||
{
|
||||
"method": "PATCH",
|
||||
"path": "/api/v1/admin/posts/:id/publish"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/posts/upload"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/settings"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/settings"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/account"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/shard/accounts"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/shard/atlas"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/atlas/approve"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/atlas/import"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/shard/atlas/path"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/atlas/reject"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/shard/audit"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/ban"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/broadcast"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/shard/char/:serial"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/shard/houses"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/kick"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/link"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/shard/pages"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/pages/:id/close"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/pages/:id/respond"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/shard/roster/:account"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/shard/sales"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/shard/unban"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/shard/vendors/:account"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/shard/visibility"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/shard/visibility"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/site-mode"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/uo-link/config"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/uo-link/config"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/uo-link/stream"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/uo-link/towncrier"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/uo-link/towncrier/:id"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/uploads"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/users"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/users"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/users/:id"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/users/:id"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/users/:id"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/users/:id/mfa/reset"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/users/:id/shard/accounts"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/users/:id/shard/houses"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/users/:id/shard/link/:account"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/users/:id/shard/online"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/users/:id/shard/sales"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/users/:id/shard/standing"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/users/:id/trusted-devices"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/users/:id/trusted-devices"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/users/:id/trusted-devices/:deviceId"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/wiki"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/wiki"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/wiki/:slug"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/wiki/:slug"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/wiki/:slug"
|
||||
},
|
||||
{
|
||||
"method": "PATCH",
|
||||
"path": "/api/v1/admin/wiki/:slug/publish"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/wiki/:slug/revisions"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/wiki/:slug/revisions/:id"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/wiki/:slug/revisions/:id/restore"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/wiki/categories"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/admin/wiki/categories"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/admin/wiki/categories/:id"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/admin/wiki/categories/:id"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/admin/wiki/tags"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/invite/:token"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/invite/:token/accept"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/login"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/login/totp"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/logout"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/me"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/me/account"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/me/account/identities"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/auth/me/account/identities/:provider"
|
||||
},
|
||||
{
|
||||
"method": "PATCH",
|
||||
"path": "/api/v1/auth/me/account/password"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/me/account/recovery-codes/generate"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/me/account/recovery-codes/status"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/me/account/totp/disable"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/me/account/totp/enable"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/me/account/totp/setup"
|
||||
},
|
||||
{
|
||||
"method": "PATCH",
|
||||
"path": "/api/v1/auth/me/account/username"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/me/devices"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/me/devices"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/auth/me/devices/:id"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/me/notifications/streams"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/me/notifications/subscriptions"
|
||||
},
|
||||
{
|
||||
"method": "PUT",
|
||||
"path": "/api/v1/auth/me/notifications/subscriptions"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/me/sessions"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/auth/me/sessions/:id"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/auth/me/trusted-devices"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/me/trusted-devices"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/me/trusted-devices"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/auth/me/trusted-devices/:id"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/mobile/login"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/mobile/logout"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/mobile/refresh"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/mobile/sso/exchange"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/mobile/sso/start"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/password/forgot"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/password/reset/:token"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/password/reset/:token"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/providers"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/register"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/sso/:provider/callback"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/sso/:provider/link"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/auth/sso/:provider/start"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/auth/sso/totp"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/account"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/account/identities"
|
||||
},
|
||||
{
|
||||
"method": "DELETE",
|
||||
"path": "/api/v1/player/account/identities/:provider"
|
||||
},
|
||||
{
|
||||
"method": "PATCH",
|
||||
"path": "/api/v1/player/account/password"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/player/account/totp/disable"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/player/account/totp/enable"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/player/account/totp/setup"
|
||||
},
|
||||
{
|
||||
"method": "PATCH",
|
||||
"path": "/api/v1/player/account/username"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/appeals"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/player/appeals"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/player/appeals/:id/withdraw"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/appeals/eligible"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/player/shard/account"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/shard/accounts"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/shard/char/:serial"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/shard/houses"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/player/shard/link"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/shard/roster/:account"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/shard/sales"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/player/shard/vendors/:account"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/atlas/champions"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/atlas/creatures"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/atlas/creatures/:slug"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/atlas/landmarks"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/atlas/meta"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/atlas/regions"
|
||||
},
|
||||
{
|
||||
"method": "POST",
|
||||
"path": "/api/v1/public/contact"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/pages/:id/preview/:token"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/pages/:slug"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/posts/:category"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/posts/:category/:idOrSlug"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/settings"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/champs"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/economy"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/features"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/feed"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/governors"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/governors/:city/history"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/guilds"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/houses"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/idoc"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/online"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/presence"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/ruleset"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/status"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/shard/stream"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/status"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/version"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/wiki"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/wiki/:slug"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/wiki/categories"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/api/v1/public/wiki/tags"
|
||||
}
|
||||
],
|
||||
"internal": [
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/health"
|
||||
},
|
||||
{
|
||||
"method": "GET",
|
||||
"path": "/internal/bot-config"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,154 +0,0 @@
|
||||
# Runic Gateway Website — Test Plan (Discord bot)
|
||||
|
||||
> Companion to [website-README.md](website-README.md) (overview) and
|
||||
> [BACKEND_DESIGN.md](BACKEND_DESIGN.md) (server API/schema/security contract).
|
||||
> Establishes the test plan for the **`bot/` workspace** (the Discord bot), the
|
||||
> last of the three `website` npm workspaces without a suite. The server and
|
||||
> client suites already exist (website PR #86); this doc closes the bot gap and
|
||||
> records the shared conventions so all three stay consistent.
|
||||
|
||||
## 1. Goal & philosophy
|
||||
|
||||
Cover **meaningful bot behavior** — the moderation/filter decisions a future
|
||||
change could silently break — not a coverage number. The bot's value is in *what
|
||||
it decides to delete, warn, mute, or let through*; a good test reads "given this
|
||||
message + config, what action does the bot take?", never "did this function call
|
||||
that function?".
|
||||
|
||||
The whole `website` repo tests on **Node's built-in runner (`node --test`)** with
|
||||
**zero external test dependencies** — no jest, no vitest, no jsdom. Unit tests
|
||||
run without a database or a live Discord gateway: the DB pool is pointed at a dead
|
||||
port and every collaborator (`db.query`, a model, a Discord `message`/`client`) is
|
||||
replaced with an in-memory fake. This mirrors the server suite exactly (the bot is
|
||||
CommonJS, like the server — `require`/`module.exports` — so the same patterns
|
||||
apply verbatim).
|
||||
|
||||
## 2. Current state
|
||||
|
||||
| Workspace | Runner | Status |
|
||||
|---|---|---|
|
||||
| `server/` | `node --test` | **Exists** — models, controllers, auth/session, shard ingest. |
|
||||
| `client/` | `node --test` (pure-logic ESM) | **Exists** (PR #86) — `lib/`, `api/`, `data/`. |
|
||||
| `bot/` | — | **This plan** — no runner or tests yet. |
|
||||
|
||||
The 46% SonarQube aggregate is dragged down by the bot (and the client's
|
||||
un-unit-testable React components) reading as 0% covered. Standing up the bot
|
||||
suite lifts the real number and, more importantly, locks the filter/moderation
|
||||
rules.
|
||||
|
||||
## 3. Harness
|
||||
|
||||
Add a test script to `bot/package.json` (mirrors server/client):
|
||||
|
||||
```json
|
||||
"scripts": { "test": "node --test" }
|
||||
```
|
||||
|
||||
Conventions, identical to `server/test/`:
|
||||
|
||||
- **No DB.** Set `process.env.DB_HOST = '127.0.0.1'` / `DB_PORT = '59999'` at the
|
||||
top of any test whose module transitively `require`s `../db`, so the pool is
|
||||
built against a dead port and a stray query fails fast instead of hanging.
|
||||
Models are exercised by monkeypatching `db.query` (or the model method the unit
|
||||
under test calls) with an in-memory fake; restore it in `afterEach`.
|
||||
- **No Discord.** The `discord.js` `Client`, `Message`, `GuildMember`, and
|
||||
`Interaction` objects are hand-rolled fakes carrying only the fields the unit
|
||||
reads (e.g. `message.mentions.users.size`, `message.member.roles.cache`,
|
||||
`message.client.fetchInvite`). Never construct a real client or open a gateway
|
||||
connection.
|
||||
- **Tests live in `bot/test/*.test.js`.** One file per module under test.
|
||||
|
||||
## 4. Units to cover
|
||||
|
||||
Ordered high-value first. Each row names the module, the behavior worth locking,
|
||||
and the seam a test drives it through.
|
||||
|
||||
### 4.1 Pure logic (no mocks beyond inputs)
|
||||
|
||||
| Module | Behaviors to lock | Notes |
|
||||
|---|---|---|
|
||||
| `filter/normalize.js` | leetspeak folding (`b4d`→`bad`, `@ss`→`ass`), 3+-repeat collapse (`sooooo`→`so`), case-fold; `matches()` is word-boundary anchored (so `bad` does not fire inside `badminton`); `findMatch()` returns the first `{word, severity}` row or `null`; the *documented gaps* stay gaps (spaced-out `b a d` is **not** caught). | The core obfuscation-resistance contract. Pin the gaps too, so a future tightening is a deliberate, test-visible change. |
|
||||
| `utils/duration.js` | `"30s"/"10m"/"2h"/"1d"` → correct ms; whitespace tolerated; garbage/empty/unknown-unit → `null`; `MAX_TIMEOUT_MS` is 28 days (the Discord cap callers clamp to). | Feeds `/mute` and temp-role expiry — a wrong parse mutes for the wrong span. |
|
||||
| `filter/spamFilter.js` | `isRateLimited` trips on the 6th message inside the 5 s window and is per-`guild:user`; it has a **side effect** (records the timestamp) so it must be called once; `isMassMention` counts users **+** roles against the threshold; `isMassEmoji` counts custom `<:name:id>` and unicode pictographs. | Drive time with a stubbed `Date.now` (or accept the real clock and use tight windows). The module-load `setInterval(...).unref()` sweep must not keep the runner alive — `.unref()` already handles this; assert no leak by letting the suite exit. |
|
||||
|
||||
### 4.2 Logic with a single mocked collaborator
|
||||
|
||||
| Module | Behaviors to lock | Seam / mock |
|
||||
|---|---|---|
|
||||
| `filter/inviteFilter.js` | returns the **code** (not a bool) of the first *foreign* invite; an invite resolving to the current guild is allowed; an invite that **fails to resolve** (expired/invalid) is treated as foreign (fail-closed); no invite in the message → `null`. | Fake `message.client.fetchInvite(code)` — resolve with `{ guild: { id } }` for local/foreign, or throw for unresolvable. Set `message.guildId` + `message.content`. |
|
||||
| `site/siteApiClient.js` | never throws on a network/timeout/HTTP failure — always returns the `{ ok, ... }` shape so a command never breaks when the site is down: `{ ok: true, data }` on success, `{ ok: false, error }` on failure, and the distinct `{ ok: false, maintenance: true, message }` for the site's 503 maintenance response. (Public read client — **no** shared secret; the keyed channel is `botInternalClient.js`.) | Stub `global.fetch` (resolve various statuses/bodies, reject/timeout via `AbortController`). Same non-throwing contract the server's `uoLinkClient` follows. |
|
||||
| `internal/requireInternalKey.js` | `401` on a missing/wrong key; `next()` on the configured key; **fail-closed** when `BOT_INTERNAL_KEY` is empty (never matches); timing-safe compare (equal-length + `crypto.timingSafeEqual`, no early-exit length leak). | Call the middleware with a fake `req` (`req.get('X-Internal-Key')`) + `res`/`next` spies; mirrors `server/test/requireInternalKey.test.js`. |
|
||||
|
||||
### 4.3 The filter pipeline (orchestration)
|
||||
|
||||
`discord/messageFilter.js` is the highest-value integration seam — it composes
|
||||
all of §4.1–4.2. Lock the **decision order and the actions**, with every
|
||||
collaborator faked:
|
||||
|
||||
- **Bypass wins first:** an allow-listed channel, or a member holding an
|
||||
allow-listed role, short-circuits the whole pipeline (`isBypassed`) — no
|
||||
delete, no hit recorded.
|
||||
- **Order:** invite link → banned word → spam (rate-limit → mass-mention →
|
||||
mass-emoji). `detectSpam` must evaluate `isRateLimited` **first and exactly
|
||||
once** (it has the timestamp side effect).
|
||||
- **Actions:** invite/spam always delete + warn (no severity tiers); a
|
||||
word-filter hit applies the action for its severity; filter-triggered mutes use
|
||||
the fixed `FILTER_MUTE_SECONDS` (600 s).
|
||||
- **Best-effort recording never breaks moderation:** `recordFilterHit` /
|
||||
`recordSpamHit` throwing must be swallowed — the delete/warn still happens.
|
||||
|
||||
Mocks: `filterCache` (in-memory `allowChannels`/`allowRoles` Sets + words), a fake
|
||||
`message` (content, author, member roles, `delete()`, `guildId`), and spies on
|
||||
`warnings`/`filterHits`/`spamHits`/`modLog` to assert what was called.
|
||||
|
||||
### 4.4 Models (`bot/model/*.js`)
|
||||
|
||||
Thin `db.query` wrappers (e.g. `warnings.js`, `filterWords.js`, `filterHits.js`,
|
||||
`spamHits.js`, `memberEvents.js`, `guildConfig.js`, `scheduledMessages.js`,
|
||||
`tempRoles.js`). Cover only those with **shaping/branching logic** (a query that
|
||||
maps rows, applies an "active/not-expired" predicate, or upserts) by asserting the
|
||||
**SQL params** passed to a fake `db.query` and the shape returned. Skip pure
|
||||
one-line inserts with no logic — testing those only proves the mock.
|
||||
|
||||
## 5. Explicitly out of scope
|
||||
|
||||
- **`discord.js` internals / the live gateway** — not ours to test; never open a
|
||||
real connection.
|
||||
- **Thin command wrappers** (`discord/commands/*.command.js`) that only validate
|
||||
args and call a Discord API + a model — cover their *logic* (arg parsing,
|
||||
duration clamping) if any, but not the Discord round-trip.
|
||||
- **`node-cron` scheduling itself** — test the job's callback logic, not that cron
|
||||
fires.
|
||||
|
||||
## 6. CI & coverage wiring
|
||||
|
||||
Mirror what PR #86 did for the client:
|
||||
|
||||
- **`.gitea/workflows/pr-checks.yml`** — add a `bot-tests` job (or a
|
||||
`Run bot tests` step) that runs `npm test --prefix bot`, gating PRs into `main`.
|
||||
- **`.gitea/workflows/sonarqube.yml`** — generate a bot LCOV from the repo root so
|
||||
`SF:` paths resolve to `bot/src/...`:
|
||||
|
||||
```
|
||||
node --test --experimental-test-coverage \
|
||||
--test-reporter=lcov --test-reporter-destination=bot/coverage/lcov.info \
|
||||
bot/test/*.test.js
|
||||
```
|
||||
|
||||
- **`sonar-project.properties`** — add `bot/test` to `sonar.tests`, the glob to
|
||||
`sonar.test.inclusions`, `bot/coverage/lcov.info` to
|
||||
`sonar.javascript.lcov.reportPaths`, and `bot/coverage/**` to `sonar.exclusions`.
|
||||
(`bot/src` is already in `sonar.sources`.)
|
||||
|
||||
Bot units need no `npm ci` to run when they import only relative files + Node
|
||||
built-ins; a test that stubs `global.fetch` or fakes `db` needs no `discord.js`
|
||||
either. Keep the pool pointed at a dead port so the run is hermetic.
|
||||
|
||||
## 7. Suggested phasing
|
||||
|
||||
1. **Harness + pure logic** — `test` script, then `normalize`, `duration`,
|
||||
`spamFilter`. Fast, zero mocks, immediate value.
|
||||
2. **Single-collaborator units** — `inviteFilter`, `siteApiClient`,
|
||||
`requireInternalKey`.
|
||||
3. **Pipeline** — `messageFilter` decision order + actions (the payoff test).
|
||||
4. **Models with logic**, then the **CI/Sonar wiring** so it all counts.
|
||||
@@ -18,7 +18,6 @@ The design reference is [BACKEND_DESIGN.md](BACKEND_DESIGN.md) (API contract, sc
|
||||
|
||||
## Contents
|
||||
|
||||
- [Architecture](#architecture)
|
||||
- [Tech stack](#tech-stack)
|
||||
- [Project structure](#project-structure)
|
||||
- [Prerequisites](#prerequisites)
|
||||
@@ -38,103 +37,6 @@ The design reference is [BACKEND_DESIGN.md](BACKEND_DESIGN.md) (API contract, sc
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
How the pieces fit together — the React SPA and native app talk to one Express backend
|
||||
(`router → controller → model → db`), which persists to MariaDB and bridges to the live
|
||||
game world only through the **uo-link** sidecar. The shard itself is never internet-facing.
|
||||
See [ARCHITECTURE.md](ARCHITECTURE.md) for the fuller write-up.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
%% ---------- Clients ----------
|
||||
subgraph clients["Clients"]
|
||||
browser["Browser<br/>React + Vite SPA<br/>(public · wiki · admin)"]
|
||||
mobile["Native mobile app<br/>(bearer tokens)"]
|
||||
end
|
||||
|
||||
idp["SSO providers<br/>Google · Discord · custom OIDC"]
|
||||
discord["Discord"]
|
||||
|
||||
%% ---------- Website (one repo) ----------
|
||||
subgraph website["website/ — Node app (one repo)"]
|
||||
direction TB
|
||||
|
||||
subgraph backend["server/ — Express backend"]
|
||||
direction TB
|
||||
mw["Middleware<br/>helmet · siteMode · noindex<br/>rateLimit · loginProtection · botScore · validate"]
|
||||
router["Router /api/v1<br/>auth (web · mobile · sso) · public · admin"]
|
||||
ctrl["Controllers"]
|
||||
auth["Session layer (auth/)<br/>sessionService · JWT/cookie · bearer · SSO+PKCE"]
|
||||
model["Models (.model + .db)<br/>raw parameterized SQL — no ORM"]
|
||||
sse["SSE fan-out<br/>public stream (allowlist) · admin stream (sensitive)"]
|
||||
|
||||
subgraph shardutil["Shard integration (utils/)"]
|
||||
ingest["shardIngest.js<br/>WS ingest dispatcher"]
|
||||
restcli["uoLinkClient.js<br/>REST client (never throws)"]
|
||||
end
|
||||
|
||||
secret["secretBox.js<br/>AES-256-GCM secrets at rest"]
|
||||
end
|
||||
|
||||
bot["bot/<br/>Discord bot"]
|
||||
end
|
||||
|
||||
db[("MariaDB<br/>users · posts · wiki · settings · activity<br/>mobileSessions · authProviders · userIdentities<br/>uoLinkConfig · shard_online/economy/houses/events")]
|
||||
|
||||
%% ---------- Shard side ----------
|
||||
subgraph shardside["Game shard (never internet-facing)"]
|
||||
direction TB
|
||||
sidecar["uo-link sidecar<br/>(Rust) — the only bridge exposed"]
|
||||
servuo["ServUO shard<br/>(C# plugin)"]
|
||||
end
|
||||
|
||||
%% ---------- Edges ----------
|
||||
browser <-->|"same-origin JSON + SSE (cookie)"| mw
|
||||
mobile -->|"REST (bearer access/refresh)"| mw
|
||||
browser -.->|"OAuth redirect + PKCE"| idp
|
||||
auth -.->|"token exchange"| idp
|
||||
|
||||
mw --> router --> ctrl
|
||||
ctrl --> auth
|
||||
ctrl --> model
|
||||
ctrl --> restcli
|
||||
ctrl --> sse
|
||||
auth --> model
|
||||
model <--> db
|
||||
auth -. reads/writes secrets .-> secret
|
||||
restcli -. reads config/token .-> secret
|
||||
ingest --> model
|
||||
ingest --> sse
|
||||
sse -->|"live events"| browser
|
||||
bot -->|"messages"| discord
|
||||
bot <--> db
|
||||
|
||||
restcli -->|"REST: /char /roster /economy /history · /link/confirm · /towncrier"| sidecar
|
||||
sidecar -->|"WebSocket live event feed (bearer + X-UOLink-Version)"| ingest
|
||||
servuo -->|"loopback TCP 127.0.0.1:7788<br/>newline-delimited JSON (shard dials out)"| sidecar
|
||||
|
||||
%% ---------- Styling ----------
|
||||
classDef ext fill:#2d2233,stroke:#7a5c94,color:#e8dff0;
|
||||
classDef store fill:#1f2d2a,stroke:#4c8c7d,color:#dff0ea;
|
||||
classDef bridge fill:#2d2620,stroke:#94764c,color:#f0e6d8;
|
||||
class idp,discord ext;
|
||||
class db store;
|
||||
class sidecar,servuo bridge;
|
||||
```
|
||||
|
||||
- **One backend, layered.** Every request flows `middleware → router → controller → model → db`.
|
||||
Web browsers authenticate with an httpOnly JWT cookie; the native app uses short-lived bearer
|
||||
access tokens plus rotated refresh tokens; SSO (Google/Discord/OIDC) is link-only and PKCE-guarded.
|
||||
All three surfaces produce the *same* session via the session layer.
|
||||
- **The shard is never reachable.** The ServUO shard *dials out* over loopback TCP to the uo-link
|
||||
sidecar; only the sidecar is exposed, and only the backend talks to it. The REST client
|
||||
(`uoLinkClient.js`) never throws, so the site degrades gracefully when the shard is down.
|
||||
- **Sensitive events stay private.** Ingested game events fan out to browsers over two SSE channels —
|
||||
a public allowlist stream and an admin-only stream that adds staff audit / cheat / login events.
|
||||
|
||||
---
|
||||
|
||||
## Tech stack
|
||||
|
||||
| Layer | Tech |
|
||||
@@ -367,18 +269,6 @@ cd server
|
||||
npm run swagger # → server/swagger/swagger-output.json
|
||||
```
|
||||
|
||||
`swagger.js` post-processes the generator's output in two ways before writing it:
|
||||
|
||||
- **Trailing slashes are stripped from path keys.** swagger-autogen builds a path by
|
||||
string-concatenating the mount prefix with the route argument, so a capability router mounted at
|
||||
`/users` whose collection route is `router.get('/')` would document as `/api/v1/admin/users/` —
|
||||
a URL no client calls, while dropping the one they all do. Express is indifferent (non-strict
|
||||
routing treats the two as one route), but the published spec is a contract.
|
||||
- **Path keys are sorted.** The generator emits them in router-traversal order, so moving a route
|
||||
between files rewrote most of this ~5k-line committed artifact even when the API was provably
|
||||
unchanged. Sorting keeps the diff proportional to the change. OpenAPI attaches no meaning to path
|
||||
order, and `scripts/routeManifest.js` already sorts for the same reason.
|
||||
|
||||
If the generated spec is missing, the server logs a warning and simply disables `/api/docs` (it does
|
||||
not crash).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user