The last phase of the workstream. ENGAGEMENT.md gets Phase 14's as-built
(the six org-lead decisions, three findings, the acceptance line discharged
and one rig trap), and its header now reads COMPLETE rather than "Phase 14
is scoped and not started".
BACKEND_DESIGN.md gets the reference: a new "Engagement retention" section
with the horizon table, why the outbox sweep is terminal-only, why each
floor is a correctness bound rather than a UI nicety, why suppressions do
not expire, and why this one got a screen when team_activity and
user_notifications did not. Plus the three new routes in the admin table and
the address_hash reversal on GET /suppressions.
Three findings worth not re-deriving:
* the per-row purge could not be built without reversing Phase 9's
decision to strip address_hash — the row had no handle, and the existing
lift asks for an address the screen has never shown.
* reclaimStale could not give up, so an interrupted send never became
terminal and no retention sweep could ever have bounded the outbox.
* the cooldown warning is unreachable through the UI by construction
(MAX_COOLDOWN_SECONDS is smaller than the horizon's floor). It is not
dead: it catches a hand-edited row, and a future raise of that ceiling.
And one the phase text got wrong in the other direction: the Play Data
Safety answers correctly did NOT move, because deploy-engagement is
deployment-scoped and PLAY_DATA_SAFETY.md is generated from the app-scoped
entries only.
Code: website#TBD.
Co-Authored-By: Claude <noreply@anthropic.com>
4480 lines
330 KiB
Markdown
4480 lines
330 KiB
Markdown
# The Engagement System — findings and plan
|
||
|
||
**Status:** design of record. **The workstream is COMPLETE (2026-09-01).**
|
||
Phases 0 through 14 are done and on `main` in every repository, and **Phase 13's acceptance walk
|
||
passed** — a clean install from `main`, released artefacts only, one email and one in-app item to the
|
||
linked owner and nothing to anyone else. It found two defects in shipped core; both are recorded in
|
||
Phase 13's as-built and both were fixed afterwards (website#181). **Phase 14 (retention) is BUILT** —
|
||
the last phase, and the one that gave the engagement schema a bound. Each phase carries its own
|
||
as-built section below, and the PRs are named there rather than here.
|
||
|
||
The cutover cut `link` **v2.1.0**, the plugin overlay **v1.1.0**, `module-uo` **v1.1.0** and the
|
||
paired bundle **2026.09.01** (protocol **5**); `MODULE_API_VERSION` is **1.9.0**. Two of Phase 13's
|
||
seven steps were missed in the window and landed afterwards — see its as-built, which is also where
|
||
the four findings of the cutover itself are recorded.
|
||
|
||
**Phases 10 and 11 were both widened on 2026-08-31, by the org lead, before any code:** the protocol
|
||
bump carries three wire enrichments rather than one, and Phase 11 ships **every ✅ row of §8.6**
|
||
rather than a single rule. **Phase 11 was then scoped on the same day** — six decisions, three
|
||
carve-outs, a seventh value in the ceiling lattice and an 11a/11b split; see its own decision block.
|
||
The scope decisions below are settled; **all nine questions in §7.1 are answered** — Q1, Q3, Q5 and
|
||
Q7 on 2026-08-28, Q6 on 2026-08-29 at the start of Phase 2 (which also settled §7.2's namespace
|
||
question), **Q2 and Q4 on 2026-08-29 at the start of Phase 4**, and **Q8 on 2026-08-31 at the start
|
||
of Phase 8**. Q1's answer added a whole phase (**Phase 1b**, unique email addresses); Q4's answer and
|
||
the phase's size split **Phase 4 into 4a and 4b**; **Q9** (core's own `news.post` emitter) was
|
||
answered **2026-08-31 at the start of Phase 11**, along with five further decisions that changed what
|
||
that phase ships — see its own decision block. Per CLAUDE.md § Conventions, no implementation starts
|
||
without the org lead's approval of the phase it belongs to; **Phase 14's six decisions were settled
|
||
on 2026-09-01 before any of its code**, and two of the four horizon answers widened it — see its
|
||
as-built.
|
||
|
||
**Branching:** every phase landed on **`edge`** in its repo, and `main` was touched once, by the
|
||
cutover (Phase 13). §6.0a records the blocking precondition it opened with — six `edge` branches
|
||
stale, three repos with none — and Phase 13's as-built records how it closed: **every `edge` was
|
||
deleted on merge rather than fast-forwarded**, which is the convention now. Phase 14 comes after the
|
||
cutover and so goes to `main` through an ordinary feature branch.
|
||
|
||
**Scope decisions, settled by the org lead (2026-08-28):**
|
||
|
||
1. **The in-app channel is in scope.** It does not exist today and has to be built, not adapted.
|
||
2. **The Teams notification pipeline is generalized and migrated onto the new system**, not built beside it.
|
||
3. **The `house.decay` protocol enrichment is in scope**, as a coordinated four-repo `PROTOCOL_VERSION`
|
||
bump. **Widened 2026-08-31:** that single bump now carries **three** enrichments (house decay,
|
||
player-vendor fee state, and a post-decision login result), because a second bump would cost another
|
||
release + bundle + operator update. See Phase 10.
|
||
4. **Gmail OAuth2 is removed, not retained as a transport.** SMTP is the baseline; the OAuth2 consent
|
||
flow, its two routes, its borrowed Google client and its stored refresh token all go. See §1.2a for
|
||
what that deletes and §6/Phase 1 for the operator cutover it forces.
|
||
5. **The system ships with a seeded set of working templates and an editor**, so a fresh deployment
|
||
sends correctly-branded mail before anyone opens the editor. See §4.6.
|
||
|
||
**Four further decisions, settled 2026-08-28** (recorded in full at §7.1, with the findings that shaped
|
||
them in §0.6):
|
||
|
||
6. **Engagement mail is opt-in only, and `users.email` becomes UNIQUE.** Standard marketing-email practice
|
||
applies: explicit opt-in, working unsubscribe, suppression. Uniqueness is not a detail — it is
|
||
**Phase 1b**, because the column is nullable-and-duplicated by design today and three code paths break
|
||
the moment an index is added. Whether an *unverified* address may receive opt-in mail is an **admin
|
||
setting**, defaulting **on for fresh installs and off for upgrades**.
|
||
7. **There is no campaigns surface.** No operator-authored send screen, no free-form list building. An
|
||
admin's expressive power lives in trigger conditions and rules.
|
||
8. **Audiences are module-declared and operator-composable.** A module registers named, queryable
|
||
audiences over *its own* data (`uo.team.members`, `uo.governors`); core exposes the same registration
|
||
surface to every module and learns no game vocabulary. An operator may compose declared audiences with
|
||
and/or/not into a saved segment — and the composed result is still bounded by the trigger's G24
|
||
audience ceiling. See §5.1a.
|
||
|
||
---
|
||
|
||
## Part 0 — Six findings that contradict the brief
|
||
|
||
Stated up front because the rest of the document is shaped by them.
|
||
|
||
### 0.1 There is no In-app channel. The target diagram's "existing In-app / Push" is one channel, not two
|
||
|
||
Core has exactly one notification sink: **content-free push tickles** to ntfy/UnifiedPush
|
||
(`website/server/src/utils/pushDispatch.js`). There is no notification table, no read API, no
|
||
mark-read, and no in-app list on either client. `android-app/.../ui/notifications/NotificationsScreen.kt`
|
||
is a *preferences* screen. `module-uo`'s SSE stream (`server/utils/shardBroadcast.js`) is a live game
|
||
feed, not an inbox.
|
||
|
||
So "add Email as a third channel" is really **add two channels to a system that has one**, and the
|
||
in-app one needs its own storage, read API and two client surfaces.
|
||
|
||
### 0.2 Email is already two-thirds of an engagement system — scoped to Teams
|
||
|
||
The brief frames the current email implementation as "the Gmail OAuth2 code". Gmail OAuth2 is only the
|
||
*transport*. Sitting on top of it, `utils/teamNotify.js` + `utils/teamDigestWorker.js` +
|
||
`team_notification_prefs` already implement:
|
||
|
||
| Engagement concern | Where it already lives |
|
||
| --- | --- |
|
||
| Trigger → recipient selection | `teamNotify.js:recipientIds` / `emailRecipients` via the Team access resolver |
|
||
| Per-user opt-in, per-scope | `team_notification_prefs.email_mode ENUM('off','digest','immediate')` (schema.sql:1285) |
|
||
| Immediate vs. digest scheduling | `teamNotify.emailImmediate` and `teamDigestWorker.tick` |
|
||
| Digest windowing + clamping | `teamDigestWorker.clampSince`, `MAX_LOOKBACK_MS`, `MAX_ITEMS` |
|
||
| One-click unsubscribe | `utils/unsubscribeToken.js` (stateless HMAC) + RFC 8058 `List-Unsubscribe-Post` |
|
||
| "Off unless configured" gate | `mailer.isConfigured()` checked *before* the recipient query |
|
||
| Multi-sink fan-out from one computed audience | `teamNotify.forumPost` → push + email + Discord bridge |
|
||
|
||
This is the prototype of the system being scoped. Building beside it would give the deployment two
|
||
unsubscribe mechanisms and two digest workers. Hence decision 2.
|
||
|
||
### 0.3 The `uo.house.idoc_warning` payload does not exist on the wire, and one field of it is not knowable
|
||
|
||
`house.decay` carries (`docs/link/INTEGRATION.md:207`,
|
||
`servuo-plugins/overlay/Scripts/Custom/Bridge/BridgeSweeps.cs:199`):
|
||
|
||
```
|
||
serial, from, to, map, x, y, z, region, name, ownerSerial, ownerAcct, ban{x,y,z}, builtOn, lastRefreshed
|
||
```
|
||
|
||
Against the brief's example payload:
|
||
|
||
| Brief field | Reality |
|
||
| --- | --- |
|
||
| `house` | ✅ `name` (from the house sign) |
|
||
| `location` | ✅ `map`, `x/y/z`, `region`, plus `ban` (where you stand to read the sign) |
|
||
| `decay_status` | ✅ `to` |
|
||
| `character` | ❌ **not emitted** — but trivially available: `WriteDecay` already reads `house.Owner`, so `owner.Name` is one line |
|
||
| `shard` | ❌ **not an event field, and should not become one** — one deployment is one shard; core already has `ctx.settings.getInstanceName()` |
|
||
| `next_stage` | ❌ **not emitted** — obtainable, see below |
|
||
| `estimated_collapse` | ❌ **not emitted, and not exactly knowable in advance** — see below |
|
||
|
||
**Why `estimated_collapse` is the hard one.** ServUO runs the *dynamic* decay system on any modern
|
||
shard (`Scripts/Multis/DynamicDecay.cs`: `Enabled => Core.ML`). Under it, each stage's duration is
|
||
drawn at random **when that stage is entered** (`GetRandomDuration`, e.g. `Greatly` = 1–2 days,
|
||
`IDOC` = 12–24 h). `BaseHouse.NextDecayStage` (`BaseHouse.cs:37`) is therefore an exact, already-persisted
|
||
timestamp for the *next* transition — but a collapse time two stages out does not exist yet, even
|
||
inside the game.
|
||
|
||
Consequences for the design:
|
||
|
||
- **`next_stage` is exact and cheap** — emit `NextDecayStage`.
|
||
- **`estimated_collapse` is exact only once the house is at IDOC**, where `NextDecayStage` *is* the
|
||
collapse time. Before that it can only be an envelope (min/max from the remaining stage table).
|
||
Emit it as `estimatedCollapse` only when `to == "IDOC"`, plus an optional
|
||
`estimatedCollapseMin`/`Max` envelope earlier — never a single number that reads as a promise.
|
||
- Under the legacy static path (`GetOldDecayLevel`, `BaseHouse.cs:205`) decay is a pure function of
|
||
`lastRefreshed + DecayPeriod`, and `lastRefreshed` is already on the wire. The **`decayPeriod`** is
|
||
not, and should be added so a consumer can compute stages without hardcoding 5 days.
|
||
|
||
**And the current mapping fires at the wrong stage for the brief's own example.** The brief's story is
|
||
"house becomes greatly damaged". `module-uo/server/config/shardStreams.js` maps `house.decay` to a
|
||
stream only when `String(event.to).toUpperCase() === 'IDOC'` — the *final* stage, which is 12–24 h from
|
||
collapse. A "greatly damaged" warning needs a `Greatly` transition mapping. That part needs **no
|
||
protocol change at all** and can ship long before the bump lands.
|
||
|
||
### 0.4 Event-name collision handling already exists — the brief's §6 forward-compat note is already satisfied
|
||
|
||
The brief asks for a one-line note that a module-id prefix would be needed *if* multi-module ever
|
||
happens. It already happened:
|
||
|
||
- `modules/registries.js:namespaced()` **requires** every stream id and announce-leg id to start with
|
||
`<owner>.`, with a small grandfathering allowlist (`LEGACY_STREAM_IDS`, `LEGACY_LEGS`) for the seven
|
||
pre-module-system ids.
|
||
- `registries.apply()` throws on any collision, **naming the current holder**, before it commits a
|
||
single claim.
|
||
- `modules/loader.js` cross-checks mounts, tables and slots across *all* loaded modules
|
||
(`for (const other of modules.values())`), so N modules is a supported configuration, not a future one.
|
||
|
||
`uo.house.idoc_warning` is therefore already the house style, and the trigger registry gets collision
|
||
handling for free by reusing `namespaced()` verbatim. **This is a resolved item, not a forward-compat
|
||
note.** The genuine forward-compat note is elsewhere — see §7.3.
|
||
|
||
### 0.5 `MODULE_API_VERSION` 1.6.0 is on `main` now, so this needs a real bump
|
||
|
||
`MODULE_API.md` §1.1 currently argues that additions may join 1.6.0 in place because "1.6.0 has only
|
||
ever been on `edge`". That is stale: `git show main:server/src/modules/version.js` reads `1.6.0`. The
|
||
Teams cutover landed it.
|
||
|
||
So the engagement additions take **1.7.0** — additions only (`api.registerEventTriggers`, `ctx.events.emit`,
|
||
`ctx.inbox.push`), no removal, no changed signature, so minor by the §1.1 table. `module-uo`'s
|
||
`coreApi: "^1.3.0"` still resolves.
|
||
|
||
---
|
||
|
||
### 0.6 A UNIQUE email is not a one-line ALTER — it breaks three paths and can stop a boot
|
||
|
||
Decision 6 (opt-in only, unique addresses) reads like a schema tweak. It is not. `users.email` is
|
||
`VARCHAR(255) NULL` with **no** unique index, and `schema.sql:24` says so deliberately: *"Optional
|
||
contact email (players). Not unique — SSO emails may repeat."* Adding the index touches registration,
|
||
SSO provisioning and the upgrade path. All four findings below were read out of the tree on 2026-08-28.
|
||
|
||
**1. The boot-time `ALTER` is how schema reaches a deployment, and it would fail loudly.** Upgrades ride
|
||
the idempotent `ALTER TABLE … IF NOT EXISTS` block at `schema.sql:1409+`, executed by `ensureSchema()`
|
||
on **every** boot (`server/src/server.js:66`). `ADD UNIQUE INDEX` against a table that already holds
|
||
duplicate addresses errors, `ensureSchema()` throws, and **the site does not start**. A de-duplication
|
||
step must run before the index, in the same release — see Phase 1b.
|
||
|
||
**2. `isDuplicateUsername()` does not inspect which index collided.** `users.model.js:24` is
|
||
`err.code === 'ER_DUP_ENTRY' || err.errno === 1062` and nothing more. Two callers misread an email
|
||
collision as a username collision the instant the index exists:
|
||
|
||
| Site | Today | After `UNIQUE(email)`, unfixed |
|
||
| --- | --- | --- |
|
||
| `auth.controller.js:137` (register) | 409 *"That username is already taken."* on a genuine username race | Same message for a duplicate **email** — wrong, and it misattributes the conflict to the one field the user did not collide on |
|
||
| `sso.controller.js:197` (`provisionSsoPlayer`) | Retries the next username suffix on collision | Retries **usernames** for an **email** conflict, which can never clear; burns `PROVISION_MAX_TRIES` and returns `null`, so SSO sign-up fails opaquely with the log blaming usernames |
|
||
|
||
The fix is to distinguish the constraint (read the index name off the driver error) before Phase 1b adds
|
||
the index — not after.
|
||
|
||
> **Amended 2026-08-29 (Phase 1b, as built).** This table names **two** callers of
|
||
> `isDuplicateUsername()`. There are **five**, and the three it omits fail worse than the two it
|
||
> names — `invite.controller.js` accepts an invite to an address already held and fails *after* the
|
||
> invitee has clicked the link and chosen a password, while `admin.controller.js` `createUser` and
|
||
> `updateUser` had **no catch at all** and turned a duplicate address into an opaque 500 for an admin
|
||
> who could see nothing wrong with the form. (`auth/account.controller.js` changeUsername is the
|
||
> fifth and is username-only, so it was already correct.) All five are handled; each answers
|
||
> differently on purpose, because a public form, an authenticated IdP callback, a half-completed
|
||
> invite and an admin screen do not owe the same person the same amount of truth.
|
||
>
|
||
> The index name is available **only in the driver's message text** — the mariadb connector exposes
|
||
> no structured field for it — so the discrimination is a regex over `for key '…'`, with its own
|
||
> test. That message also embeds the bound parameters, so on an email collision it *contains the
|
||
> address*: a second, independent reason these errors must never be echoed to a client.
|
||
|
||
**3. SSO auto-provisioning is the source of the duplicates, and CLAUDE.md is stale about it.** CLAUDE.md
|
||
states *"identities are never auto-provisioned"*. `provisionSsoPlayer` (`sso.controller.js:197`) does
|
||
exactly that whenever `player_registration ∈ {sso, both}`, writing `profile.email` straight into
|
||
`users.email`. It also sets `emailVerified: Boolean(profile.email)` — **verified because an address was
|
||
present, not because the IdP asserted `email_verified`**. That matters twice over: it manufactures the
|
||
duplicates Phase 1b must clean up, and it makes `email_verified` too weak a signal to arbitrate *which*
|
||
duplicate wins (which is why §7.1 Q1's answer is oldest-wins, not verified-wins).
|
||
|
||
**4. There is no self-serve email flow at all.** No route lets a user set or change their own address
|
||
after signup — `router/v1/auth/me.routes.js`, the self-service surface, has none, and
|
||
`users.model.js:72`'s `update()` is reached only by admin user management. An address is captured once,
|
||
at registration or SSO provisioning, and is thereafter unchangeable by its owner. A verification gate
|
||
presupposes a change-and-verify flow, so Phase 1b builds one; it is not an add-on to an existing
|
||
screen.
|
||
|
||
> Read out of the tree on 2026-08-28, when self-service had *three* URL surfaces onto one controller
|
||
> (`/auth/me/account/*`, `/player/account/*`, `/admin/account/*`), so a new field meant writing it
|
||
> three times. **The two smaller ones were deleted on 2026-08-29** — see Phase 1a below. Phase 1b's
|
||
> change-and-verify routes are written once, on `/auth/me/account`.
|
||
|
||
**One consequence for the error surface.** A unique constraint needs a user-facing failure, and the
|
||
obvious wording (*"that email is already registered"*) makes account existence queryable — a step back
|
||
from a posture the codebase holds deliberately elsewhere (`passwordReset.controller.js` answers a generic
|
||
200 *"to avoid account enumeration"*). §7.1 Q1 settles it: the message stays generic, the real reason is
|
||
logged not returned, the endpoint stays rate-limited, and the failure is **not** fed to the bot scorer —
|
||
an honest typo on a taken address must not push a legitimate user toward an IP ban.
|
||
|
||
---
|
||
|
||
## Part 1 — Current-state map
|
||
|
||
### 1.1 Notification system, end to end
|
||
|
||
```
|
||
registerNotificationStreams() [modules/registries.js]
|
||
│
|
||
config/coreStreams.js (news.post, team.*) ──┤ ← core, via registerCore()
|
||
module-uo/config/shardStreams.js (7 ids) ──┘ ← module, via api.*
|
||
|
||
catalog read back through registries.allStreams()
|
||
│
|
||
GET /auth/me/notifications/streams ──────────────────► web + Android
|
||
GET·PUT /auth/me/notifications/subscriptions ─────────► notification_subscriptions
|
||
│
|
||
shard event ──► module-uo/utils/shardPush.js ─┤
|
||
news publish ─► posts controller ─────────────┤──► pushDispatch.publish(streamId,{ref,ownerUserId})
|
||
team event ───► utils/teamNotify.js ──────────┘──► pushDispatch.publishToUsers(streamId,{ref,userIds})
|
||
│
|
||
push_devices rows (SSRF-gated endpoints)
|
||
│
|
||
POST {stream, ref} ──► ntfy / UnifiedPush ──► app wakes, PULLS content
|
||
```
|
||
|
||
**Files.**
|
||
|
||
| Concern | File |
|
||
| --- | --- |
|
||
| Catalog registry (core + modules) | `server/src/modules/registries.js` — `registerNotificationStreams`, `allStreams`, `isValidStream`, `personalStreams` |
|
||
| Core's own streams | `server/src/config/coreStreams.js` — `news.post` + four `team.*` |
|
||
| Module streams | `module-uo/server/config/shardStreams.js` — seven grandfathered ids + `mapShardEvent` |
|
||
| Fan-out | `server/src/utils/pushDispatch.js` — `publish` (all-subscribers / one-owner), `publishToUsers` (computed set), `isAllowedEndpoint` (SSRF gate) |
|
||
| Shard → push adapter | `module-uo/server/utils/shardPush.js` — resolves `ownerAcct` → website user via `shardLinks` |
|
||
| Team fan-out | `server/src/utils/teamNotify.js` — one audience, three sinks |
|
||
| Self-service API | `server/src/router/v1/auth/notifications.controller.js` + `notifications.routes.js` |
|
||
| Subscriptions model | `server/src/model/notificationSubs/` |
|
||
| Devices model | `server/src/model/pushDevices/` |
|
||
| Android surface | `android-app/.../data/api/NotificationsApi.kt`, `dto/NotificationsDto.kt`, `ui/notifications/` |
|
||
|
||
**Data model.**
|
||
|
||
```sql
|
||
-- schema.sql:411
|
||
push_devices(id, user_id, transport ENUM('unifiedpush','fcm'), endpoint, platform, created_at, last_seen_at)
|
||
-- schema.sql:428
|
||
notification_subscriptions(user_id, stream_id VARCHAR(64), created_at, PRIMARY KEY(user_id, stream_id))
|
||
-- schema.sql:1285
|
||
team_notification_prefs(user_id, team_id, muted, email_mode ENUM('off','digest','immediate'), last_digest_at, updated_at)
|
||
```
|
||
|
||
**Can it cleanly take Email as a third channel? Partly — and the coupling is in one place.**
|
||
|
||
- ✅ **The catalog is channel-neutral.** A stream entry is `{ id, label, description, personal, requiresLinkedAccount }`.
|
||
Nothing in it is push-specific. It can describe an email or in-app subscription unchanged.
|
||
- ✅ **The registry mechanism generalizes.** `stage()` / `apply()` / `namespaced()` are about *claims and
|
||
collisions*, not about push.
|
||
- ❌ **`notification_subscriptions` has no channel dimension.** `PRIMARY KEY (user_id, stream_id)` means a
|
||
subscription is a boolean, and "subscribed" currently means exactly "push me a tickle".
|
||
- ❌ **The wire shape is frozen by a shipped client.** `NotificationSubscriptionsDto` is
|
||
`{ streams: List<String> }` and the app PUTs the whole set. Turning that array into objects breaks
|
||
every installed app. (The DTO ignores *unknown keys*, so purely **additive** fields are safe — this
|
||
is recorded in the app's own comments.) A per-channel model must therefore arrive as a **new
|
||
endpoint**, with the old one preserved as the push projection.
|
||
- ❌ **Email opt-in semantics differ from push, deliberately.** `team_notification_prefs` documents the
|
||
asymmetry in its own DDL comment: push is opt-*out* (`muted` defaults 0), email is opt-*IN*
|
||
(`email_mode` defaults `'off'`), because digest-by-default would start mailing everyone the moment an
|
||
operator connects a mailbox. Any unified model must keep per-channel defaults, not one shared default.
|
||
|
||
**Self-hosted ntfy / UnifiedPush integration points** (email must sit beside these, not duplicate them):
|
||
|
||
- `pushDispatch.isAllowedEndpoint` — HTTPS-only, private-host denylist, plus an origin allow-set from
|
||
`NTFY_ALLOWED_ORIGINS` / `NTFY_BASE_URL`. **There is no hardcoded default host.** Empty allow-set is
|
||
the dev fallback.
|
||
- `NTFY_PUBLISH_TOKEN` — optional bearer for the relay.
|
||
- The **content-free tickle** invariant: `{ stream, ref }` and nothing else, because ntfy is treated as
|
||
an untrusted relay. Email deliberately breaks that rule (a mailbox is a destination the recipient
|
||
chose) and `teamNotify.js`'s header comment is the standing argument for why the asymmetry is the
|
||
security model rather than an inconsistency. **The engagement system must preserve this per channel,
|
||
not flatten it.**
|
||
|
||
### 1.2 Current email implementation
|
||
|
||
**Transport.** `server/src/utils/mailer.js` (267 lines). nodemailer over `smtp.gmail.com:465` with
|
||
`auth.type: 'OAuth2'`. Client id/secret are *reused from the `google` auth_providers row*; only the
|
||
refresh token is email-specific. `buildTransport()` returns `null` when unconfigured, and every sender
|
||
handles that itself.
|
||
|
||
**Config.** `email_config` singleton (schema.sql:336) — `provider` (already a `VARCHAR(20)` defaulting
|
||
`'gmail_oauth2'`, so the column is ready for a second provider), `enabled`, `sender_email`, `sender_name`,
|
||
`refresh_token_enc` (AES-256-GCM via `utils/secretBox.js`), `status`, `status_detail`, `last_verified_at`.
|
||
Admin-managed, never env — `docs/website/BACKEND_DESIGN.md` §7.
|
||
|
||
**Admin API.** `router/v1/admin/email.router.js` — six routes: `GET·PUT /config`, `GET /connect/start`,
|
||
`GET /connect/callback`, `POST /test`, `POST /disconnect`. Client: `client/src/routes/admin/views/EmailDelivery.jsx`.
|
||
|
||
**Every caller — the full migration surface.** Six call sites, five sender functions:
|
||
|
||
| Caller | Function | Failure contract |
|
||
| --- | --- | --- |
|
||
| `router/v1/public/public.controller.js:156` (contact form) | `sendContactMessage` | **Never throws when unconfigured** — returns `{sent:false, fallback:'mailto', email}` and the client renders a `mailto:` link |
|
||
| `router/v1/admin/emailConfig.controller.js:188` (admin "Send test") | `sendTest` | Throws `NOT_CONFIGURED` / `NO_RECIPIENT`; 502 to the admin |
|
||
| `router/v1/admin/invites.controller.js:46` | `sendInvite` | Returns `{sent:false, reason:'NOT_CONFIGURED'}` so the admin gets the accept link to share by hand |
|
||
| `router/v1/auth/passwordReset.controller.js:49` | `sendPasswordReset` | Returns `{sent:false, …}`; caller still answers a generic 200 to avoid account enumeration |
|
||
| `utils/teamNotify.js:234` (immediate) | `sendTeamNotification` | **Never throws at all** — logged and swallowed; the forum write already returned |
|
||
| `utils/teamDigestWorker.js:73` (digest) | `sendTeamNotification` | Same; return value gates the `last_digest_at` stamp |
|
||
|
||
Nothing in `website/bot`, `module-uo`, or any other repo sends mail. `mailer` is not on `ctx` — **modules
|
||
already cannot send email**, which matches the target architecture.
|
||
|
||
**Hardcoded assumptions the abstraction has to remove:**
|
||
|
||
1. **One provider, compiled in.** The Gmail host, port and OAuth2 auth type are literals in
|
||
`buildTransport()`. `email_config.provider` exists but nothing reads it.
|
||
2. **The credential shape is Gmail's.** One `refresh_token_enc` column plus a borrowed OAuth client.
|
||
SMTP needs host/port/secure/user/password; SES needs a region and IAM keys; Mailgun/SendGrid need a
|
||
domain and an API key. None of those fit the current column set.
|
||
3. **One transport built per send.** `buildTransport()` runs on every call — a fresh DB read, a fresh
|
||
decrypt and a fresh nodemailer transport per message, with no pooling. Fine for a password reset;
|
||
the serial per-recipient loop in `emailImmediate` is explicitly a rate-limit workaround for it.
|
||
4. **Text-only, composed inline.** Every body is a template literal inside `mailer.js`. There is **no
|
||
HTML part anywhere** and no template storage. `sendTeamNotification` builds its body by pushing lines
|
||
into an array.
|
||
5. **Synchronous send, no queue, no retry.** A send either succeeds inside the request/tick or is lost.
|
||
`recordStatus` writes the last outcome to a singleton column — there is no per-message record, so
|
||
"did user X get the IDOC mail?" is unanswerable today.
|
||
6. **`recordStatus` is global.** One transient failure sets `email_config.status='error'` for the whole
|
||
deployment, from any of six unrelated call sites.
|
||
7. **No suppression, no bounce handling, no verification gate.** `users.email` is **not unique**
|
||
(SSO addresses repeat) and `users.email_verified` is set to 1 only on invite-accept
|
||
(`invite.controller.js:55`) and SSO (`sso.controller.js:208`). Self-registration accepts an address
|
||
and leaves it unverified (`auth.controller.js:124`). Today only *transactional* mail goes out, so
|
||
this is tolerable; the moment game events drive volume it is a deliverability and complaint problem.
|
||
|
||
### 1.2a Removing Gmail OAuth2 — the deletion inventory *(decision 4)*
|
||
|
||
Gmail OAuth2 is not a transport we keep beside SMTP. It goes. That is a **subtraction with a live
|
||
deployment behind it**, so the exact surface is worth writing down before anyone starts.
|
||
|
||
**Server — deleted:**
|
||
|
||
| Thing | Where |
|
||
| --- | --- |
|
||
| `GET /admin/email/connect/start` | `router/v1/admin/email.router.js` |
|
||
| `GET /admin/email/connect/callback` | same |
|
||
| `connectStart` / `connectCallback` | `router/v1/admin/emailConfig.controller.js` (~half the file's 211 lines) |
|
||
| The `email_oauth_tx` signed cookie, the PKCE verifier and CSRF nonce plumbing | same controller |
|
||
| `EMAIL_SCOPE = 'https://mail.google.com/ openid email'` | same |
|
||
| `googleClient()` — the borrowed `google` auth_providers credential read | same |
|
||
| The OAuth2 nodemailer transport (`auth.type: 'OAuth2'`, `smtp.gmail.com:465` literals) | `utils/mailer.js:buildTransport` |
|
||
| `emailConfig.getWithSecret()`'s `refreshToken` decrypt | `model/emailConfig/emailConfig.model.js` |
|
||
|
||
**Client — deleted:** the "Connect Gmail" button and `connect()` handler, the
|
||
`?email_connected` / `?email_error` redirect-banner handling, and the five Gmail-specific error strings
|
||
(`bad_state`, `no_client`, `no_refresh_token`, …) in `client/src/routes/admin/views/EmailDelivery.jsx`.
|
||
Replaced by an ordinary credential form driven by the transport's `credentialFields` (§3.1).
|
||
|
||
**Database — deprecated, not dropped.** `email_config.refresh_token_enc` and `provider` stay as columns
|
||
(additive-only discipline; core's `schema.sql` contains exactly one `DROP` and it is documented as such).
|
||
They stop being read. A later cleanup PR may drop them once every deployment has booted past the cutover.
|
||
|
||
**Three consequences worth naming:**
|
||
|
||
1. **SSO is unaffected.** The `google` auth_providers row exists for SSO in its own right; email merely
|
||
*borrowed* its client id/secret. Removing the borrow removes a coupling — one of the better side
|
||
effects of this decision, since today an admin who rotates the Google SSO secret silently breaks
|
||
outbound mail with no indication that the two are related.
|
||
2. **`sender_email` changes meaning.** Today it is read back from Google's `userinfo` and is therefore
|
||
guaranteed to be an address the mailbox owns. Under SMTP it is **operator-typed**, so nothing stops a
|
||
mismatch between the envelope sender and what the SMTP account is permitted to send as — which is a
|
||
silent deliverability failure (SPF/DMARC), not an error. The admin "Send test" path has to become the
|
||
real verification, and its failure text has to be specific enough to diagnose a rejected `From`.
|
||
3. **The live deployment goes dark at cutover unless the operator acts.** UOMysticmoon is connected via
|
||
Gmail OAuth2 today. On upgrade, `transport` backfills to `smtp` with **no credentials**, so
|
||
`isConfigured()` returns false and every sink politely does nothing — the contact form falls back to
|
||
`mailto`, invites surface a copyable link, password resets still answer a generic 200. Nothing breaks
|
||
loudly, which is precisely the risk: **email silently stops and nobody is told.** Phase 1 therefore
|
||
owes three things: an admin dashboard warning when `transport='smtp'` and credentials are absent, a
|
||
release note naming the required action, and `INSTALL.md`-style operator guidance. Gmail itself
|
||
remains usable as plain SMTP (`smtp.gmail.com:587` with an app password), which is the shortest
|
||
migration path for the existing deployment and should be the documented one.
|
||
|
||
### 1.3 Module contract fit
|
||
|
||
**The registration surface, as it stands** (`modules/registries.js`, `MODULE_API.md` §2.4):
|
||
|
||
| Call | Shape | Cardinality |
|
||
| --- | --- | --- |
|
||
| `registerRoutes` | tier → prefix → router | declared in `module.json`, cross-checked |
|
||
| `registerExtension(slot, router)` | fills a **core-declared** slot | one filler per slot |
|
||
| `registerNotificationStreams([…])` | push catalog entries | many, `<owner>.`-prefixed |
|
||
| `registerAnnounceLeg({leg, label, dispatch, classify})` | a **delivery leg** with retry classification | many, `<owner>.`-prefixed |
|
||
| `registerPostHook({onSaved, onDeleted})` | idempotent state mirroring | one per owner |
|
||
| `registerTeamProvider({…})` | core **calls the module and waits** | one per deployment |
|
||
| `registerSlashCommands([…])` | definition travels, handler stays | many, *not* namespaced (Discord grammar) |
|
||
|
||
**Is there a natural extension point? Yes — and `registerAnnounceLeg` is the closest structural match,
|
||
but for the *delivery* half, not the *trigger* half.**
|
||
|
||
The engagement system needs **two** things a module does not have today:
|
||
|
||
1. **A way to declare a domain event and its data contract** — nothing like this exists. `registerNotificationStreams`
|
||
declares a *subscription toggle*; it carries a label and two booleans, and no statement whatsoever
|
||
about payload. A module cannot tell core what a `uo.house.idoc_warning` *contains*.
|
||
2. **A way to emit one.** Today `ctx.push.publish(streamId, {ref, ownerUserId})` is the only outbound
|
||
path, and it is deliberately content-free. A module that wanted to send a *rendered* message has to
|
||
go through `pushDispatch`, which will not carry the data.
|
||
|
||
So this needs a **new registration surface**, not a reuse. It should be modelled on `registerNotificationStreams`
|
||
(shape-checked at the call, collision-checked at `apply()`, `<owner>.`-prefixed) rather than on
|
||
`registerAnnounceLeg` (which is a *core-calls-module* dispatch with retry classification — the wrong
|
||
direction: a module *reports* an event, it does not deliver one).
|
||
|
||
**What module-uo exposes to core today.** Only what the registries take: seven stream ids, one announce
|
||
leg, one extension router, a Team provider, one slash command, five route mounts. Everything else — 27
|
||
`shard_*` tables, the sidecar client, the visibility framework — is module-internal (`MODULE_API.md` §1.2).
|
||
There is **no data channel from a module into core carrying structured game data**. `ctx.teams.activity.push`
|
||
is the nearest thing, and it is instructive: core stores `summary` **already rendered by the module**,
|
||
because core cannot phrase a sentence in a vocabulary it does not know, and `kind`/`payload` are opaque.
|
||
|
||
The engagement system deliberately takes the **opposite** position — core *does* interpolate module data
|
||
into a template — which is only safe because the operator authors the template and the module *declares*
|
||
the variables. That is the whole reason §4.3's variable contract has to exist rather than being optional.
|
||
|
||
### 1.4 Job, scheduling and queue infrastructure that already exists
|
||
|
||
**There is no cron. There is no Redis, no BullMQ.** The stack has exactly two patterns:
|
||
|
||
**(a) In-process `setInterval` + `unref()` + `stop()`, wired into `server.js` start/shutdown.** Six of them:
|
||
`announceWorker`, `teamDigestWorker`, `teamActivityPrune`, `teamForumUploadSweep`, `teamVoiceSync`, and
|
||
`middleware/botScore`'s sweeper.
|
||
|
||
**(b) A durable job table with per-leg backoff** — `announce_jobs` + `announce_job_legs` (schema.sql:737/757),
|
||
swept by `announceWorker.tick`. This is a real outbox: `status`, `attempts`, `last_error`,
|
||
`next_attempt_at`, `INDEX idx_announce_leg_due (status, next_attempt_at)`, and a `classify()` that maps a
|
||
delivery result to done / retry / terminal. **It was explicitly designed so a module can add a delivery
|
||
leg without altering a core table** — the child-table shape and the `VARCHAR` (not `ENUM`) `leg` column
|
||
are both justified in the DDL comment on exactly those grounds.
|
||
|
||
**Which of the brief's two scheduling use cases each pattern serves:**
|
||
|
||
- **(a) Delayed send per event** — "wait 30 min in case the player fixes it". Needs a durable row with a
|
||
`due_at`, and — the part the brief does not name but which is the actual point — the ability to
|
||
**cancel** a pending row when a later event resolves the condition. `announce_jobs` is the exact
|
||
precedent; this needs its own table because a module cannot alter a core one and the payload differs.
|
||
- **(b) Batched / digest** — `teamDigestWorker` already does this, and its header comment is the design
|
||
argument: it **computes at send time and keeps no queue**, whose three consequences are (1) a
|
||
deployment down for two days sends *one* digest, not a replay, (2) content hidden after it was written
|
||
is not in the query so not in the mail, and (3) **a user who lost access between the post and the send
|
||
is no longer in the recipient set** — which it calls out as the one that would have been a security bug.
|
||
|
||
**That split is the answer to the brief's §4 scheduling question: (a) is a queue, (b) must not be.**
|
||
Copying (a) for digests would reintroduce all three problems.
|
||
|
||
**One gap in both patterns: neither is multi-instance safe.** No advisory lock, no leader election, no
|
||
`SELECT … FOR UPDATE SKIP LOCKED`. Two app containers means two digest sweeps and two announce workers.
|
||
The current deployment is single-instance (`website/docker-compose.yml`), so this is latent — but an
|
||
engagement mailer doubles messages rather than doubling reads, so it becomes visible here first.
|
||
|
||
**Reusable primitives worth naming:**
|
||
|
||
- `utils/unsubscribeToken.js` — a stateless HMAC whose whole capability is "set `muted` for one (user, Team)
|
||
pair". Generalizes to (user, channel, trigger) with no structural change.
|
||
- `utils/settingsJson.js` — the fail-safe JSON-settings parse (malformed ⇒ *absent*, never an error).
|
||
- `blocks/registry.js` + `blocks/types/*` + `sanitizeBlocks.js` + `validateBlocks.js` — a versioned,
|
||
schema-validated, sanitize-on-save visual block system already driving `pages.blocks` (MEDIUMTEXT JSON).
|
||
This is the template editor's foundation; see §4.4.
|
||
- `utils/secretBox.js` — AES-256-GCM for provider credentials at rest.
|
||
- `model/settings/settings.model.js` + the `settings` key/value table — right for a handful of scalars,
|
||
**wrong for cooldowns** (see §4.1).
|
||
|
||
---
|
||
|
||
## Part 2 — Gap list against the target architecture
|
||
|
||
```
|
||
Game Module ──► Domain Events + Data ──► Core ──► Engagement ──► Preferences ──► Template ──► Delivery
|
||
```
|
||
|
||
| # | Layer | Gap | Severity |
|
||
| --- | --- | --- | --- |
|
||
| G1 | Module → events | No way for a module to **declare** a domain event or its payload contract | Blocking |
|
||
| G2 | Module → events | No way for a module to **emit** one carrying data (`ctx.push.publish` is content-free by design) | Blocking |
|
||
| G3 | Events → Core | No event **catalog** surface for the admin UI to enumerate triggers | Blocking |
|
||
| G4 | Engagement | No **rules** concept at all — today a trigger's consequence is hardcoded in the emitting file | Blocking |
|
||
| G5 | Engagement | No **cooldown / rate-limit** state of any kind, per-recipient or otherwise | Blocking (brief calls this out as pre-first-trigger) |
|
||
| G6 | Engagement | No **delayed-send** queue and no cancellation | High |
|
||
| G7 | Engagement | Digest exists but is **Team-shaped**, not generic (`last_digest_at` lives on `team_notification_prefs`) | High |
|
||
| G8 | Preferences | `notification_subscriptions` has **no channel dimension**; the shipped app's wire shape is frozen | Blocking |
|
||
| G9 | Preferences | Per-channel **defaults differ** (push opt-out, email opt-in) and there is nowhere to express that generically | High |
|
||
| G10 | Preferences | Unsubscribe is **Team-scoped** (`unsubscribeToken.sign(userId, teamId)`) | Medium |
|
||
| G11 | Template | **No template storage, no HTML part, no renderer, no preview, no plain-text fallback.** Every body is a string literal in `mailer.js` | Blocking |
|
||
| G12 | Template | No **variable contract** — nothing declares what a template may interpolate | Blocking (see §4.3) |
|
||
| G13 | Delivery | **One hardcoded provider**; `email_config.provider` is written but never read | Blocking |
|
||
| G14 | Delivery | Credential schema is Gmail-shaped (one refresh token + a borrowed OAuth client) | Blocking |
|
||
| G15 | Delivery | No per-message record — **no send log, no delivery status, no audit** | High |
|
||
| G16 | Delivery | No **suppression list**, no bounce/complaint handling, no unverified-address policy | High |
|
||
| G17 | Channels | **In-app channel does not exist** — no table, no read API, no web surface, no app surface | Blocking (in scope) |
|
||
| G18 | Infra | Workers are **not multi-instance safe** — latent today, doubles *messages* under engagement | Medium |
|
||
| G19 | Contract | `MODULE_API_VERSION` must go 1.6.0 → **1.7.0** (§0.5) | Process |
|
||
| G20 | Wire | `house.decay` lacks `ownerName`, `nextStage`, `decayPeriod`, collapse estimate (§0.3) | High (in scope) |
|
||
| G21 | Ops | No **preview/test-send** path for a template against a real trigger payload | Medium |
|
||
| G22 | Delivery | Removing Gmail OAuth2 leaves the live deployment **silently unconfigured** — every sink degrades quietly, so email stops with no signal (§1.2a) | High (in scope) |
|
||
| G23 | Template | No **seeded default templates** — without them, "add a trigger" implies "and now author a template", and a fresh install mails nothing (§4.6.1) | High (in scope) |
|
||
| G24 | Engagement | An audience **ceiling** per trigger. `shardStreams.js` already filters sensitive kinds off the public push path; the engine needs the equivalent or a rule can widen a staff-only trigger to everyone (§8.6) | Blocking (security) |
|
||
| G25 | Engagement | No **time-based** trigger kind — "nothing happened for 30 days" is a periodic evaluator, not an event (§8.5) | Medium (design in Phase 2, build later) |
|
||
|
||
---
|
||
|
||
## Part 3 — The delivery abstraction (brief §5)
|
||
|
||
### 3.1 Name it `DeliveryChannel`, and split *channel* from *transport*
|
||
|
||
The brief asks whether the interface should be `EmailProvider` or something more generic. **Neither
|
||
alone.** Two axes are being conflated, and the current code conflates them too:
|
||
|
||
- A **channel** is *what kind of sink this is* — email, push, in-app, later Discord DM. It determines the
|
||
address kind (mailbox / endpoint URL / user id / snowflake), the render contract (subject + HTML + text
|
||
vs. `{stream, ref}` vs. an embed), the preference semantics, and whether content may ride at all.
|
||
- A **transport** is *how one channel actually delivers* — SMTP / Gmail-OAuth2 / Mailgun / SES / SendGrid
|
||
for email; ntfy-UnifiedPush / FCM for push.
|
||
|
||
Push already has this shape and nobody named it: `push_devices.transport ENUM('unifiedpush','fcm')` is a
|
||
transport column on a channel that has exactly one implementation today.
|
||
|
||
**Recommended surface:**
|
||
|
||
```js
|
||
// core-internal registry, mirroring modules/registries.js's shape
|
||
registerDeliveryChannel({
|
||
id: 'email', // 'email' | 'push' | 'inapp' | later 'discord.dm'
|
||
label: 'Email',
|
||
carriesContent: true, // false for push — enforces the tickle invariant structurally
|
||
defaultMode: 'off', // G9, expressed here once. All three are opt-IN as built —
|
||
// 'push opt-OUT' was wrong; see Phase 3's as-built
|
||
supportsDigest: true, // in-app and push are instant-only in v1
|
||
addressFor(userId), // → [{ address, meta }] ; email reads users.email, push reads push_devices
|
||
render(template, vars, ctx), // → the channel's own payload shape
|
||
deliver(address, payload), // → { ok, retryable, error } — never throws
|
||
})
|
||
|
||
registerMailTransport({
|
||
id: 'smtp', // 'smtp' | 'mailgun' | 'ses' | 'sendgrid' — NOT gmail_oauth2 (removed)
|
||
label: 'SMTP',
|
||
credentialFields: [...], // drives the admin form AND the encrypted credential blob
|
||
build(config), // → a nodemailer transport (or an API client)
|
||
verify(config), // → the admin "Send test" path
|
||
})
|
||
```
|
||
|
||
**Answering the brief's actual question: no, this will not need a breaking rename when Discord DM
|
||
arrives.** A Discord DM is a `registerDeliveryChannel({ id: 'discord.dm', carriesContent: true, … })`
|
||
whose `deliver` calls `utils/botInternalClient.js` — the bot-internal API already exists and
|
||
`utils/teamBridge.js` is the working precedent for core handing a composed message to the bot. Nothing
|
||
in the interface above says "email".
|
||
|
||
**One vocabulary warning.** The announce pipeline already calls a delivery a **leg**
|
||
(`registerAnnounceLeg`, `announce_job_legs.leg`). `channel` and `leg` will coexist and mean *nearly* the
|
||
same thing. They should stay distinct rather than being unified: a leg is a **one-shot delivery of one
|
||
artifact** with retry and terminal classification; a channel is a **per-recipient sink** with preferences,
|
||
addresses and digest semantics. `teamBridge.js`'s header already argues this distinction for the Discord
|
||
case ("one-shot, not queued… a notification is the moment it describes"). The design doc should say so
|
||
explicitly so nobody "tidies" them together later.
|
||
|
||
### 3.2 No phone-home — the existing posture is already correct, and the registry must preserve it
|
||
|
||
Confirmed, and there is nothing to fix — only something to not break:
|
||
|
||
- `email_config` is **DB-backed and admin-managed**, never env (`BACKEND_DESIGN.md` §7). There is no
|
||
default host, no default sender, and `status` starts `'unconfigured'`.
|
||
- `pushDispatch.allowedOrigins()` reads `NTFY_ALLOWED_ORIGINS` / `NTFY_BASE_URL` and **returns empty when
|
||
neither is set**. No Runic Gateway host appears anywhere in it.
|
||
- `mailer.isConfigured()` gates every sink, and `teamNotify.emailImmediate` checks it *before* the
|
||
recipient query so an unconfigured deployment pays nothing. **As of Phase 6 that sink is the engine's**
|
||
and the gate moved with it: `mailer.sendNotification` returns a *retryable* failure when mail is
|
||
unconfigured, so an operator midway through typing SMTP credentials finds the outbox drains rather
|
||
than a backlog the worker gave up on. The digest worker still checks `isConfigured()` up front, before
|
||
any query that costs anything.
|
||
|
||
**Rules to carry into the abstraction:**
|
||
|
||
1. No transport may ship a default host, endpoint, API base or sender. A transport with no operator
|
||
configuration is `unconfigured` and its channel is **off**, not defaulting to anything.
|
||
2. No engagement code may read an env var naming an external service that the operator did not set.
|
||
3. The "off unless configured" gate is checked before recipient resolution, per channel.
|
||
4. A CI guardrail: extend `scripts/checkModuleIdentifiers.js`'s sibling pattern with a check that no
|
||
file under `server/src/engagement/` contains a bare external hostname literal. (Cheap; the check
|
||
pattern and its self-test discipline already exist — see `test/checkModuleIdentifiers.test.js`, which
|
||
feeds the checker code it *must* reject precisely so a check cannot silently stop checking.)
|
||
|
||
---
|
||
|
||
## Part 4 — Proposed schema additions
|
||
|
||
All additive. All `CREATE TABLE IF NOT EXISTS` / `ALTER … ADD COLUMN IF NOT EXISTS`, replayed on every
|
||
boot, per `MODULE_API.md` §2.6's rules (which core's own `schema.sql` follows too). **Core tables, no
|
||
prefix** — every one of these is game-agnostic.
|
||
|
||
> **MariaDB trap, already learned twice in this codebase:** a `PRIMARY KEY` column is coerced `NOT NULL`,
|
||
> so "NULL means the default row" is unrepresentable in a PK. `team_integration_config` (schema.sql:1335)
|
||
> and `teams.active_key` both work around it with a surrogate key plus a generated column folding NULL onto
|
||
> a sentinel. Two tables below need the same treatment; both are flagged.
|
||
|
||
### 4.1 Cooldowns — its own table, not `settings`
|
||
|
||
The brief asks whether the `settings.model.js` JSON-value pattern is a reasonable fit. **No.**
|
||
`settings` is `(key VARCHAR(64) PRIMARY KEY, value TEXT)` — a single-row-per-key store read whole. Cooldown
|
||
state is high-cardinality (recipients × rules × subjects), written on every fire, and queried as
|
||
"is this one pair still cooling?". A JSON blob under one key would be a read-modify-write of the entire
|
||
deployment's cooldown state on every event, with a lost-update race between two concurrent triggers. It is
|
||
the wrong shape by an order of magnitude.
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS engagement_cooldowns (
|
||
rule_id INT NOT NULL,
|
||
user_id INT NOT NULL,
|
||
-- The SUBJECT the cooldown is about, opaque to core: a house serial, a vendor id, ''.
|
||
-- NOT NULL with a '' default, because this is a PRIMARY KEY column and MariaDB
|
||
-- would coerce a NULL one anyway. '' is "this rule cools per user, not per subject".
|
||
subject_key VARCHAR(190) NOT NULL DEFAULT '',
|
||
last_fired_at DATETIME NOT NULL,
|
||
fire_count INT NOT NULL DEFAULT 1,
|
||
PRIMARY KEY (rule_id, user_id, subject_key),
|
||
CONSTRAINT fk_engc_rule FOREIGN KEY (rule_id) REFERENCES engagement_rules(id) ON DELETE CASCADE,
|
||
CONSTRAINT fk_engc_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
|
||
INDEX idx_engc_sweep (last_fired_at)
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
```
|
||
|
||
**Why `subject_key` is not optional.** "One IDOC mail per player per day" is the wrong rule — a player
|
||
with four houses decaying should hear about all four, once each. Cooling per (rule, user) alone silently
|
||
drops three of them. The module supplies `subject` on emit; core stores it opaquely.
|
||
|
||
`INDEX idx_engc_sweep (last_fired_at)` exists so a prune worker can drop rows older than the longest
|
||
configured cooldown — otherwise this table grows without bound, which is the failure mode
|
||
`teamActivityPrune` was written for.
|
||
|
||
**The check must not be a read-then-write**, or two concurrent emits both see an expired cooldown and
|
||
both send. The obvious single statement — `INSERT … ON DUPLICATE KEY UPDATE` with the interval guard in
|
||
the assignments, reading the answer out of `affectedRows` — **does not work against this codebase's
|
||
pool**, and Phase 4a is where that was found: the mariadb connector defaults `foundRows: true`, so a
|
||
no-op update reports 1 rather than 0 and every cooldown passes. What ships instead is a guarded UPDATE
|
||
(the interval in a WHERE clause, where a row either matches or does not) falling back to an
|
||
`INSERT IGNORE` for the first fire. See Phase 4a's as-built for the statements and the races.
|
||
|
||
### 4.2 Scheduling — a queue for delay, and deliberately no queue for digest
|
||
|
||
**(a) Delayed send ⇒ `engagement_outbox`.** Modelled on `announce_jobs`/`announce_job_legs`.
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS engagement_outbox (
|
||
id BIGINT AUTO_INCREMENT PRIMARY KEY,
|
||
rule_id INT NOT NULL,
|
||
trigger_id VARCHAR(96) NOT NULL, -- denormalized; survives a rule edit
|
||
user_id INT NOT NULL,
|
||
channel VARCHAR(32) NOT NULL, -- 'email' | 'push' | 'inapp' | … VARCHAR, never ENUM
|
||
subject_key VARCHAR(190) NOT NULL DEFAULT '',
|
||
payload JSON NOT NULL, -- the module's declared variables, snapshotted at emit
|
||
-- Idempotent enqueue. A sidecar reconnect that replays the same event must not
|
||
-- produce a second mail. Same reasoning as ctx.teams.activity.push's dedupeKey.
|
||
dedupe_key VARCHAR(190) NULL,
|
||
status ENUM('scheduled','sending','sent','failed','cancelled','suppressed') NOT NULL DEFAULT 'scheduled',
|
||
due_at DATETIME NOT NULL,
|
||
attempts SMALLINT NOT NULL DEFAULT 0,
|
||
last_error TEXT NULL,
|
||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||
sent_at DATETIME NULL,
|
||
CONSTRAINT fk_engo_rule FOREIGN KEY (rule_id) REFERENCES engagement_rules(id) ON DELETE CASCADE,
|
||
CONSTRAINT fk_engo_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
|
||
-- SCOPED, not global. One event legitimately becomes one row per (rule, user,
|
||
-- channel); a global unique index would admit the first recipient's row and
|
||
-- silently ignore every other. Corrected in Phase 4a — see its as-built.
|
||
UNIQUE KEY uq_engo_dedupe (rule_id, user_id, channel, dedupe_key),
|
||
INDEX idx_engo_due (status, due_at),
|
||
INDEX idx_engo_cancel (rule_id, user_id, subject_key, status)
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
```
|
||
|
||
Three things this buys that a straight send does not:
|
||
|
||
- **`due_at` is the 30-minute grace window.** The worker sweeps `status='scheduled' AND due_at <= now`.
|
||
- **`status='cancelled'` is the actual point of that window.** `idx_engo_cancel` is what a *resolving*
|
||
event queries: a `house.decay` back up to `LikeNew` cancels every scheduled row for that
|
||
(rule, user, house). Without cancellation, a delay is just a late mail.
|
||
- **`dedupe_key` UNIQUE per (rule, user, channel) makes replay safe.** The sidecar has no
|
||
schema-migration mechanism and a reconnect backfills; an at-least-once feed must not become an
|
||
at-least-once mailer. The scope matters as much as the constraint: a dedupe key names the *event*,
|
||
and the event fans out to every recipient of every channel of every matching rule.
|
||
|
||
`channel` is `VARCHAR(32)` and not an `ENUM` for exactly the reason `announce_job_legs.leg` is —
|
||
the channel set is data, and a module (or a later core channel) must not require an `ALTER`.
|
||
|
||
**(b) Digest ⇒ no queue.** Keep `teamDigestWorker`'s compute-at-send-time design and generalize its
|
||
state, not its absence of one:
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS engagement_digest_state (
|
||
user_id INT NOT NULL,
|
||
channel VARCHAR(32) NOT NULL,
|
||
scope_key VARCHAR(190) NOT NULL DEFAULT '', -- '' = deployment-wide; a Team id for the Teams case
|
||
last_digest_at DATETIME NULL,
|
||
PRIMARY KEY (user_id, channel, scope_key),
|
||
CONSTRAINT fk_engd_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
|
||
INDEX idx_engd_due (channel, last_digest_at)
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
```
|
||
|
||
`team_notification_prefs.last_digest_at` backfills into this with `channel='email'`,
|
||
`scope_key = CONCAT('team:', team_id)`. The three properties from `teamDigestWorker`'s header comment must
|
||
be preserved verbatim by the generic worker, and the third one (a user who lost access is no longer in the
|
||
recipient set) should get its own named test, the way the Teams phase-5 work gave the
|
||
leader-can't-see-reports rule its own test.
|
||
|
||
**Phase 6 as built, and it corrects §4.2a rather than only implementing this.** Keeping compute-at-send-time
|
||
means a digest-mode recipient must get **no outbox row at all**: Phase 4a's `subscribedTo` enqueued them
|
||
("what changes in Phase 6 is who drains it") and what changed in Phase 6 is that nothing drains it. An
|
||
outbox row holds a payload snapshotted at emit time and therefore has none of the three properties above —
|
||
including the security one. The engine now enqueues `instant` only. Two more things landed with it:
|
||
`engagement_outbox` gained a **`scope_key`** column (what a preference and an unsubscribe are keyed on,
|
||
which is not `subject_key` — see Phase 6's as-built), and the backfill's replay-safety is a property of the
|
||
PRIMARY KEY rather than of a flag, so a window the worker has since moved forward is never dragged
|
||
backwards by a restart.
|
||
|
||
### 4.3 The template variable contract — a code-declared schema, mirrored to a checked-in manifest
|
||
|
||
The brief asks whether anything in the codebase already does this. **Two things do, and they are the two
|
||
halves of the right answer:**
|
||
|
||
- **`blocks/registry.js`** — a registered definition carries `version` (a *prop-schema* version, bumped
|
||
when props change so a migration can transform older blocks), `schema: (props) => [errors]`, and
|
||
`sanitize: (props) => props` run on save *after* validation. That is the validation half.
|
||
- **`server/scripts/routeManifest.js` + `routes.manifest.json`, checked in CI with `--check`** — a
|
||
generated artifact committed to the repo, whose diff is the review signal. That is the drift half.
|
||
(`module-uo` carries its own `routes.manifest.json` for the same reason, and ships a prebuilt
|
||
`swagger-fragment.json` because core never has its sources to analyse — `MODULE_API.md` §6.1a.)
|
||
|
||
**Proposal: the trigger declaration carries its variables, and a generated manifest freezes them.**
|
||
|
||
```js
|
||
api.registerEventTriggers([{
|
||
id: 'uo.house.idoc_warning', // <owner>.-prefixed, checked by the existing namespaced()
|
||
label: 'House approaching collapse',
|
||
description: 'A player house dropped into a late decay stage.',
|
||
kind: 'event', // 'event' | 'scheduled' (G25); default 'event'
|
||
subjectKey: 'house', // which variable identifies the subject, for cooldowns
|
||
audience: 'owner', // the DEFAULT a rule is created with
|
||
ceiling: 'owner', // the widest a rule may EVER be given (G24, §5.1a)
|
||
version: 1, // bumped on a rename or a type change
|
||
variables: [
|
||
{ name: 'character', type: 'string', required: true, example: 'Darrow' },
|
||
{ name: 'house', type: 'string', required: true, example: 'The Silver Anvil' },
|
||
{ name: 'location', type: 'string', required: true, example: 'Britain, Trammel (1119, 1794)' },
|
||
{ name: 'decayStatus', type: 'string', required: true, example: 'Greatly' },
|
||
{ name: 'nextStage', type: 'datetime', required: false, example: '2026-08-30T04:00:00Z' },
|
||
{ name: 'estimatedCollapse',type: 'datetime', required: false, example: '2026-09-01T04:00:00Z' },
|
||
],
|
||
}])
|
||
```
|
||
|
||
Four properties, each with a reason:
|
||
|
||
1. **Validated at emit, not at render.** `ctx.events.emit` checks the payload against the declaration.
|
||
A missing `required` variable or a wrong type is **dropped and logged in production, thrown in
|
||
development** — the same posture `ctx.teams.activity.push` takes ("a malformed item is dropped and
|
||
logged"), because this is called from inside a game-event handler and a storage problem of core's must
|
||
not become the module's control flow.
|
||
2. **The editor reads it, so autocomplete is real.** `GET /admin/engagement/triggers` serves the
|
||
declarations; the template editor offers exactly those names and refuses to save a template
|
||
referencing one that is not declared. That is G12 closed — the editor never blindly interpolates
|
||
module JSON.
|
||
3. **`example` is not decoration — it is the preview and the test-send.** Without it, previewing a
|
||
template requires a live game event, which is the reason template systems go untested.
|
||
4. **Drift is caught by a committed manifest.** `npm run engagement:manifest` writes
|
||
`server/engagement-triggers.json` (core's) and CI runs it with `--check`, exactly as
|
||
`routes:manifest -- --check` already gates the URL surface. Changing a variable's name or type
|
||
without regenerating is a red build; the diff is what a reviewer reads. **A module ships its own
|
||
prebuilt `engagement-triggers.json` in its bundle**, for the same reason it ships a prebuilt
|
||
swagger fragment: core never has its sources.
|
||
|
||
**Two corrections from building it (Phase 2).** `audience` and `ceiling` are two fields, not one: the
|
||
first is the default a rule is created with, the second is the maximum it may be raised to, and the
|
||
registry refuses a default that the ceiling does not permit. And `'computed'` is gone from the
|
||
`audience` vocabulary — a rule pointing at a composed segment says so by naming the segment, so a
|
||
third pseudo-value that means "look elsewhere" would be a value the ceiling arithmetic cannot compare.
|
||
Both fields take the same six-value vocabulary (§5.1a).
|
||
|
||
**`example` is enforced, not encouraged.** A variable without one is refused at registration. Property
|
||
3 above is right and a soft version of it is worth nothing: the moment one variable has no example,
|
||
previewing that template needs a live game event again.
|
||
|
||
**Versioning.** A variable's *addition* is additive and needs nothing. A **rename or a type change** breaks
|
||
every stored template referencing it, so a trigger declaration carries `version`, bumped like a block's
|
||
prop-schema version, and templates store the trigger version they were authored against. A template
|
||
pinned to an older version renders with a warning in the admin list rather than silently interpolating
|
||
`undefined`.
|
||
|
||
### 4.4 Templates — reuse the block registry, do not build a second editor
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS engagement_templates (
|
||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||
`key` VARCHAR(96) NOT NULL UNIQUE, -- stable id a rule points at
|
||
name VARCHAR(160) NOT NULL,
|
||
trigger_id VARCHAR(96) NULL, -- NULL = a reusable/shared template
|
||
trigger_version INT NULL, -- what its variables were authored against (§4.3)
|
||
channel VARCHAR(32) NOT NULL, -- one template per channel; a rule names a set
|
||
subject VARCHAR(300) NULL, -- email only; may interpolate
|
||
blocks MEDIUMTEXT NOT NULL, -- JSON array — the pages.blocks pattern
|
||
text_body MEDIUMTEXT NULL, -- authored plain-text override; else generated
|
||
status ENUM('draft','published') NOT NULL DEFAULT 'draft',
|
||
-- A seeded template that the system itself depends on (password reset, invite).
|
||
-- Editable, NOT deletable — the pages.protected flag, for the same reason.
|
||
protected TINYINT(1) NOT NULL DEFAULT 0,
|
||
-- Which seed revision this row came from, and whether an operator has since
|
||
-- touched it. Together they let a later release ship an improved default
|
||
-- WITHOUT overwriting an operator's edits. See §4.6.
|
||
seed_key VARCHAR(96) NULL,
|
||
seed_version INT NULL,
|
||
customized TINYINT(1) NOT NULL DEFAULT 0,
|
||
updated_by INT NULL,
|
||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||
CONSTRAINT fk_engt_user FOREIGN KEY (updated_by) REFERENCES users(id) ON DELETE SET NULL,
|
||
INDEX idx_engt_trigger (trigger_id, channel, status)
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
```
|
||
|
||
**Why blocks and not raw HTML.** The brief asks for a visual editor taking inspiration from the CMS
|
||
page builder and the hero editor. The CMS builder is *already* a registry-driven block system with
|
||
server-side prop validation and sanitize-on-save (`blocks/registry.js`, `blocks/types/*`,
|
||
`sanitizeBlocks.js`). Storing raw operator HTML would give up all of that and hand the renderer an
|
||
injection surface.
|
||
|
||
**But email needs its own block set, not the page one.** Page blocks emit modern CSS that mail clients
|
||
do not support. Register a parallel family — `email.heading`, `email.text`, `email.button`,
|
||
`email.divider`, `email.image`, `email.itemList` — that render to table-based, inline-styled HTML. The
|
||
registry is designed for exactly this: "adding a block later means adding ONE entry".
|
||
|
||
**Plain-text fallback is generated by default, overridable per template.** Every block type gets a
|
||
`toText(props)` alongside its renderer, so a text part always exists. `teamNotify.excerpt()` is the
|
||
existing markup-to-text helper and should move into that family rather than being duplicated.
|
||
|
||
**A `text_body` that is empty for a published template is a save-time error, not a runtime one** — a
|
||
mail with no text part is a spam-filter signal, and finding out at send time means finding out from a
|
||
deliverability report.
|
||
|
||
### 4.5 Rules, channel preferences, send log, suppression
|
||
|
||
```sql
|
||
-- What an operator actually configures: trigger → audience → template → timing.
|
||
CREATE TABLE IF NOT EXISTS engagement_rules (
|
||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||
trigger_id VARCHAR(96) NOT NULL,
|
||
name VARCHAR(160) NOT NULL,
|
||
enabled TINYINT(1) NOT NULL DEFAULT 0, -- OFF by default; an operator turns it on
|
||
audience VARCHAR(32) NOT NULL DEFAULT 'owner',
|
||
audience_segment_id INT NULL, -- a composed segment (§5.1a); NULL = the plain audience above
|
||
-- §7.1 Q3: the hard stop that makes operator-editable rules safe to choose over
|
||
-- code-registered ones. Counted in engagement_sends, enforced before the outbox
|
||
-- row is written, never overridable from the rule editor beyond this column.
|
||
max_sends_per_hour INT NOT NULL DEFAULT 100,
|
||
channels JSON NOT NULL, -- ['email','inapp'] — a rule may span channels
|
||
template_keys JSON NOT NULL, -- { email: 'idoc-warning', inapp: 'idoc-warning-short' }
|
||
conditions JSON NULL, -- declared-variable predicates, e.g. decayStatus in [Greatly, IDOC]
|
||
cooldown_seconds INT NOT NULL DEFAULT 0,
|
||
delay_seconds INT NOT NULL DEFAULT 0, -- the grace window (§4.2a)
|
||
cancel_on JSON NULL, -- trigger ids that cancel a pending row for the same subject
|
||
updated_by INT NULL,
|
||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||
CONSTRAINT fk_engr_user FOREIGN KEY (updated_by) REFERENCES users(id) ON DELETE SET NULL,
|
||
INDEX idx_engr_trigger (trigger_id, enabled)
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
|
||
-- §5.1a: an operator-composed segment over module-declared audiences. Stored as a
|
||
-- boolean tree of audience ids + params; `ceiling` is DERIVED at save time as the
|
||
-- NARROWEST ceiling in the tree and re-checked against the trigger's own ceiling,
|
||
-- so composition can never widen. It is a column rather than a runtime computation
|
||
-- so an audit can read what a rule was allowed to reach without re-resolving it.
|
||
CREATE TABLE IF NOT EXISTS engagement_audience_segments (
|
||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||
name VARCHAR(160) NOT NULL,
|
||
expression JSON NOT NULL, -- { op: 'and'|'or'|'not', nodes: [...] | { audienceId, params } }
|
||
ceiling VARCHAR(32) NOT NULL, -- derived, never operator-typed
|
||
updated_by INT NULL,
|
||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||
CONSTRAINT fk_engseg_user FOREIGN KEY (updated_by) REFERENCES users(id) ON DELETE SET NULL
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
|
||
-- G8/G9: the channel dimension notification_subscriptions lacks.
|
||
CREATE TABLE IF NOT EXISTS notification_channel_prefs (
|
||
user_id INT NOT NULL,
|
||
stream_id VARCHAR(64) NOT NULL, -- a stream OR a trigger id; one namespace, see §7.2
|
||
channel VARCHAR(32) NOT NULL,
|
||
mode ENUM('off','instant','digest') NOT NULL DEFAULT 'off',
|
||
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||
PRIMARY KEY (user_id, stream_id, channel),
|
||
CONSTRAINT fk_ncp_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
|
||
INDEX idx_ncp_channel (channel, mode)
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
|
||
-- G15: per-message record. Today "did user X get the mail?" is unanswerable.
|
||
CREATE TABLE IF NOT EXISTS engagement_sends (
|
||
id BIGINT AUTO_INCREMENT PRIMARY KEY,
|
||
outbox_id BIGINT NULL,
|
||
rule_id INT NULL,
|
||
trigger_id VARCHAR(96) NOT NULL,
|
||
user_id INT NULL, -- SET NULL, so the log survives an account deletion
|
||
channel VARCHAR(32) NOT NULL,
|
||
transport VARCHAR(32) NULL, -- which mail transport actually carried it
|
||
address_hash CHAR(64) NULL, -- sha256; the log must not be a second address book
|
||
status ENUM('sent','failed','suppressed','bounced','complained') NOT NULL,
|
||
detail VARCHAR(500) NULL,
|
||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||
CONSTRAINT fk_engs_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE SET NULL,
|
||
INDEX idx_engs_trigger (trigger_id, created_at),
|
||
INDEX idx_engs_user (user_id, created_at)
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
|
||
-- G16. Keyed on the ADDRESS, not the user. That was written when users.email was
|
||
-- not unique; after Phase 1b it stays, for a better reason: a bounce arrives as
|
||
-- an ADDRESS, does not know which account was behind it, and stays true after
|
||
-- that account changed its address or was deleted.
|
||
--
|
||
-- **`address_masked` and `created_by` are Phase 9's additions to this DDL.** The
|
||
-- hash-only table cannot be operated: an operator reading sha256 digests cannot
|
||
-- tell three typos from a whole domain refusing mail, and un-suppressing somebody
|
||
-- who fixed their mailbox is the one action the table must support. The domain
|
||
-- survives so a domain-wide failure is visible; the local part is DESTROYED
|
||
-- rather than shortened, so the column can never be read back as an address book.
|
||
-- `created_by` is what separates a row an admin typed from one the outbox worker
|
||
-- wrote (NULL).
|
||
CREATE TABLE IF NOT EXISTS engagement_suppressions (
|
||
address_hash CHAR(64) NOT NULL PRIMARY KEY, -- sha256 of the lowercased address
|
||
address_masked VARCHAR(190) NULL, -- d***@example.com; never the local part
|
||
channel VARCHAR(32) NOT NULL DEFAULT 'email',
|
||
reason ENUM('bounce','complaint','manual','unverified') NOT NULL,
|
||
detail VARCHAR(500) NULL,
|
||
created_by INT NULL, -- the admin, for a manual row; NULL if automatic
|
||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||
CONSTRAINT fk_engsup_user FOREIGN KEY (created_by) REFERENCES users(id) ON DELETE SET NULL,
|
||
INDEX idx_engsup_created (created_at),
|
||
INDEX idx_engsup_reason (reason, created_at)
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
|
||
-- G17: the in-app inbox. Core, game-agnostic, content-carrying.
|
||
CREATE TABLE IF NOT EXISTS user_notifications (
|
||
id BIGINT AUTO_INCREMENT PRIMARY KEY,
|
||
user_id INT NOT NULL,
|
||
trigger_id VARCHAR(96) NOT NULL,
|
||
title VARCHAR(300) NOT NULL,
|
||
body TEXT NULL, -- rendered by the inapp template, sanitized on write
|
||
url VARCHAR(500) NULL, -- relative only, validated like pageUrlTemplate
|
||
dedupe_key VARCHAR(190) NULL,
|
||
read_at DATETIME NULL,
|
||
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||
CONSTRAINT fk_un_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
|
||
UNIQUE KEY uq_un_dedupe (user_id, dedupe_key),
|
||
INDEX idx_un_unread (user_id, read_at, created_at)
|
||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
|
||
```
|
||
|
||
**`email_config` grows rather than being replaced** (additive-only discipline):
|
||
|
||
```sql
|
||
ALTER TABLE email_config ADD COLUMN IF NOT EXISTS transport VARCHAR(32) NOT NULL DEFAULT 'smtp';
|
||
ALTER TABLE email_config ADD COLUMN IF NOT EXISTS credential_enc TEXT NULL; -- AES-GCM JSON blob, transport-shaped
|
||
ALTER TABLE email_config ADD COLUMN IF NOT EXISTS reply_to VARCHAR(255) NULL;
|
||
```
|
||
|
||
`transport` defaults to `'smtp'` because Gmail OAuth2 is gone (§1.2a) — there is no longer a transport
|
||
for an existing `provider='gmail_oauth2'` row to backfill *into*, so the upgrade lands every deployment
|
||
on SMTP with empty credentials and an explicit admin warning rather than on a transport that no longer
|
||
exists. `provider` and `refresh_token_enc` stay as dead columns (additive-only) and stop being read.
|
||
|
||
A single opaque `credential_enc` JSON blob is better than one column per provider field: SMTP, SES and
|
||
Mailgun have disjoint credential shapes and the transport's `credentialFields` already describes its own.
|
||
It also means adding Mailgun later is a registration plus an admin form, with **no schema change at all**.
|
||
|
||
**The backfill of `notification_subscriptions` → `notification_channel_prefs`** follows the precedent
|
||
already in `schema.sql:770` (the `announce_jobs` → `announce_job_legs` migration): an `INSERT IGNORE …
|
||
SELECT` guarded so replay on every boot is a no-op after the first.
|
||
|
||
```sql
|
||
INSERT IGNORE INTO notification_channel_prefs (user_id, stream_id, channel, mode)
|
||
SELECT user_id, stream_id, 'push', 'instant' FROM notification_subscriptions;
|
||
```
|
||
|
||
### 4.6 Basic templates and the editor *(decision 5)*
|
||
|
||
Two halves, and the first is the one that decides whether the second gets used.
|
||
|
||
#### 4.6.1 The seeded set — a fresh deployment mails correctly before anyone opens the editor
|
||
|
||
Today every message body is a template literal inside `mailer.js`. Phase 5 moves them into
|
||
`engagement_templates` **as seeded rows**, so the migration is a relocation rather than a regression:
|
||
nothing that sends mail today starts depending on an operator authoring something first.
|
||
|
||
Seeded on boot by an idempotent seeder alongside `db/seed.js`'s admin seed — `INSERT … ON DUPLICATE KEY
|
||
UPDATE` keyed on `seed_key`, **and it refuses to overwrite a row whose `customized` flag is set**.
|
||
|
||
**Transactional (`protected = 1`, editable but not deletable — the system breaks without them):**
|
||
|
||
| `seed_key` | Replaces | Variables |
|
||
| --- | --- | --- |
|
||
| `auth.password-reset` | `mailer.sendPasswordReset` | `username`, `resetUrl`, `expiresIn`, `siteName` |
|
||
| `auth.invite` | `mailer.sendInvite` | `acceptUrl`, `role`, `invitedByName`, `siteName` |
|
||
| `auth.email-verify` | *(new — Phase 9)* | `username`, `verifyUrl`, `expiresIn` |
|
||
| `admin.contact-message` | `mailer.sendContactMessage` | `fromName`, `fromEmail`, `message` |
|
||
| `admin.test` | `mailer.sendTest` | `siteName`, `transport`, `sentAt` |
|
||
|
||
**Notification (`protected = 0`, replaceable):**
|
||
|
||
| `seed_key` | Replaces | Variables |
|
||
| --- | --- | --- |
|
||
| `notify.event` | the generic single-event mail | `title`, `intro`, `items[]`, `actionUrl`, `unsubscribeUrl` |
|
||
| `notify.digest` | `teamDigestWorker`'s body | `intro`, `periodLabel`, `items[]`, `moreCount`, `scopeUrl`, `unsubscribeUrl` |
|
||
| `notify.team-post` | `mailer.sendTeamNotification` immediate | `teamName`, `authorName`, `threadTitle`, `excerpt`, `postUrl` — **`postUrl`, not `threadUrl`**: it has to be the name `team.forum.post` DECLARES, or the mail links nowhere. Renamed in Phase 6 (`seedVersion` 2) |
|
||
| `inapp.event` | *(new)* — the in-app channel's short form | `title`, `body`, `url` |
|
||
|
||
**Three properties of the seeded set that are design, not packaging:**
|
||
|
||
1. **`notify.event` and `notify.digest` are generic on purpose.** A new trigger — from core or from any
|
||
module — renders through them with no authoring at all, because their variables are structural
|
||
(`title`, `intro`, `items[]`) rather than domain-specific. An operator who wants a bespoke IDOC mail
|
||
writes one; an operator who does not still gets a sane one. **This is what stops "add a trigger"
|
||
from meaning "and now write a template."**
|
||
2. **They are branded from data, not hardcoded.** `BRAND_*` env and the `theme_visual` / `brand_assets`
|
||
settings already drive the site's colours and logo (`THEMING_AND_NAV.md` §4.4); the seeded templates
|
||
read the same resolved values, so one prebuilt image running as any shard mails in that shard's
|
||
colours. No template contains a literal hex code or a logo URL.
|
||
3. **A later release can improve a default without stealing an operator's work.** `seed_version` +
|
||
`customized` is the whole mechanism: on boot, a seed whose version is newer updates rows where
|
||
`customized = 0` and **skips** rows where it is 1, surfacing "an updated default is available" in the
|
||
admin list instead. Same posture `settingsJson` takes — a stored value that is unusable is treated as
|
||
absent, never as an error.
|
||
|
||
#### 4.6.2 The editor
|
||
|
||
Built on the existing block machinery (`blocks/registry.js`, the prop panels, `sanitizeBlocks.js`),
|
||
with an `email.*` block family (§4.4) — **not a second editor**. What it adds over the page builder:
|
||
|
||
- **A variable palette from the trigger declaration.** The right-hand panel lists exactly the variables
|
||
§4.3 declares for this template's trigger, with type and example. Inserting one writes a token; it
|
||
is never free-text. A template referencing an undeclared variable is **refused at save, naming the
|
||
variable** — the editor validates, it does not blindly interpolate module JSON.
|
||
- **Live preview from `example` values.** No live game event needed. This is the reason `example` is a
|
||
required part of the trigger declaration rather than documentation.
|
||
- **A side-by-side HTML / plain-text view.** The text part is generated from each block's `toText`, and
|
||
is overridable per template. A published template with an empty text part is a **save-time error**.
|
||
- **Test send to an address of the admin's choosing**, through the configured transport, recorded in
|
||
`engagement_sends` like any other message.
|
||
- **Three preview widths** (desktop / mobile / plain-text) and a dark-mode preview, because mail clients
|
||
invert backgrounds and a light-only template renders as unreadable dark-on-dark in about a third of
|
||
inboxes.
|
||
- **A duplicate action**, which is how an operator customizes a `protected` template safely: duplicate,
|
||
edit, point the rule at the copy, leave the original intact.
|
||
|
||
**Security posture, stated because this is the one new place operator HTML reaches a rendered surface:**
|
||
blocks are validated and sanitized on **write** (the existing `validateBlocks` → `sanitize` order), the
|
||
preview renders in a sandboxed iframe with no `allow-scripts`, and variable interpolation is
|
||
**HTML-escaped by default** with no raw-HTML variable type in v1. A module supplies data; it does not
|
||
supply markup.
|
||
|
||
---
|
||
|
||
## Part 5 — The module registration mechanism
|
||
|
||
### 5.1 Three additions to the contract, all modelled on what already works
|
||
|
||
```js
|
||
// api — what the module registers (MODULE_API.md §2.4). Modelled on
|
||
// registerNotificationStreams: shape-checked at the call, collision-checked at
|
||
// apply(), <owner>.-prefixed by the existing namespaced() helper.
|
||
api.registerEventTriggers([{ id, label, description, variables, subjectKey, audience, version }])
|
||
|
||
// ctx — what core hands the module (§2.3). Modelled on ctx.teams.activity.push:
|
||
// fire-and-forget, never throws, never rejects, malformed input dropped and logged.
|
||
ctx.events.emit(triggerId, { subject, data, ownerUserId?, dedupeKey?, occurredAt? })
|
||
|
||
// ctx — the in-app sink, for a module that wants to write the inbox directly
|
||
// without a rule. Optional; most modules will only emit.
|
||
ctx.inbox.push(userId, { triggerId, title, body, url, dedupeKey })
|
||
|
||
// api — the audiences a module can resolve over its own data (decision 8).
|
||
// Same registration discipline as the triggers above; see §5.1a.
|
||
api.registerAudiences([{ id, label, description, ceiling, resolve }])
|
||
```
|
||
|
||
**Why a new surface rather than extending `registerNotificationStreams`.** A stream entry is a
|
||
*subscription toggle* — label plus two booleans, with no statement about payload. A trigger is a *data
|
||
contract*. Overloading the stream entry with a `variables` array would make every existing push stream
|
||
look like it has an (empty) payload contract, and would put the emit path for content-free tickles and
|
||
content-carrying events through one function whose behaviour depends on which fields the caller filled
|
||
in. The two should stay separate for the same reason `registerPostHook` was kept out of
|
||
`registerAnnounceLeg` ("a leg is a one-shot DELIVERY with retry and classification; a post hook maintains
|
||
idempotent STATE" — `registries.js`).
|
||
|
||
**What core reuses verbatim:** `stage()` / `apply()`'s validate-then-commit-per-registrant discipline,
|
||
`namespaced()` for the `<owner>.` prefix, the collision message that names the current holder, and the
|
||
rule that nothing a registrant claims takes effect until the whole registrant is known good.
|
||
|
||
**Core registers its own triggers through the same door**, in `registerCore()`, exactly as it does for
|
||
streams and the Discord leg. That is not ceremony — `registries.js`'s header states the reason: "a registry
|
||
only core's hardcoded base bypasses is a registry whose first real exercise is a module, which is the drift
|
||
this PR exists to prevent."
|
||
|
||
### 5.1a Audiences — module-declared, operator-composable *(decision 8)*
|
||
|
||
The org lead's correction to Q7 is precise and worth stating exactly: **there is no campaign surface, but
|
||
lists exist — powered by game data, through the module, on a surface every module shares.** "Team X's
|
||
members" and "the governors" are legitimate audiences; "everyone who opened the last mail" is not, and
|
||
nothing here builds it.
|
||
|
||
```js
|
||
api.registerAudiences([{
|
||
id: 'team.members', // namespaced() prefixes it → 'uo.team.members'
|
||
label: 'Members of a team',
|
||
params: [{ id: 'teamId', type: 'int', required: true }],
|
||
ceiling: 'members', // the widest this audience can EVER resolve to (G24)
|
||
resolve: async (params, ctx) => [/* user ids */],
|
||
}])
|
||
```
|
||
|
||
**Four rules, each of which exists because of something already in the tree:**
|
||
|
||
1. **Core learns no game vocabulary.** Core never knows what a governor is; it knows an id, a label and
|
||
a `resolve` it may call. This is the same boundary `registerNotificationStreams` holds, and
|
||
`check:modules` already proves core's own ids name no game concept.
|
||
2. **The resolver returns user ids and nothing else.** It is not handed a template, a channel or an
|
||
address, and it cannot enumerate them — a module still cannot send mail (§1.2), and this must not
|
||
become the back door that lets it. The engine maps ids to addresses on core's side, after
|
||
preferences, suppression and the verification gate.
|
||
3. **A composed segment is bounded by the *narrowest* ceiling it contains, not the widest.** Operators
|
||
may combine declared audiences with and/or/not into a saved segment. That is real power and it is the
|
||
part with a security edge: composition must never *widen*. `A OR B` takes the tighter of the two
|
||
ceilings, and the result is still checked against the trigger's own G24 ceiling before a rule using it
|
||
can be saved. Union-widens is the intuitive implementation and it is the wrong one.
|
||
4. **An audience whose module is uninstalled goes dormant, exactly as a rule does** (§7.3). It resolves
|
||
to the empty set and the rule referring to it shows as dormant — never an error, never auto-deleted,
|
||
never a silent send to a *different* set of people because the id stopped resolving.
|
||
|
||
**The lattice itself — settled in Phase 2, because this document named it everywhere and defined it
|
||
nowhere.** "Narrowest" needs an ordering, and the obvious one is wrong:
|
||
|
||
```
|
||
everyone anyone at all, signed in or not
|
||
└── authenticated any signed-in user
|
||
├── subscribers signed-in users who opted into this id
|
||
├── members a module-declared list (a Team, the governors)
|
||
├── staff admin / editor / moderator
|
||
│ └── admin admins only ← added in Phase 11
|
||
└── owner the one user the event is about
|
||
```
|
||
|
||
**`admin` is the one node that was added after Phase 2 shipped the lattice, and the reason is worth
|
||
recording** (org lead, 2026-08-31, at the start of Phase 11). Phase 11's §8.6 table describes three
|
||
triggers as admin-audience — `uo.audit.staff_action`, `uo.economy.milestone`, `uo.world.saved` — and
|
||
`account.login.result` is classified by [`../link/v5.md`](../link/v5.md) §4 as *admin, by omission*,
|
||
"there is no rung below admin that a frame carrying an IP address and an auth verdict belongs on".
|
||
The lattice had no such rung: its narrowest staff-ish value was `staff`, which `ceilings.js` defines
|
||
as **admin / editor / moderator**, so ceilinging the operator-facing family there would have let a
|
||
rule mail the staff audit digest to every editor. It is the only true refinement in the tree — an
|
||
admin *is* staff, unlike any other pair of leaves — so it is a child rather than a seventh leaf, and
|
||
`permits`/`meet`/`meetAll` need no change at all beyond the new `PARENT` entry. It is a **contract
|
||
change** (a module may now declare `ceiling: 'admin'`) and takes `MODULE_API_VERSION` to **1.8.0**.
|
||
Cheap in Phase 11 because no row anywhere stores an `admin` audience yet; expensive after cutover.
|
||
|
||
**It is containment, not size.** The tempting model is a flat total order — `self < owner < staff <
|
||
members < authenticated < everyone`, compared with `<=` — and under it a trigger ceilinged at `staff`
|
||
also permits `owner`, so a rule could mail `uo.cheat.detected` to the player it detected. Fewer people
|
||
is not less exposure; the question is always *which* people.
|
||
|
||
So the four branches are **mutually incomparable**, deliberately: `owner` is not a subset of
|
||
`subscribers` (an owner need not have subscribed), `staff` is not a subset of `members`, and no pair
|
||
of them has a common descendant. `admin` is the single exception and it proves the rule — it is
|
||
narrower than `staff` because every admin is staff, which is exactly the containment the other pairs
|
||
lack. Three consequences:
|
||
|
||
- `permits(ceiling, candidate)` is "walk `candidate` up the tree and see whether you reach `ceiling`",
|
||
and it **fails closed** on anything it does not recognise.
|
||
- `meet(a, b)` — the narrower of two — exists only when one is an ancestor of the other. Two
|
||
incomparable ceilings have **no bound at all**, and the composition is REFUSED rather than resolved
|
||
to a guess. Union-widens is the intuitive implementation and it is the wrong one; picking a side
|
||
when there is no answer is the second-wrong one.
|
||
- The direction of the boolean operator is irrelevant. `A AND B` takes the tighter ceiling exactly as
|
||
`A OR B` does, because a ceiling states what an expression is *allowed* to reach, not what it will
|
||
resolve to. An empty composition has no bound and is `null`, never `everyone`.
|
||
|
||
The vocabulary travels with the trigger catalog (`GET /admin/engagement/triggers` serves it), so the
|
||
rule editor never offers an audience the server will refuse. The server is still the boundary: Phase 4
|
||
re-checks every rule save.
|
||
|
||
**Where it lands.** The registration surface and the ceiling arithmetic belong in **Phase 2**, with the
|
||
trigger declaration — G24's reasoning applies unchanged, and both are cheap now and expensive to retrofit
|
||
into a rule model that already has rows in it. The composition UI belongs with the rules screen in
|
||
**Phase 4**. `module-uo`'s first real audiences come in **Phase 11**.
|
||
|
||
**The composition UI, as built in Phase 4b.** Its own nav entry, **Admin → Engagement → Audiences**,
|
||
rather than a tab of the rules screen: a segment is reusable, independently edited, and deleted by a
|
||
refusal that has to say how many rules still point at it. The composer offers one top-level
|
||
all-of/any-of over audience leaves, each leaf rendering the `params` its declaration names (an `int`
|
||
param is sent as a number, because the server type-checks it). Three things it does that follow from
|
||
the rules above rather than from taste:
|
||
|
||
- **“Exclude” appears only under “all of”.** `not` is legal only as a child of `and`, so switching the
|
||
group to “any of” drops the exclusions rather than composing a tree the server will refuse — and
|
||
says so, because keeping them and failing at save would be worse than either.
|
||
- **The stored ceiling is displayed, never chosen.** The list shows what each saved audience *may*
|
||
reach beside what it *does* reach right now (on demand, as a count), which is the pair an operator
|
||
needs in order to understand a refusal.
|
||
- **A tree nested deeper than the composer renders is not editable, and says so** rather than being
|
||
flattened — the same posture the rule editor takes with a nested condition. The stored grammar
|
||
allows more nesting than any screen should offer.
|
||
|
||
The rule editor's half is the other side of the same rule: it offers only the audiences and saved
|
||
segments the trigger's ceiling **permits**, derived from the `permits` list `GET /admin/engagement/triggers`
|
||
serves. That is an affordance and not a boundary — the client must not hold a second copy of the
|
||
lattice — and the save re-checks every time.
|
||
|
||
### 5.2 The seam, end to end
|
||
|
||
```
|
||
module-uo core
|
||
───────── ────
|
||
register(ctx, api)
|
||
api.registerEventTriggers([...]) ─────────► registries: shape-check, namespace-check, collide-check
|
||
│
|
||
shard event arrives (uoLinkSocket) │ GET /admin/engagement/triggers ──► the rule + template editors
|
||
shardIngest → mapper │
|
||
ctx.events.emit('uo.house.idoc_warning', { │
|
||
subject: serial, │
|
||
data: { character, house, location, … }, │
|
||
ownerUserId: <resolved via shardLinks> ▼
|
||
}) ───────────────────────────────────────► engagement engine
|
||
1. validate payload against the declaration (§4.3)
|
||
2. find enabled rules for this trigger, eval conditions
|
||
3. resolve audience → user ids
|
||
4. per user: check notification_channel_prefs per channel
|
||
5. check engagement_cooldowns (rule, user, subject)
|
||
6. delay_seconds ? enqueue engagement_outbox : deliver now
|
||
7. cancel_on: cancel pending rows for the same subject
|
||
│
|
||
▼
|
||
render template (blocks → HTML + text)
|
||
│
|
||
▼
|
||
DeliveryChannel.deliver → transport → engagement_sends
|
||
```
|
||
|
||
**Owner resolution stays in the module.** `module-uo/server/utils/shardPush.js` already turns an
|
||
`ownerAcct` into a website user via `shardLinks` — core has no idea what a game account is and must not
|
||
learn. The module resolves and passes `ownerUserId`; core never sees `ownerAcct`.
|
||
|
||
### 5.3 What this costs the contract
|
||
|
||
`MODULE_API_VERSION` 1.6.0 → **1.7.0**. Additions only (`registerEventTriggers`, `registerAudiences`,
|
||
`ctx.events.emit`, `ctx.inbox.push`), no removal, no changed signature ⇒ minor by §1.1's table. `module-uo`'s
|
||
`coreApi: "^1.3.0"` still resolves, so no module is broken by the bump.
|
||
|
||
Knock-on obligations:
|
||
|
||
- `client/src/modules/version.js` carries the same number and a test asserts they agree.
|
||
- `integration-kit/ci/core-ref.json` pins a website `main` sha and `scripts/checkCoreApi.js` asserts
|
||
**equality** with `MODULE_API_VERSION`. **A bump turns the integration kit red on purpose** — that is
|
||
the mechanism, not a bug: someone must re-read the chapters and move the pin. Budget a kit PR.
|
||
- `docs/website/MODULE_API.md` §1.1, §2.3 and §2.4 need the new members, and §1.1's "1.6.0 has only ever
|
||
been on `edge`" paragraph needs correcting (§0.5).
|
||
|
||
---
|
||
|
||
## Part 6 — The phased plan
|
||
|
||
Same shape as the API v2 router-split plan: grouped, reviewable increments, one acceptance check per
|
||
phase, and an explicit note on which guardrails apply.
|
||
|
||
**Every phase carries the standing obligations** — `npm test --prefix server` green, `npm run swagger`
|
||
regenerated when a route changes, `npm run routes:manifest -- --check` clean, `npm run check:modules`
|
||
clean, **the documentation edits §6.0b assigns it**, Conventional Commits, the AI-disclosure trailer,
|
||
and a branch cut from a freshly-pulled base.
|
||
|
||
**Stage A (1–1b–2) is prerequisite. Stage B (3–6) is the engagement system. Stage C (7–8) is the in-app
|
||
channel. Stage D (9) is deliverability. Stage E (10–11) is the shard enrichment and runs in parallel
|
||
from day one. Stage F (12–13) is the public site and the cutover.**
|
||
|
||
### 6.0a The branching model — everything lands on `edge`, then one cutover to `main`
|
||
|
||
**Every phase PR in every repo targets `edge`. `main` is touched exactly once, by the cutover
|
||
(Phase 13).** This is the model the module system, protocol v3, Teams and the M12 theming workstream
|
||
each used, and it is the right one here for a specific reason: this workstream changes a wire protocol,
|
||
a module API version and the mail path simultaneously, and those three land in different repos on
|
||
different days. `main` must never hold a half-applied set of them.
|
||
|
||
**Two operational findings, both checked on 2026-08-28 and both blocking before Phase 1:**
|
||
|
||
1. **Every existing `edge` is stale.** `git rev-list --left-right --count origin/main...origin/edge`
|
||
says `edge` is **0 ahead** and behind `main` by: `docs` 16, `module-uo` 9, `installer` 7,
|
||
`servuo-plugins` 7, `website` 5, `link` 3. They are leftovers from previous cutovers that were never
|
||
refreshed after merging. **Fast-forward each `edge` to `main` before the first phase PR** — it is
|
||
lossless (0 ahead), and skipping it means the cutover diff carries stale content or conflicts that
|
||
have nothing to do with this workstream.
|
||
2. **Three repos have no `edge` at all** and need one cut from `main`: `android-app` (its M12 branch
|
||
was deleted after that cutover), `runicgateway.com`, and `Integration-kit`.
|
||
|
||
**Phase -1, executed 2026-08-28.** Five fast-forwards — `module-uo` 9 behind, `installer` 7,
|
||
`servuo-plugins` 7, `website` 5, `link` 3 — plus three branches cut from `main` (`android-app`,
|
||
`runicgateway.com`, `Integration-kit`). `docs`' `edge` was fast-forwarded earlier and is ahead. Every
|
||
fast-forward was 0 ahead, so all were lossless. **Phase -1 is complete.**
|
||
|
||
**One trap worth recording, because it produced a wrong answer here first.** `git fetch origin` does
|
||
**not** prune, so a `refs/remotes/origin/edge` left over from a branch that was deleted server-side after
|
||
a previous cutover still resolves. `git rev-parse --verify origin/edge` succeeds and
|
||
`git rev-list --left-right --count origin/main...origin/edge` returns a plausible count — for
|
||
`android-app` it reported "1 behind", which read exactly like a stale-but-present branch and is why this
|
||
section was briefly "corrected" to say two repos rather than three. The branch had not existed on the
|
||
server since the M12 cutover. **Use `git ls-remote --heads origin edge` (or fetch with `--prune`) to ask
|
||
whether a remote branch exists**; a remote-tracking ref is a cache, not an answer.
|
||
|
||
**Android CI does not run on `edge`.** `android-app/.gitea/workflows/pr-checks.yml` triggers only on
|
||
PRs into `main`, so every Phase 8 PR lands with **zero CI** and the cutover is the first real run. That
|
||
was true of all nine M12 phase PRs and it is true again here. Either fix the trigger as Phase 8's first
|
||
commit or budget for the cutover being the first honest build — decide deliberately rather than
|
||
discovering it.
|
||
|
||
**The cutover is per-repo but not independent.** Phase 13 names the order, because a `main` that has the
|
||
v5 sidecar and the v4 overlay is a shard that cannot pair.
|
||
|
||
### 6.0b Documentation is a phase deliverable, not an appendix
|
||
|
||
Every phase below owes specific documentation, and the phase is **not done until it lands in the same
|
||
PR** (or, for cross-repo docs, a companion PR in the same review window). CLAUDE.md's rule — "a code
|
||
change is not complete until `docs/` reflects it" — is the floor; this table is the assignment.
|
||
|
||
| Phase | `docs/` | Other repos |
|
||
| --- | --- | --- |
|
||
| **1** Remove Gmail OAuth2, SMTP | `website/BACKEND_DESIGN.md` §7 **rewritten** (not amended — it documents Gmail OAuth2 as *the* mechanism); route tables lose `/admin/email/connect/*` | `website/README.md` + `.env.example` wherever they point at Connect Gmail · **`runicgateway.com`**: `notifications-and-email.mdx` (its "There is no SMTP option" aside is now false), `configuration.mdx:62`, `troubleshooting.mdx:101`, `system-architecture.mdx:117` · a release note |
|
||
| **1a** One self surface | `website/BACKEND_DESIGN.md` — the `/auth/me` prose and the `/player`+`/admin` router trees · `api-route-inventory.json` regenerated · `website/ENGAGEMENT.md` this phase | `android/PLAN.md` §6.4/§8 — the "routes stay for web back-compat" note is now false · `website/API_V2_PLAN.md` historical tables get a pointer |
|
||
| **1b** Unique email | `website/BACKEND_DESIGN.md` — the `users` table (the "not unique" note is now false), the new change/verify routes, and the de-dupe migration as an operator-visible upgrade step | `website/README.md` upgrade notes · a release note naming the admin report and the verification-gate default |
|
||
| **2** Trigger registry | `website/MODULE_API.md` §1.1 (**1.7.0** + correct the stale "1.6.0 has only ever been on `edge`" paragraph), §2.3 (`ctx.events`, `ctx.inbox`), §2.4 (`registerEventTriggers`, `registerAudiences`), the dormant-rule note (landed as §6.8) · `website/ENGAGEMENT.md` §4.3 and §5.1a kept true · `BACKEND_DESIGN.md` route table | **Both deferred to the Phase 13 cutover window, deliberately — see Phase 2's as-built.** `Integration-kit`'s `ci/core-ref.json` pins a **`main`** sha, so the equality check stays green (and must stay green) for the whole `edge` period; `runicgateway.com`'s `checkFacts.mjs` *fetches* from `main`, so setting `platform.json.moduleApi` → 1.7.0 now would turn that repo red immediately |
|
||
| **3** Channel preferences | `website/BACKEND_DESIGN.md` route table · `android/PLAN.md` §11 | — |
|
||
| **4a** Engine | `website/ENGAGEMENT.md` (rules/cooldown/outbox as built, and the two §4 defects it corrects) · `BACKEND_DESIGN.md` table inventory | — |
|
||
| **4b** Rules screen ✅ | `website/BACKEND_DESIGN.md` route table (the twelve routes, incl. the `PATCH …/enabled` argument and the count-only preview) · `website/ENGAGEMENT.md` §5.1a composition UI | Landed with the phase (docs#184) |
|
||
| **5a** Templates ✅ | `website/ENGAGEMENT.md` §4.6 as built · `BACKEND_DESIGN.md` — the `engagement_templates` table, the two block registries, the token grammar, and §7's multipart/subject changes | Landed with the phase |
|
||
| **5b** The editor | `website/ENGAGEMENT.md` §4.6.2 as built · `BACKEND_DESIGN.md` route table | **`runicgateway.com`**: a new admin docs page for the template editor |
|
||
| **6** Email channel + Teams migration ✅ | `website/TEAMS.md` §6.3/§6.4 **rewritten** — the Team pipeline it describes no longer exists as its own thing · `website/ENGAGEMENT.md` §4.2b + this phase as built · `BACKEND_DESIGN.md` route table and table inventory | **`runicgateway.com`**: `administration/teams.mdx` notification section. Landed with the phase |
|
||
| **7** In-app channel (core+web) ✅ | `website/BACKEND_DESIGN.md` routes + tables (the four inbox routes, `user_notifications`) · `website/ENGAGEMENT.md` this phase as built | **`runicgateway.com`**: `notifications-and-email.mdx` gains the in-app channel. Landed with the phase |
|
||
| **8** In-app (Android) ✅ | `android/PLAN.md` §7 (the Room exception) + §11 (the inbox as built) · `website/ENGAGEMENT.md` this phase as built | `android-app/README.md`. Landed with the phase |
|
||
| **9** Deliverability | `website/BACKEND_DESIGN.md` §7 · a suppression/bounce operator section (the verification flow is Phase 1b's) | **`runicgateway.com`**: `troubleshooting.mdx` gains bounce/suppression · **`PLAY_DATA_SAFETY.md` + `/privacy`** — see Phase 12 |
|
||
| **10** Protocol bump | `link/INTEGRATION.md` §Housing **and §Market** (tables + examples) + the new `account.login.result` row · `link/PLAN.md` §5/§7 · **a `link/v5.md`** — the bump now carries three enrichments across three subsystems, which is what earned v3 and v4 their own design docs | `servuo-plugins/overlay.toml` · `servuo-plugins/patches/README.md` — `vendor.sale` stays in the tier and the docs must say what that costs · **`runicgateway.com`**: `platform.json.protocol` → 5, `bundle.*`, `architecture/protocol-versions.mdx` |
|
||
| **11a** module-uo triggers | `modules/uo/API.md` — **the full trigger catalogue, its audiences and its ceilings**, not one entry · `modules/uo/README.md` · `website/ENGAGEMENT.md` §8.6 kept true as rows ship · `website/MODULE_API.md` §1.1 (**1.8.0**) and the ceiling vocabulary wherever it is enumerated · `BACKEND_DESIGN.md` — the `news.post` publish path now runs through the engine | `module-uo/README.md` |
|
||
| **11b** Seeded rules + templates | `website/ENGAGEMENT.md` this phase as built · a release note naming **the seeded-disabled `news.post` rule as an upgrade step** (decision 5) — without it a deployment loses news push silently | **`runicgateway.com`**: `capabilities.mjs` and the notifications page — "one rule" and "the whole catalogue" are different marketing claims |
|
||
| **12** Public site | `website/ENGAGEMENT.md` this phase as built, incl. the 12a/12b split and Phase 13's fill-in step | **`runicgateway.com`**, in full — see the phase. **Two PRs**: 12a (#26) mergeable now, 12b (#27) a draft held for the cutover window |
|
||
| **13** Cutover ✅ | `README.md` index rows · every doc's status line · **this phase as built**, incl. the two steps the window missed | `.profile/README.md` — the org lead called this a headline capability, so it landed |
|
||
| **14** Retention | `website/ENGAGEMENT.md` this phase as built · `BACKEND_DESIGN.md` table inventory · the settings keys it adds | **`runicgateway.com`**: `/privacy` + **`PLAY_DATA_SAFETY.md`**, both generated from `src/data/collection.mjs`, whose `deploy-engagement` row says "nothing here expires on its own" and stops being true |
|
||
|
||
**One thing this table is protecting against.** `runicgateway.com` appears in eight rows, and it is the
|
||
only repo here whose checks are *fetching* these values rather than being told them — see Phase 12.
|
||
|
||
|
||
|
||
---
|
||
|
||
### Phase 0 — Design of record ✅
|
||
|
||
This document, landed as `docs/website/ENGAGEMENT.md` with the five settled decisions recorded at the
|
||
top. No code.
|
||
|
||
**Acceptance:** merged into `docs/`; §7.1's open questions each answered or explicitly deferred before
|
||
the phase that depends on them starts. **Q1, Q3, Q5 and Q7 were answered on 2026-08-28** (§7.1), which
|
||
unblocked Phases 1, 2 and 4 and added Phase 1b; **Q6 and §7.2 were answered on 2026-08-29**, at the
|
||
start of Phase 2 and before a line of it was written. Still outstanding: **Q2 before Phase 4**, **Q4
|
||
before Phase 5b**, **Q8 before Phase 8**.
|
||
|
||
This acceptance criterion has now paid for itself twice. Q6 and §7.2 were both tagged "decide in
|
||
Phase 2", and §7.2's answer went **against** the recommendation in the text — which is exactly the
|
||
outcome that is cheap to act on before the registry exists and expensive afterwards.
|
||
|
||
---
|
||
|
||
### Phase 1 — Remove Gmail OAuth2; the transport registry + SMTP ✅
|
||
|
||
**This phase is a subtraction and a replacement in one PR**, because leaving the OAuth2 flow half-wired
|
||
across a release is worse than either end state.
|
||
|
||
Delete everything in §1.2a's inventory. Extract `utils/mailer.js` behind the §3.1 interface with `smtp`
|
||
as the sole registered transport. Admin → Email becomes a credential form driven by `credentialFields`.
|
||
`email_config` gains `transport` / `credential_enc` / `reply_to`, defaulting to `smtp` with no
|
||
credentials. All six existing call sites keep their **exact** failure contracts — the contact form's
|
||
`mailto` fallback, the invite's copyable-link fallback, the password reset's generic 200, and
|
||
`sendTeamNotification`'s never-throws.
|
||
|
||
Ships with the cutover safety net §1.2a demands: an admin dashboard warning when the transport is
|
||
configured but credential-less, a release note naming the operator action, and the
|
||
`smtp.gmail.com:587` + app-password migration path documented as the shortest route for the existing
|
||
deployment.
|
||
|
||
**Acceptance:** `test/mailer.test.js` and `test/emailConfig.model.test.js` pass (amended only where they
|
||
assert OAuth2 specifics); a fresh install with SMTP configured sends every one of the five current
|
||
message types; an upgraded install with no SMTP credentials degrades exactly as an unconfigured
|
||
deployment does today — contact form falls back to `mailto`, invites surface the link, resets answer 200
|
||
— **and shows the warning**; `grep -r "smtp.gmail.com\|mail.google.com" server/src` returns nothing.
|
||
**Guardrails:** swagger regen + `routes:manifest --check` (two routes removed); no-hardcoded-host check
|
||
(§3.2 rule 4) — which the deleted `smtp.gmail.com` literal is the first real test of.
|
||
|
||
#### As built (website#165, docs#178)
|
||
|
||
Delivered as specified, with four things worth recording because they are not what the plan above says.
|
||
|
||
1. **Only half of §3.1 was built, deliberately.** `registerMailTransport` ships;
|
||
`registerDeliveryChannel` does not. Phase 1 has no consumer for `addressFor`/`render`/`deliver` —
|
||
the engine that calls them is Phase 4 — and a registered channel nothing calls is a shape frozen
|
||
before anything tried to use it. It arrives with the phase that consumes it. `credentialFields` is
|
||
the piece that mattered here, since it is what makes the admin form generic.
|
||
2. **`enabled` now gates every sender, which it did not before.** `buildTransport()` used to test only
|
||
"is there a refresh token and a sender", so the contact form kept sending after an admin unticked
|
||
*Enable email sending* — `isConfigured()` honoured the toggle but the five direct senders bypassed
|
||
it. The connect flow used to set `enabled` as a side effect of a consent redirect; with a credential
|
||
form it has to mean what it says, so the gate moved onto the one path every sender shares. **A
|
||
deliberate behaviour change, not a refactor**, and the only one in the phase.
|
||
3. **The G22 warning reads the deprecated column.** `refresh_token_enc` stays unread *as
|
||
configuration*, but its presence is exactly "this deployment had working mail before the upgrade",
|
||
which is the warning's whole condition. `hadLegacyConnection && !hasCredential` fires for the one
|
||
deployment this happens to and stays silent on a fresh install, which has never had mail and would
|
||
only learn to ignore the banner. The warning clears itself once a credential is saved — nothing has
|
||
to remember to dismiss it.
|
||
4. **`npm run swagger` was already broken and had to be fixed first** (website#164, its own PR). It
|
||
died with swagger-autogen's "invalid array length" runaway on a pristine `edge`, under Node 20 and
|
||
24, and at the commit whose own PR last regenerated the spec — so no phase that touches a route
|
||
could have met the standing obligation. Bisected to one statement in `teams.router.js`:
|
||
`param('teamId').custom((v) => v === 'default' || TEAM_ID.test(v))`. **The rule is that nothing
|
||
reaching `.test(` may sit inside a route statement**, and the "per-file route limit" that phases 8
|
||
and 9 recorded does not exist — a three-route file carrying only that one route dies too. Worth
|
||
knowing for every later phase in this workstream, all of which add routes.
|
||
|
||
Two smaller decisions: the credential is one encrypted JSON blob rather than a column per field, so
|
||
adding a transport is never a schema change; and a blob that will not decrypt reads as **absent**
|
||
rather than raising, so a rotated `SECRET_ENC_KEY` lands an admin on an "unconfigured" screen instead
|
||
of a 500 that takes the contact form with it.
|
||
|
||
---
|
||
|
||
### Phase 1a — One self-service surface: `/auth/me/account` ✅
|
||
|
||
**Not in the original plan.** It was added on 2026-08-29, ahead of Phase 1b, when mapping Phase 1b's
|
||
ground truth turned up that self-service account security had **three URL surfaces onto one
|
||
controller** — `/auth/me/account/*`, `/player/account/*` and `/admin/account/*`, all mounting the same
|
||
`account.controller` handlers. Phase 1b adds a self-service field (email), Phase 3 adds another
|
||
(channel preferences), and each would otherwise have been written three times.
|
||
|
||
**`/auth/me/account` was already a strict superset**, which is what settled which one to keep: it was
|
||
the only surface carrying recovery codes, and `/admin/account` carried no username or password change
|
||
at all. The web client already reached into `/auth/me` for two calls on a screen it otherwise served
|
||
from `/admin/account` — the split was leaking before anyone touched it. Gating was equivalent where it
|
||
overlapped: `/player` and `/auth/me` apply byte-identical `noindex, requireAuth`, and `staffOnly` on
|
||
`/admin/account` was strictly narrower while buying nothing, since every handler is self-scoped to
|
||
`req.user.id`.
|
||
|
||
**14 routes deleted, 0 added, no handler changed.** `account.controller.js` moved from
|
||
`router/v1/admin/` to `router/v1/auth/`, beside the one router that still reaches it. The web client's
|
||
14 call sites moved onto the root-level `api.myAccount` / `api.changeUsername` / … group. The Android
|
||
app needed **nothing** — `MeApi.kt` was already 100% `/auth/me/account/*`.
|
||
|
||
It is a **breaking change to the published OpenAPI surface**, accepted deliberately: both consumers are
|
||
in this org, and deprecating-then-deleting would have meant Phase 1b deciding whether to add its email
|
||
routes to surfaces already marked for removal.
|
||
|
||
**As built:** `routes.manifest.json` shows exactly 14 deletions and 0 additions; the OpenAPI spec loses
|
||
the same 14 paths with **zero surviving path definitions changed** and identical `components` (its large
|
||
textual diff is pure reordering — removing the first-mounted router shifts every later path). Two
|
||
swagger tags — `Admin · Account` and `Player` — were declared *only* by the deleted routes and went with
|
||
them. 1203 server tests and 288 client tests green.
|
||
|
||
**One thing this phase did not fix, on purpose:** `docs/website/api-route-inventory.json`, the docs
|
||
mirror of the route manifest, had drifted badly on its own (168 entries against the manifest's 203 —
|
||
missing every Teams route, and still listing the two Gmail `connect` routes Phase 1 deleted). It is a
|
||
generated mirror with no CI gate, so it was regenerated wholesale here rather than partially updated.
|
||
**A mirror nothing checks will drift again** — a gate for it belongs in a later phase.
|
||
|
||
---
|
||
|
||
### Phase 1b — Unique, changeable, verifiable email addresses ✅ *(decision 6)*
|
||
|
||
**Lands alone, between 1 and 2, and before any engagement mail exists.** It touches registration, SSO
|
||
provisioning and the boot-time schema path — three security-sensitive surfaces — and retrofitting
|
||
uniqueness *after* a send log and a suppression list hold rows is strictly worse than doing it now.
|
||
§0.6 is the finding this phase discharges.
|
||
|
||
**Four pieces, in this order within the PR:**
|
||
|
||
1. **Distinguish the constraint before adding one.** Replace `isDuplicateUsername()`'s bare
|
||
`ER_DUP_ENTRY`/`1062` test with a check that reads the violated index off the driver error, and give
|
||
its two callers (`auth.controller.js:137`, `sso.controller.js:197`) separate branches. **This must be
|
||
in the tree before the index is**, or the register path starts lying and SSO sign-up starts failing
|
||
opaquely the moment the ALTER runs.
|
||
2. **De-duplicate, then index.** A migration step that runs *before* the `ALTER`: for each duplicated
|
||
address, the **earliest-created** account keeps it; every later duplicate has `email` set to `NULL`
|
||
and `email_verified` to `0`. Multiple `NULL`s are legal under a UNIQUE index, so nobody loses an
|
||
account and nothing cascades. The affected accounts are written to an admin-visible report — *who*
|
||
was cleared and *what* address they lost — because they are exactly the users who must be contacted.
|
||
Then a UNIQUE index — **on a generated `email_norm` column, not on `email`.** The reasoning above
|
||
(`Foo@x.com` and `foo@x.com` are one mailbox; folding belongs in the index rather than in bypassable
|
||
application code) is right, but the collation this originally named is not: see the amendment below.
|
||
3. **A self-serve change-and-verify flow**, which does not exist today (§0.6 finding 4). Set/change
|
||
address, a signed time-boxed verification link, `email_verified` set only on link use. It lands on
|
||
**`/auth/me/account` and nowhere else** — Phase 1a made that the single self-service surface. SSO's
|
||
`emailVerified: Boolean(profile.email)` is corrected at the same time to honour the IdP's actual
|
||
`email_verified` claim rather than the mere presence of an address.
|
||
4. **The verification gate as an admin setting** — `on` for fresh installs, `off` for upgrades, so the
|
||
live deployment does not silently stop mailing its existing opted-in users on the day it upgrades.
|
||
The asymmetry is deliberate and is the same lesson as G22: a safe default must not be applied
|
||
retroactively to a running system without telling anyone.
|
||
|
||
**The error surface stays anti-enumeration.** A collision returns a generic failure, the real reason is
|
||
logged and not returned, the endpoint keeps its rate limit, and the failure is **not** scored by the bot
|
||
detector — a legitimate user typing a colleague's address must not be pushed toward an IP ban for it.
|
||
|
||
**Acceptance:** a DB seeded with three accounts sharing an address boots clean, keeps the oldest, nulls
|
||
two, and lists both in the admin report; registering with a taken address returns the generic failure,
|
||
logs the specific one, increments no bot score, and does **not** say "username"; SSO sign-up with an
|
||
address already held by another account fails with a distinguishable reason rather than exhausting
|
||
`PROVISION_MAX_TRIES`; a user can change their address and it stays `email_verified = 0` until the link
|
||
is used; `Foo@x.com` collides with `foo@x.com`; an upgraded install has the gate `off` and a fresh one
|
||
`on`.
|
||
**Guardrails:** swagger regen + `routes:manifest --check` (the change/verify routes are new); the
|
||
migration is idempotent and re-running `ensureSchema()` is a no-op; **no destructive DDL** — the
|
||
de-dupe nulls a column, it never deletes a row.
|
||
|
||
#### As built (2026-08-29)
|
||
|
||
**The index is on a generated column, because every `_ci` collation is also accent-insensitive.**
|
||
Step 2 above said to pin `email` to a case-insensitive collation "for the same reason `username` was".
|
||
Tested against the deployment's own MariaDB 11.8, that is wrong in a way that would have destroyed
|
||
data: under **both** `utf8mb4_general_ci` and the server-default `utf8mb4_uca1400_ai_ci`,
|
||
`josé@x.com` and `jose@x.com` compare EQUAL. They are different mailboxes. A UNIQUE index over either
|
||
collation refuses the second address forever, and the de-duplication below would have nulled a
|
||
legitimate account's address and reported it as a duplicate that never was.
|
||
|
||
The accent-sensitive, case-insensitive collations that would be exactly right
|
||
(`utf8mb4_general1400_as_ci`, `utf8mb4_0900_as_ci`) are MariaDB 11.4+ only, so pinning one moves the
|
||
"a UNIQUE email can stop a boot" failure of §0.6 to a different trigger. What shipped instead:
|
||
|
||
```sql
|
||
email VARCHAR(255) NULL,
|
||
email_norm VARCHAR(255) COLLATE utf8mb4_bin AS (LOWER(email)) STORED,
|
||
UNIQUE KEY uq_users_email_norm (email_norm)
|
||
```
|
||
|
||
`LOWER()` under a `_bin` collation folds case without folding accents — verified, not assumed. The
|
||
fold still lives in the schema rather than in bypassable application code, which was the point of the
|
||
original rule. Multiple NULLs remain legal, which is what lets the de-dupe clear an address without
|
||
deleting an account. No foreign key references `users.email`, so the STORED-generated-column trap from
|
||
TEAMS.md phase 2 (`ER_GENERATED_COLUMN_FUNCTION_IS_NOT_ALLOWED` on `ON DELETE SET NULL`) does not apply.
|
||
|
||
**The de-dupe must group on that same column, and the first version did not.** Written as
|
||
`LOWER(u2.email) = LOWER(u.email)`, the comparison uses the *column's* collation — accent-insensitive
|
||
— so it over-folds even when the index does not. A seeded fixture caught it nulling `jose@x.com` as a
|
||
"duplicate" of `josé@x.com`: the exact defect the index change was made to prevent, reintroduced one
|
||
statement later. The migration therefore **adds `email_norm` before de-duplicating and groups on it**,
|
||
so the two agree by construction rather than by a hand-matched `COLLATE` clause a later edit can get
|
||
wrong. Order in `schema.sql` is load-bearing and commented as such.
|
||
|
||
**Four decisions taken at build time**, all approved before any code:
|
||
|
||
| | Decision | Why |
|
||
|---|---|---|
|
||
| Index folding | generated `LOWER()` column + `_bin` index | above |
|
||
| Change flow | **pending column**, live address untouched | a typo cannot silently redirect account-recovery mail. Cost: a pending address reserves nothing, so two users may both be pending on one address and the second to confirm loses — with the same generic failure |
|
||
| Re-auth | `currentPassword` required, SSO carve-out | an address is where recovery lands, so repointing it is credential-grade; mirrors `changePassword` |
|
||
| Report surface | table + dashboard warning + read route | reuses the Phase 1 G22 shape: narrow, self-clearing, silent on installs it does not concern |
|
||
|
||
**One deviation from the text above, deliberate:** step 3 says a "signed" link. Every comparable flow
|
||
in this codebase (`user_invites`, `password_resets`, `mobile_refresh_tokens`) uses an opaque random
|
||
token with only its sha256 at rest, and `email_verifications` matches them rather than introducing a
|
||
second token mechanism for one caller.
|
||
|
||
**`provisionSsoPlayer` now returns `{ user }` or `{ error }`** instead of the user or a bare null. Two
|
||
ways to fail need two things said to the person at the browser; the two call sites map `error`
|
||
straight onto the `sso_error` code the login pages already render.
|
||
|
||
**Verified on a live rig**, not only in unit tests — a real MariaDB 11.8 seeded with the pre-upgrade
|
||
schema plus three accounts sharing an address, upgraded by booting the real server, with a real SMTP
|
||
send into a mail catcher (which also discharges Phase 1's outstanding "no live SMTP send"):
|
||
|
||
- the upgrade **boots clean**; oldest kept the address, two were nulled and reported with the exact
|
||
addresses they lost; `josé@` and `jose@` both survived
|
||
- the gate seeded **`off` on the upgrade** and `on` on a fresh install
|
||
- the dashboard warning fired with the right count and cleared on acknowledge
|
||
- a change request staged the address and **left the live one receiving mail**; the link went only to
|
||
the new address; opening it from a session-less client installed the address and **set no cookie**
|
||
- a replayed link, and a second account confirming an address the first had just taken, both returned
|
||
the **byte-identical** generic 404 — the real reason logged, never returned
|
||
- `NEWMAIL@RIG.TEST` was refused at registration as a duplicate of `newmail@rig.test`, while
|
||
`néwmail@rig.test` registered successfully beside it
|
||
|
||
**Left for later, deliberately:** nothing consumes `email_verification_required` yet — the engine that
|
||
would honour it is Phase 4 and the deliverability rules are Phase 9. It is seeded and editable now
|
||
because the fresh-vs-upgrade distinction is only knowable at the migration that adds it, and
|
||
reconstructing "was this install fresh?" afterwards is guesswork.
|
||
|
||
---
|
||
|
||
### Phase 2 — The trigger registry and the variable contract ✅
|
||
|
||
`api.registerEventTriggers` + `ctx.events.emit` in `modules/registries.js` and `modules/loader.js`;
|
||
`MODULE_API_VERSION` → 1.7.0 on both halves; core registers its own triggers (news, the four Team
|
||
events) through `registerCore()`. `npm run engagement:manifest` + the CI `--check`. **No delivery yet** —
|
||
emit validates, logs and stops.
|
||
|
||
A trigger declaration also carries its **audience ceiling** (G24) — the widest audience a rule may ever
|
||
give it — and its `kind` (`event` now, `scheduled` reserved for G25). Both are cheap here and expensive
|
||
to retrofit into the rule model later.
|
||
|
||
Phase 2 also lands `api.registerAudiences` and the ceiling arithmetic (§5.1a) — the same registration
|
||
discipline, and the same "cheap now, expensive later" argument G24 makes for the trigger ceiling.
|
||
|
||
**Acceptance:** core's triggers appear in `GET /admin/engagement/triggers`; a module registering an
|
||
un-namespaced trigger or audience fails to load with the holder named; **an `A OR B` composition takes
|
||
the narrower of the two ceilings, not the wider**; an audience whose module is uninstalled resolves
|
||
empty and shows dormant rather than erroring; a payload missing a `required` variable
|
||
throws in dev and is dropped+logged in prod; **a rule cannot be saved with an audience wider than its
|
||
trigger's ceiling**; `engagement-triggers.json` diffs zero in CI.
|
||
**Guardrails:** the new manifest `--check` (this is where the "manifest-style guardrail" the brief asks
|
||
about belongs); `check:modules` proves core's own trigger ids name no game concept.
|
||
|
||
#### As built (2026-08-29)
|
||
|
||
**Three design questions were settled by the org lead before any code**, and two of them the plan had
|
||
deferred to this phase on purpose:
|
||
|
||
| | Question | Decision |
|
||
|---|---|---|
|
||
| §7.2 | one namespace, or two? | **ONE.** A trigger is a payload contract attached to an id that may also carry a subscription toggle |
|
||
| §7.1 Q6 | time-based triggers | **declare now, build after Phase 9** — `kind: 'scheduled'` is in the contract, the manifest and every stored declaration from today; no evaluator yet |
|
||
| — | what "narrower" means for a ceiling | an explicit **subset lattice**; two incomparable ceilings have no bound and the save is REFUSED, never guessed |
|
||
|
||
**One namespace was the more invasive of the two options and it is worth stating what it bought and
|
||
what it cost.** It buys `notification_channel_prefs.stream_id` staying single-keyed (§4.5): under two
|
||
namespaces its primary key would have needed a `kind` discriminator, and `news.post` would have named
|
||
two different things forever. It costs a new rule in `registries.js` — **an id has exactly one owner
|
||
across both facets** — enforced in both directions, so a module cannot attach a payload contract to
|
||
another module's stream and cannot claim a stream id another module declared a trigger for. Core's
|
||
five trigger ids *are* its five stream ids, so the same-owner upgrade case is exercised on every boot
|
||
rather than only by a module.
|
||
|
||
Three consequences fell out of it that the §7.2 text did not anticipate:
|
||
|
||
- **The id grammar had to be relaxed, not just shared.** `STREAM_ID` did not admit `_`, and §4.3's own
|
||
worked example is `uo.house.idoc_warning`. Two grammars over one namespace would mean an id that is
|
||
legal as a trigger and illegal as the stream it is the same event as. It is a relaxation only —
|
||
every id valid before is valid now, and no stored id changes.
|
||
- **The legacy allowlist had to be shared too.** The seven grandfathered `uo.*` stream ids are exempt
|
||
from the prefix rule for triggers as well, and it cannot be otherwise: under one namespace
|
||
`idoc.warning` is a single id, so if `uo` may hold it unprefixed as a stream it may hold it
|
||
unprefixed as a trigger. Any other answer means those seven could never gain a payload contract.
|
||
- **The push catalog is unchanged.** `allStreams()` still serves the stream facet only, so a
|
||
trigger-only id (`uo.house.idoc_warning`) does not appear in the catalog the shipped Android client
|
||
reads. A trigger-only id gets email and in-app preferences in Phase 3 and no push toggle, which is
|
||
correct — there is nothing to push it to.
|
||
|
||
**The ceiling lattice is the one thing this plan named everywhere and defined nowhere**, and getting
|
||
it wrong would have been a security defect rather than a rough edge. §5.1a now carries the definition;
|
||
the short version is that the tempting flat ordering — self < owner < staff < members < authenticated
|
||
< everyone — permits a `staff`-ceilinged trigger to be given an `owner` audience, which is a rule that
|
||
mails `uo.cheat.detected` to the player it detected. Fewer people is not less exposure.
|
||
|
||
**What landed:**
|
||
|
||
- `server/src/modules/ceilings.js` — the six-value lattice, `permits`, `meet`, `meetAll`
|
||
*(Phase 11 adds a seventh, `admin`, as a child of `staff`. See §5.1a for why it was not there from
|
||
the start and why the arithmetic is unchanged.)*
|
||
- `registerEventTriggers` / `registerAudiences` on the existing `stage()` + `apply()` discipline, with
|
||
the cross-facet collision checks and the shared legacy allowlist
|
||
- `ctx.events.emit` (`utils/engagementEmit.js`) — validate, log, **stop**; throws in dev, drops and
|
||
logs in prod; the owner is bound by core and never read from the arguments
|
||
- `ctx.inbox.push` — present and **throws** until Phase 7, the shape 1.6.0 settled on
|
||
*(Phase 7 filled it in. Not a version bump: the signature is the one 1.7.0 declared.)*
|
||
- `config/coreTriggers.js` — core's five, registered through `registerCore()`
|
||
- `GET /admin/engagement/{triggers,audiences}` — admin-only, served from the registries, no table
|
||
- `npm run engagement:manifest` (+ `--check` in CI) and the committed `engagement-triggers.json`
|
||
- `MODULE_API_VERSION` → **1.7.0** on both halves; `MODULE_API.md` §1.1, §2.3, §2.4 and a new §6.8
|
||
|
||
**Two things §6.0b's Phase 2 row assigns that deliberately do NOT land in this window**, and neither
|
||
is a slip:
|
||
|
||
- **`integration-kit`'s `ci/core-ref.json`.** The pin names a website **`main`** sha and
|
||
`checkCoreApi.js` asserts equality with what that sha declares. 1.7.0 is on `edge`, `main` still
|
||
says 1.6.0, so the kit is green and moving the pin now would pin the book to a commit that is still
|
||
in flight — which the pin's own prose forbids ("the kit is written against what shipped"). The
|
||
re-pin and chapter 2's "registering a trigger" section belong in the **Phase 13 cutover window**.
|
||
- **`runicgateway.com`'s `platform.json.moduleApi` → 1.7.0.** `scripts/checkFacts.mjs` *fetches* the
|
||
authority from the source repo's `main`. Setting it to 1.7.0 today turns that repo red immediately
|
||
and for the whole `edge` period. It is Phase 12's, in the cutover window, exactly as the phase says.
|
||
|
||
**Four smaller decisions taken at build time:**
|
||
|
||
| | Decision | Why |
|
||
|---|---|---|
|
||
| `ceiling` required, no default | registration fails without one | there is no safe value to guess: `owner` silently breaks a broadcast, `authenticated` silently widens a staff-only event |
|
||
| `example` required per variable | registration fails without one | §4.3 property 3 is right and a soft version of it is worthless — without an example, preview needs a live game event, which is how template systems ship untested |
|
||
| audiences get their own id space | not the trigger/stream namespace | an audience names a set of PEOPLE, a trigger names an EVENT; `uo.team.members` as both is not a collision |
|
||
| `url` variables are site-relative | validated like `pageUrlTemplate` | a payload variable ends up in an href in an email; `//evil.test/x` passes an "is it rooted" check and is protocol-relative |
|
||
|
||
**One acceptance criterion is only half-dischargeable here, and that is not a slip.** "A rule cannot be
|
||
saved with an audience wider than its trigger's ceiling" needs `engagement_rules`, which is Phase 4's.
|
||
What Phase 2 owes and delivers is the arithmetic that check will call (`permits` / `meet` / `meetAll`,
|
||
with tests for the incomparable cases a total order would have waved through) **and** the same rule
|
||
applied where a row already exists: a declaration whose default `audience` is wider than — or
|
||
incomparable with — its own `ceiling` is refused at registration. Phase 4 adds the second call site,
|
||
not the second implementation.
|
||
|
||
**Left for later, deliberately:** nothing consumes an emitted event yet. `emit` validates and logs,
|
||
and Phase 4 replaces that log line with the engine call. Core's five declarations are registered but
|
||
not yet *emitted* — the Team pipeline keeps its hardcoded mail until Phase 6 migrates it onto the
|
||
engine, and this is what it migrates onto.
|
||
|
||
---
|
||
|
||
### Phase 3 — Channel preferences ✅
|
||
|
||
`notification_channel_prefs` + the idempotent backfill from `notification_subscriptions`. New
|
||
`GET·PUT /auth/me/notifications/channels`. **`/auth/me/notifications/subscriptions` keeps its exact wire
|
||
shape** and becomes the push projection — writes fan out to both.
|
||
|
||
**Acceptance:** the shipped Android app's flat `{streams:[…]}` PUT still round-trips, including the
|
||
empty-array case the app's DTO comment warns about; a per-channel PUT sets `email` without touching
|
||
`push`; a fresh user's email mode defaults `off` and ~~push defaults `instant`~~ **push defaults
|
||
`off` too** (§4.5's `defaultMode` — the struck text was wrong; see the as-built below).
|
||
**Guardrails:** swagger + route manifest; a test pinning the legacy wire shape byte-for-byte.
|
||
|
||
---
|
||
|
||
#### As built (2026-08-29)
|
||
|
||
**Three decisions were settled by the org lead before any code, and one of them corrects this
|
||
phase's own acceptance criterion.**
|
||
|
||
| | Question | Decision |
|
||
|---|---|---|
|
||
| — | how much of §3.1's `registerDeliveryChannel` lands now | **the declarative half only** — id, label, `carriesContent`, `defaultMode`, `supportsDigest`. `addressFor` / `render` / `deliver` wait for the phases that can exercise them |
|
||
| — | push's `defaultMode` | **`off`.** The acceptance line below said `instant`; it could not be |
|
||
| — | whole-set PUT or sparse | **sparse**, on the `(id, channel)` pair — deliberately unlike the two whole-set PUTs either side of it |
|
||
|
||
**The acceptance line was wrong, and it is worth saying exactly how.** "A fresh user's email mode
|
||
defaults `off` and push defaults `instant`" reads naturally beside §3.1's "email opt-IN, push
|
||
opt-OUT", and §3.1 got that from `team_notification_prefs`, where no row genuinely does mean notified.
|
||
But push **stream subscriptions** have never worked that way: `notification_subscriptions` holds a row
|
||
only when a user opted in, so no row means not subscribed. A `defaultMode` of `instant` would have
|
||
projected the **entire catalog** into `GET /auth/me/notifications/subscriptions` for every existing
|
||
user, and the shipped Android client would have shown every toggle switched on after an upgrade
|
||
nobody asked for. It is a live behaviour change dressed as a default. All three channels declare
|
||
`off`, and a test asserts the legacy GET returns `{streams:[]}` for a fresh user so it cannot drift
|
||
back.
|
||
|
||
**Why the channel registry could not wait for Phase 6.** §3.1 says `defaultMode` is expressed once,
|
||
and reading a preference means knowing it — a row exists only where a user has said something. The
|
||
alternative was a constant list beside the prefs model, i.e. that expression in a second place, two
|
||
phases before the registry replaced it. What did *not* land is the behavioural half: registering a
|
||
`deliver` nothing calls freezes a signature before anything has tried to use it, which is the reason
|
||
`transports/index.js` deferred the whole file in Phase 1. Core's three channels are declared, and
|
||
`inapp` is declared `off` for a reason particular to it — the inbox does not exist until Phase 7, and
|
||
a default of `instant` would mean every user is opted into a surface with no rows, so the first thing
|
||
Phase 7 shipped would be a backlog. **Phase 7 changed it to `instant`** once there was a surface to
|
||
look at: an inbox item wakes no device and leaves no building, and the backlog this paragraph feared
|
||
cannot happen against an empty table. See Phase 7's decision 1.
|
||
|
||
**The sparse PUT is the one place this phase leaves the router's idiom, and it buys two things.** A
|
||
whole-set body forces a client that only manages email to send every push row back or wipe them. And
|
||
`off` becomes a mode rather than an omission — which means this endpoint has **no empty-array case at
|
||
all**, so the kotlinx gotcha `putTeamPrefs` had to document (a defaulted array field is dropped from
|
||
the body, and "clear the last one" arrives as no array) simply cannot arise here. `prefs` is still
|
||
required, so a request DTO with no default is still the right shape on the app side.
|
||
|
||
**The projection, stated as an invariant.** `notification_subscriptions` stays exactly what
|
||
`utils/pushDispatch` reads, so this phase touches no delivery path at all. Both endpoints maintain:
|
||
**a `push` pref with `mode <> 'off'` ⟺ a `notification_subscriptions` row** — the legacy PUT with a
|
||
whole-set sweep, the channels PUT one pair at a time. An explicit `off` is *stored* rather than
|
||
deleted, because folding "I turned this off" back into "I never said" is only harmless while the
|
||
default happens to be off.
|
||
|
||
**One thing landed that the phase did not name, and it is a G24 consequence rather than scope creep.**
|
||
A trigger whose ceiling is `staff` can never reach a non-staff user, so offering them a toggle is
|
||
offering a control that does nothing *and* disclosing that the event exists — `uo.cheat.detected`
|
||
would otherwise appear by name in every player's preferences screen the moment Phase 11 declared it.
|
||
It is filtered from the catalog and gated on write, not merely hidden. `members` is deliberately not
|
||
filtered: membership is a runtime resolver's answer, and a preference set before joining a Team should
|
||
already be in place when you join. This gave `ceilings.js` its first consumer for the `staff` label's
|
||
long-standing claim of "admin / editor / moderator", now written down as `STAFF_CEILING_ROLES` — and
|
||
deliberately **not** `teamGrants.STAFF_ROLES` (`['admin','moderator']`), which answers the different
|
||
question of who may act on a Team they are not in.
|
||
|
||
**What landed:**
|
||
|
||
- `server/src/engagement/channels.js` — `registerDeliveryChannel`, `MODES`, `defaultMode`, `modesFor`,
|
||
`acceptsMode`; `coreChannels.js` declares push / email / inapp, registered through the subsystem's
|
||
one door (`require('./engagement')` from `app.js`, beside `registerCore()`)
|
||
- `notification_channel_prefs` + the replay-safe `INSERT IGNORE … SELECT` backfill, copying the
|
||
`announce_jobs → announce_job_legs` precedent
|
||
- `model/notificationChannelPrefs/` — the catalog union, effective-mode resolution, the sparse apply,
|
||
and `mirrorPushSet` for the legacy path; two single-row helpers on `notificationSubs.db`
|
||
- `GET · PUT /auth/me/notifications/channels`, swagger schemas, route manifest
|
||
- `ceilings.STAFF_CEILING_ROLES` / `isStaffRole`
|
||
|
||
**Left for later, deliberately:** no web or app surface. The endpoint exists and is documented
|
||
(`../android/PLAN.md` §11); the screens are Phase 7 (web) and Phase 8 (app), which is where a user can
|
||
see something a preference actually governs.
|
||
|
||
---
|
||
|
||
### Phase 4 — The engine: rules, cooldowns, outbox ✅
|
||
|
||
**Split into 4a and 4b** at the start of the phase, on the same argument that split Phase 5: the half
|
||
that first makes this system capable of sending is worth reviewing without a React screen in the same
|
||
diff, and the ceiling arithmetic that decides who a rule may reach is worth reading on its own.
|
||
|
||
#### 4a — the engine (server only) ✅
|
||
|
||
`engagement_rules`, `engagement_audience_segments`, `engagement_cooldowns`, `engagement_outbox`,
|
||
`engagement_sends`, the sweep worker (`setInterval` + `unref` + `stop`, wired into `server.js` like
|
||
its five siblings), audience resolution, condition evaluation, delay and cancellation, and the
|
||
save-path validation the admin surface will call. **No HTTP surface at all** — provably done when a
|
||
fired trigger produces an outbox row and a send-log entry with no UI in the picture.
|
||
|
||
#### 4b — the admin surface ✅
|
||
|
||
Admin → Engagement → Rules, the §5.1a segment composition UI, and the routes underneath them. Q4's
|
||
answer places it in **its own top-level nav group** (below), so 4b also creates the group that
|
||
Triggers, Templates and the send log join in Phase 5.
|
||
|
||
**Acceptance:** a trigger fired twice inside `cooldown_seconds` for the same (rule, user, subject) sends
|
||
once; the same trigger for a *different* subject sends again; a scheduled row is cancelled by a
|
||
`cancel_on` trigger and never sends; a restart mid-window still sends exactly once; a duplicate
|
||
`dedupe_key` is a successful no-op.
|
||
**Guardrails:** swagger + route manifest (4b — 4a adds no routes); a named test for the multi-house
|
||
cooldown case (§4.1).
|
||
|
||
---
|
||
|
||
#### As built — 4a (2026-08-29)
|
||
|
||
**Three decisions were settled by the org lead before any code, and two of them are §7.1 questions
|
||
this phase was blocked on.**
|
||
|
||
| | Question | Decision |
|
||
|---|---|---|
|
||
| **Q2** | multi-instance: `SKIP LOCKED`, or document single-instance | **Neither, exactly**: a compare-and-set claim — `UPDATE … SET status='sending' WHERE id=? AND status='scheduled'`, the winner being whoever the server reports `affectedRows = 1` to. It is what §4.2a's ENUM was already shaped for (nothing else needs a `sending` state), it needs no open transaction and no MariaDB version floor, and it delivers Q2's intent |
|
||
| **Q4** | where the engagement admin surface lives | **its own top-level nav group**, "Engagement", beside Content / Moderation / System — Rules now, Triggers / Templates / Send Log in Phase 5. Email Delivery stays a section of Settings for now |
|
||
| — | one PR or two | **4a / 4b**, as above |
|
||
|
||
**What Q2's answer does and does not buy.** It makes the *outbox* safe for two app instances. It does
|
||
not make the deployment multi-instance: `announceWorker`, `teamDigestWorker`, `teamForumUploadSweep`
|
||
and `teamActivityPrune` are all still written for one, and widening them is not this phase's scope.
|
||
What it buys is that the one table that will carry mail is ready for the day it is, which is cheap now
|
||
and expensive after mail has doubled once.
|
||
|
||
**Two defects in this document's own §4, both found by building it.**
|
||
|
||
1. **§4.2a's `UNIQUE KEY uq_engo_dedupe (dedupe_key)` was a data-loss bug, not a style question.** A
|
||
dedupe key names the EVENT — "house 0x4001 entered IDOC" — and one event legitimately becomes many
|
||
outbox rows: an audience of fifty users is fifty rows, a rule spanning email and in-app doubles
|
||
that, and a second rule on the same trigger doubles it again. Under a *global* unique index the
|
||
first of those inserts wins and every other one is silently ignored, so ninety-nine recipients are
|
||
dropped by the mechanism that exists to stop a replayed event becoming a second mail. Shipped as
|
||
**`UNIQUE (rule_id, user_id, channel, dedupe_key)`**, which keeps exactly the replay guarantee and
|
||
nothing more. A test asserts one key fans out to six rows.
|
||
|
||
2. **§4.1's single `INSERT … ON DUPLICATE KEY UPDATE` cooldown claim does not work against this
|
||
codebase's pool**, and the way it fails is silent. Its answer is read out of `affectedRows` on the
|
||
usual contract — 1 inserted, 2 updated-and-changed, **0 for a duplicate key whose update changed
|
||
nothing**, that 0 being "still cooling". **The mariadb Node connector defaults `foundRows: true`**,
|
||
which makes `affectedRows` report rows *matched* rather than rows *changed*, and `utils/db.js` does
|
||
not override it. Under that pool the no-op returns 1 and is indistinguishable from a fresh insert:
|
||
**every cooldown passes, always.** Shipped as two statements instead, each of which is its own
|
||
atomic decision and neither of which asks `affectedRows` to mean two things:
|
||
|
||
```sql
|
||
-- 1. claim by moving the row, guarded in a WHERE clause where a row either matches or does not
|
||
UPDATE engagement_cooldowns SET last_fired_at = ?, fire_count = fire_count + 1
|
||
WHERE rule_id = ? AND user_id = ? AND subject_key = ?
|
||
AND last_fired_at <= ? - INTERVAL ? SECOND;
|
||
-- 2. matched nothing? then the row is absent or cooling; INSERT IGNORE separates the two
|
||
INSERT IGNORE INTO engagement_cooldowns (rule_id, user_id, subject_key, last_fired_at, fire_count)
|
||
VALUES (?, ?, ?, ?, 1);
|
||
```
|
||
|
||
Still race-free, and each race resolves the right way: two concurrent first fires both fall to the
|
||
INSERT and the primary key picks one; two concurrent fires after expiry serialise on the row lock
|
||
and the second re-evaluates its guard against the committed `last_fired_at`.
|
||
|
||
**The second defect is the reason this phase has a second test file.** `engagementEngine.test.js`
|
||
stubs the five tables and runs the engine's logic against in-memory stand-ins, which is right for
|
||
everything the engine *decides* — and it was **green against the broken cooldown claim**, because a
|
||
stub can only agree with whoever wrote it, and the same misreading produced both. The statements whose
|
||
correctness is a *server* contract now run against a real MariaDB in `engagementEngineSql.test.js`,
|
||
which creates a throwaway database, drops it, and **skips when there is none** so CI stays green
|
||
without one. The general lesson: a stub is a fine stand-in for a table and a poor one for a protocol.
|
||
|
||
**The gate order is the design.** Enabled rules → conditions → audience → **ceiling re-check** →
|
||
per-channel preference → per-rule hourly ceiling → cooldown → deduped enqueue. Two of those placements
|
||
are load-bearing:
|
||
|
||
- **The G24 ceiling is re-checked at SEND time, not only at save.** The save path already ran the same
|
||
`ceilings.permits`, so the only way this can fail is the case it exists for — a module upgrade that
|
||
*narrows* its trigger's declaration underneath a rule saved when it was wider. Without it, a rule
|
||
written against yesterday's declaration keeps reaching yesterday's population forever. This is the
|
||
second call site §5.1a promised, not a second implementation.
|
||
- **The hourly ceiling is checked before the cooldown**, because the ceiling is about the rule and the
|
||
cooldown is about one recipient. A rule that has hit its ceiling should not also burn every
|
||
recipient's cooldown slot on sends that never happen.
|
||
|
||
**Segments (§5.1a) as built, with one rule the design did not state.** `not` is legal **only as a
|
||
child of `and`**. A complement needs a universe, and the only one available that does not widen is
|
||
the set its siblings produced: `A AND NOT B` is "A, less B", which is what an operator wants and
|
||
cannot be composed into a broadcast. A bare `NOT B` — or `A OR NOT B` — would have to mean "everyone
|
||
except…", which is a way to build the whole deployment out of one narrow audience and is precisely the
|
||
widening rule 3 forbids. It is refused at save with that sentence.
|
||
|
||
The other half of that: **a `not` contributes no ceiling to the meet.** Excluding people cannot widen
|
||
who an expression reaches, so folding the excluded audience's ceiling in would refuse safe segments —
|
||
`members AND NOT staff` would hit `meet('members','staff') = null` and be rejected even though it
|
||
reaches strictly fewer people than `members` alone.
|
||
|
||
**Dormancy, three ways, and none of them deletes anything** (§7.3, §5.1a rule 4). A rule naming an
|
||
unregistered trigger, a rule whose channel is gone, and a rule whose segment was deleted are all
|
||
*listed, flagged and left alone*. In particular `engagement_rules.audience_segment_id` deliberately
|
||
carries **no foreign key**: `ON DELETE CASCADE` would delete an operator's rules and `ON DELETE SET
|
||
NULL` would silently fall the rule back to its plain `audience` column — and that fallback reaches a
|
||
*different set of people*, which is the exact failure §5.1a rule 4 exists to prevent. Deleting a
|
||
segment that a rule still uses is refused in the model, with the count.
|
||
|
||
**Conditions are a small closed grammar**, not an expression language: and/or/not over comparisons of
|
||
one *declared* variable against a literal, every operator renderable as a dropdown, bounded in list
|
||
length and nesting depth because the tree comes out of a JSON column an admin can write and is walked
|
||
on the emit path. Two properties worth keeping:
|
||
|
||
- **An absent variable makes every comparison false, including `ne`.** "Not equal to IDOC" reads as
|
||
satisfied by nothing at all, and treating it that way would fire a rule on every event that omits an
|
||
optional variable. `present` / `absent` are the honest way to ask.
|
||
- **A tree that no longer parses evaluates false**, never "no conditions". Failing closed stops the
|
||
mail; failing open mails everyone the rule could ever reach.
|
||
|
||
**`ctx.events.emit` does not await the engine.** It is called from inside a game-event handler, and
|
||
the caller's job is to say the event happened — not to wait on rule lookups, audience resolution and a
|
||
dozen inserts to find out whether it is allowed to carry on. That is the same argument the C# side's
|
||
`Emit()` makes about the Core thread. `dispatch` catches everything internally and never rejects. The
|
||
consequence a caller must know: **`emit` returns before the outbox rows exist**, so a test that wants
|
||
the delivery decision calls `engine.dispatch` directly.
|
||
|
||
**Nothing is delivered, and that is visible rather than pretended.** A channel's `deliver` arrives with
|
||
email in Phase 6 and the in-app inbox in Phase 7. Until then the worker claims the row, finds no
|
||
`deliver`, finishes it `failed`, and writes a send-log row saying so in as many words. Recording
|
||
`sent` would be a lie in the one table whose entire purpose is answering "did they get it"; leaving the
|
||
row `scheduled` would mean an IDOC warning queued today arriving three weeks later on the deploy that
|
||
first shipped a mailer. On a real deployment the path is unreachable anyway — core seeds no rules and
|
||
`enabled` defaults to 0, so nothing enqueues until 4b's screen exists and an operator uses it.
|
||
|
||
**Three smaller things the phase settled:**
|
||
|
||
- **`everyone` and `authenticated` resolve identically.** A signed-out visitor has no address, no
|
||
device and no inbox, so the widest set the engine can deliver to is the active user table. The
|
||
lattice still distinguishes them — a trigger ceilinged `everyone` permits an `authenticated` rule
|
||
and not the reverse — and only the *resolution* coincides.
|
||
- **A plain `members` audience with no segment reaches nobody.** `members` is the ceiling for "a
|
||
module-declared list"; without a segment there is no list, and core knows no game vocabulary with
|
||
which to guess. Inert and visible, rather than quietly falling back to something wider.
|
||
- **Every audience is filtered through `users.status = 'active'`, including a module's.** A module's
|
||
resolver returns ids over its own store and has no notion of account status; a banned account must
|
||
not be mailable by a module returning its id.
|
||
|
||
---
|
||
|
||
#### As built — 4b (2026-08-29)
|
||
|
||
The admin surface over the Phase 4a engine: **Admin → Engagement → Rules**, **Admin → Engagement →
|
||
Audiences**, the twelve routes under them and the reach preview. Nothing in the engine changed; what
|
||
changed is that an operator can now reach it, which is what makes every rule in 4a's model a rule a
|
||
person will actually meet.
|
||
|
||
**Four decisions the org lead settled before any code**, all four the recommendation:
|
||
|
||
| | Question | Decision |
|
||
|---|---|---|
|
||
| Segments | a fifth nav entry, a sub-tab, or inline in the rule editor | **its own nav entry, "Audiences"** — a segment is a reusable object with its own list, its own edit and a delete that is *refused with a count*, and none of those has anywhere to live inside a screen about rules |
|
||
| The switch | a `PATCH …/enabled` route, or a full `PUT` | **its own route**, writing that column and no other |
|
||
| Preview | count only, count plus a sample, or none in 4b | **count only, on demand** |
|
||
| Delete | hard delete with a confirm, or disable-only | **hard delete** — the send log survives it |
|
||
|
||
**The enable switch is the decision with the most content in it, and it is a correctness argument
|
||
rather than a convenience one.** A `PUT` re-validates, and validation is against the registries *as
|
||
they are now*. So the rules a re-validating toggle cannot switch off are exactly: a rule whose module
|
||
has been uninstalled, a rule naming a channel that is gone, and a rule whose trigger has since
|
||
**narrowed** its ceiling underneath a saved audience. Those are the three rules an operator most
|
||
urgently wants stopped. `PATCH …/enabled` writes one column and always works. Switching a rule *on*
|
||
without re-validation is safe for a different reason: `audiences.permitted` runs again at send time,
|
||
so an enabled-but-no-longer-permitted rule resolves to nobody rather than to the wrong people.
|
||
|
||
**The reach preview calls the engine's own resolver**, `audiences.resolveForRule`, rather than a
|
||
second query that agrees with it today — a preview built out of its own SQL can be wrong about the
|
||
one thing it exists to say. It answers a **count and nothing else**: not a sample, not names. The
|
||
resolver's output for a module-declared segment is a set of players derived from game data, and an
|
||
editor that rendered them would be a user-enumeration surface reached from a screen about mail
|
||
scheduling. Three fields exist because the bare number would otherwise be a lie:
|
||
|
||
- **`capped`** — every audience query is bounded at `MAX_AUDIENCE` (5000), so a count landing exactly
|
||
on the bound is a floor. The screen says "at least 5000", never "5000".
|
||
- **`reason`** — an `owner` audience resolves per event from an id the event carries, so it has no
|
||
advance answer. It reports 0 *with the reason*, because a bare 0 reads as "nobody".
|
||
- **`permitted`** — whether the trigger's G24 ceiling allows the reach just counted. Without it the
|
||
editor shows a healthy number beside a save the server will refuse, which reads as a bug in the
|
||
save rather than as the ceiling doing its job.
|
||
|
||
**Two defects found by walking it against a live server**, neither of which any test in the tree
|
||
would have caught, and both of them in Phase 4a's code rather than 4b's:
|
||
|
||
1. **A rule pointing at a *dormant* segment was reported as healthy.** `listAnnotated` asked only
|
||
whether the segment ROW still existed (§7.3's case: the segment was deleted). The other shape of
|
||
the same failure is §5.1a rule 4's: the segment row is exactly where it was, and every audience in
|
||
it belongs to a module that has been uninstalled. Both leave the rule reaching nobody; only one
|
||
leaves nothing behind. Uninstalling a module under an enabled rule produced a rule the screen
|
||
showed as **on, healthy and firing**, which is the one thing this screen exists not to do. The
|
||
walk over the expression now lives in `engagement/segments.js` as `missingAudiences`, and both the
|
||
segment list and the rule list ask it, so the two cannot disagree about what dormant means.
|
||
2. **"1 rule still use this segment."** The delete refusal pluralised the noun and not the verb. It is
|
||
a one-word fix and it is in the sentence an operator reads at the moment they are being told no.
|
||
|
||
**Six more defects came out of driving the two screens in a browser**, after the API walk above had
|
||
already found the two in Phase 4a's code. None of them is visible from a test or from `curl`, and two
|
||
of them cost an operator something real:
|
||
|
||
1. **The Audience dropdown rendered empty before a trigger was chosen.** `audienceChoicesFor(null)`
|
||
answers `[]` — correctly, because without a trigger there is no ceiling and therefore nothing it
|
||
may legitimately offer. But a `<select>` with zero options reads as a control that is *broken*,
|
||
not one that is waiting. It now carries "Choose a trigger first…" and is disabled.
|
||
2. **A `members` audience with no saved audience reaches nobody, and only the preview button said
|
||
so.** This is the design (§5.1a: `members` is the ceiling for "a module-declared list", and
|
||
without a list core cannot guess) — but it is also the **default** the instant an operator picks
|
||
any `members`-ceiling trigger, which turns a documented property into a trap: the rule saves, gets
|
||
switched on, and mails nobody, with nothing on the screen saying so unless the operator happens to
|
||
press Preview. The editor now says it inline, before the save, and stands down once a preview has
|
||
answered the same question more precisely.
|
||
3. **The composer offered "exclude" on the only row**, which builds an `and` whose every child is a
|
||
complement. The server refuses that correctly — "has nothing but complements, there is no set to
|
||
exclude from" — but only after a save. It is one checkbox away at all times, so the composer now
|
||
refuses it inline, in the operator's words.
|
||
4. **The template-key input truncated its own placeholder** and the text said "optional until Phase
|
||
5", which is a sentence about this document rather than about the operator's deployment.
|
||
5. **"segment" leaked into a screen that says "saved audience" everywhere else.** The API, the schema
|
||
and this document should keep saying *segment* — one word for one table — but an operator meets
|
||
the concept under a heading that reads "Audiences", and a sentence that switches vocabulary
|
||
mid-screen reads as a sentence about something else. Translated at the point of display only.
|
||
6. The composer repeated its "AUDIENCE" heading above every row; the heading belongs to the group.
|
||
|
||
**And one thing that is worth knowing but is not a defect in this phase:** rebuilding
|
||
`client/dist` while the server is running blanks the whole SPA. The HTML shell resolves core's hashed
|
||
bundle filename at boot, so after a rebuild it still points at a filename that no longer exists;
|
||
core's bundle 404s, `window.__rg` is never published, and `modules/uo/entry.js` throws the
|
||
"core did not publish its shared dependencies" error from `MODULE_API.md` §3.1 into a blank page.
|
||
**Restart the server after every client build.** The error names core, and core is not at fault.
|
||
|
||
**A `window.confirm` freezes browser automation.** The two destructive actions here use it, matching
|
||
`InvitesAdmin` and `ModulesAdmin`, and that is the right call for the product — but a native dialog
|
||
blocks CDP entirely, so a delete cannot be driven from a script and a session that opens one is stuck
|
||
until a human dismisses it. Exercise those two paths over the API instead.
|
||
|
||
**Three smaller things the screens decided:**
|
||
|
||
- **A rule's trigger is fixed once the rule exists.** `engagementRules.db.update` never carried
|
||
`trigger_id`, and 4b is where that becomes a stated rule rather than an omission: a rule's cooldown
|
||
rows, its pending outbox rows and its send-log history are all about one trigger id, and
|
||
re-pointing the rule silently re-attributes all three. The editor renders the field read-only and
|
||
says why; changing the trigger means a new rule.
|
||
- **A condition tree the editor cannot render is shown, not flattened.** The editor offers the flat
|
||
half of the grammar — one and/or over a list of comparisons, every operator a dropdown narrowed to
|
||
the picked variable's declared type. `A AND (B OR C)` flattened to `A AND B AND C` fires on
|
||
different events and the operator would have no way to know the save had done it, so such a rule
|
||
opens read-only with its JSON visible and one honest choice: leave it, or clear it and start again.
|
||
- **Literals are coerced to the type the trigger declared**, in the client, before the save. Every
|
||
value in an HTML input is a string and the server rightly refuses `{cmp: 'gt', value: "5"}` against
|
||
an `int`. A value that does not parse is passed through **unchanged** rather than becoming `NaN` or
|
||
`false`, so the refusal names the variable instead of the rule saving cleanly having compared
|
||
against a number nobody typed.
|
||
|
||
**Where the logic lives.** `client/src/lib/engagementRules.js` — plain JS, 25 tests — holds
|
||
everything the two screens *decide*: the form↔payload mapping, which audiences a trigger permits
|
||
(derived from the `permits` list the server sends, never a second copy of the lattice), the reach
|
||
sentence, the `not`-placement check, the condition round trip. None of it is a boundary; the server
|
||
decides and the engine re-checks. It is in a `.js` because the test runner cannot reach a `.jsx`, and
|
||
because a client-side copy of a security rule is a copy that drifts — so the client holds the
|
||
*affordance* (do not offer what will be refused) and the server holds the answer.
|
||
|
||
**The rig that made the live walk possible, and worth reusing.** Core declares **no audiences** — it
|
||
knows no game vocabulary — and `module-uo` 0.3.0 declares none either, so on a stock local stack the
|
||
whole segment half of this phase is untestable. A ~20-line throwaway module dropped into
|
||
`website/modules/rig/` (a `module.json` plus a `server/index.js` calling `api.registerAudiences`)
|
||
declares four audiences at three different ceilings and makes the entire §5.1a arithmetic walkable:
|
||
`OR` taking the tighter ceiling, two incomparable ceilings being refused rather than guessed,
|
||
`members AND NOT staff` being allowed, a parameterised audience, the 409 on a segment in use, and —
|
||
by deleting the directory and restarting — dormancy. `modules/*` is gitignored, and the module needs
|
||
its ids **namespaced to its own id** (`rig.roster.governors`, not `roster.governors`); the registry
|
||
refuses anything else, and it also refuses an id with no dot in it at all.
|
||
|
||
---
|
||
|
||
### Phase 5 — Templates: the seeded set, then the editor ✅
|
||
|
||
Two slices, landing in this order **on purpose** — the seeded set has to exist before the editor, so the
|
||
editor is opening something rather than facing a blank page.
|
||
|
||
**5a — storage, blocks, renderer, seeds.** ✅ `engagement_templates`, the `email.*` block family with
|
||
`toText`, the HTML + plain-text renderer, brand-value resolution, and the §4.6.1 seeded set. The five
|
||
transactional bodies move out of `mailer.js` into seeded rows and `mailer` renders them. **No editor
|
||
yet** — this slice is provably done when the same mail goes out from a template that used to come from a
|
||
string literal.
|
||
|
||
**5b — the editor.** ✅ The §4.6.2 surface: variable palette from the trigger declaration, live preview
|
||
from `example` values, side-by-side HTML/text, preview widths plus dark mode, test send, duplicate.
|
||
Built on the existing block/prop-panel machinery, not a second one — and, because a mail body is rendered
|
||
by the SERVER, the preview is rendered there too and framed rather than redrawn in React. It also lands
|
||
the other two screens Q4 promised Phase 5: **Triggers** (read-only, from the registries) and the
|
||
**Send Log** (G15).
|
||
|
||
**Acceptance (5a):** every one of the five current message types renders byte-comparably from its seeded
|
||
template; re-running the seeder is a no-op; a seeder bump updates a `customized = 0` row and **skips** a
|
||
`customized = 1` one; two deployments with different `BRAND_*` produce differently-branded mail from the
|
||
same seed.
|
||
**Acceptance (5b):** a template referencing an undeclared variable is refused at save **with the variable
|
||
named**; preview renders from examples with no live event; a published template with an empty text part
|
||
is refused; a `protected` template cannot be deleted but can be duplicated; a template pinned to an
|
||
older `trigger_version` is flagged in the admin list; an interpolated variable containing `<script>`
|
||
renders escaped.
|
||
**Guardrails:** **CSP — this is the one genuinely new CSP surface.** The preview renders
|
||
operator-authored HTML; it must be sandboxed (`<iframe sandbox>` with no `allow-scripts`, `srcdoc`,
|
||
`about:blank` origin) so it never executes under the site's origin, and `sanitizeHtml` runs on write as
|
||
well as on render. Worth a `test/csp.test.js` sibling asserting the preview frame's attributes.
|
||
|
||
#### As built — 5a (2026-08-29)
|
||
|
||
Built as website#TBD. Three decisions were settled by the org lead before any code, each because the
|
||
tree contradicted something the plan assumed.
|
||
|
||
| | Question the survey raised | Decision |
|
||
| --- | --- | --- |
|
||
| **The HTML part** | **Not one existing mail has one.** All six senders in `mailer.js` set `text:` only, so "renders byte-comparably" is a statement about a *text* body and whether 5a introduces HTML to live mail was an open choice | **Multipart now, text byte-identical.** The text part is byte-for-byte what went out before; the HTML alternative is new. The renderer is therefore exercised by real mail in the phase that builds it rather than shipping dead until 5b |
|
||
| **The block registry** | **The server block registry has no renderer of any kind.** Page blocks are drawn by React on the client; `registerBlock` freezes a fixed field set and would silently DROP a `toHtml`/`toText` | **A sibling registry, with the machinery shared by binding.** §4.4's "do not build a second editor" is about the editor, and 5b still drives these through the existing block/prop-panel machinery |
|
||
| **The seed scope** | §4.6.1 lists nine seeds but `teamNotify`/`teamDigestWorker` are explicitly Phase 6's to rewrite | **Seed all nine, wire the six transactional.** Phases 6 and 7 open something rather than each shipping seeds of their own |
|
||
|
||
**And a correction to §4.6.1 itself:** it lists `auth.email-verify` as "*(new — Phase 9)*". Phase 1b
|
||
already shipped `mailer.sendEmailVerification`, so it is a **current** message type, not a future one —
|
||
six bodies moved, not five, and all six are pinned by the byte-comparison test.
|
||
|
||
##### The two registries are siblings, and that is an argument rather than a preference
|
||
|
||
Sharing one Map would have cost three things. Email blocks **render on the server** and so carry
|
||
`toHtml`/`toText`, which the page registry's frozen entry shape has nowhere to put. One Map is one
|
||
namespace, and the page registry's only server consumer is `pages.model.js` — so `email.heading` in it
|
||
means a CMS page containing an email block validates and saves, with nothing on the client able to draw
|
||
it. And the entry shapes genuinely differ: `cacheTTL` and `container` mean nothing to a mail body.
|
||
|
||
What *is* the same rule for both is shared by binding, not by copy. `blocks/validateBlocks.js` and
|
||
`sanitizeBlocks.js` became factories over a registry lookup (`makeValidateBlocks` /
|
||
`makeSanitizeBlocks`), each exporting the page-bound instance every existing caller already imports, and
|
||
`emailBlocks/` binds the same walk to its own registry. The envelope rules, id uniqueness, schema
|
||
dispatch and the validate-then-sanitize order therefore cannot drift between the families. There is a
|
||
test asserting both directions of the isolation: an `email.heading` fails page validation, and a plain
|
||
`heading` fails email validation.
|
||
|
||
##### The token grammar has no conditional, so the ternaries stayed at the call site
|
||
|
||
`{{ name }}`, a bare declared variable, and nothing else — no filters, conditionals, loops or dotted
|
||
paths. Repetition is a block (`email.itemList` renders a declared *list* variable), which is the one
|
||
place a template needs "for each" and it already has a typed, validated home.
|
||
|
||
That has a visible consequence. `mailer` built ` for the account “Darrow”` with a ternary, and a
|
||
logic-free template cannot. So the ternary stays where a ternary belongs and its **result** arrives as a
|
||
variable — `forWhom`, `roleLabel`, `invitedBy`, `moreNote` — each declared with an `example` showing
|
||
exactly what it produces, leading space and quotes included. It is not pretty in the editor and it is
|
||
the price of not giving operator-authored data a conditional to get wrong. **Both branches of every
|
||
ternary are asserted**, because the empty one is what a template language with a conditional would most
|
||
likely get wrong.
|
||
|
||
Where a conditional would otherwise be reached for, "nothing in, nothing out" stands in: a block whose
|
||
content interpolates to nothing renders nothing, in **both** parts. `{{moreNote}}` on its own line is a
|
||
line the caller can decline to supply.
|
||
|
||
##### Three properties of the renderer that are load-bearing
|
||
|
||
- **The shell contributes structure and no content.** No appended footer, no injected logo, no "sent by"
|
||
line. An unsubscribe line is a *variable inside the template*, so an operator can move it, reword it,
|
||
or see that a transactional mail correctly has none — and, more importantly, the HTML and text parts
|
||
say the same things. A footer in one and not the other is a deliverability signal and means the text
|
||
reader is told less than the HTML reader.
|
||
- **Only the accent comes from the theme.** Every shipped preset is a DARK palette, and §4.6.2 already
|
||
names the failure: a light-only template "renders as unreadable dark-on-dark in about a third of
|
||
inboxes", because clients invert or force their own background. Deriving a light palette from a dark
|
||
one is a guess at six colours; taking the one colour that carries the brand is exact. §4.6.1's
|
||
property 2 holds either way — no seeded template contains a hex code, asserted by a test.
|
||
- **A URL built from a variable is re-checked after substitution.** A stored `{{resetUrl}}` says nothing
|
||
about where it points. Checking only the literal would let a variable carrying `javascript:` become an
|
||
href; a substituted value that fails `isSafeUrl` loses its href and renders as inert text rather than
|
||
vanishing, because silently dropping it would hide from the reader that the mail meant to offer them
|
||
something.
|
||
|
||
##### A missing row renders the shipped default, which is what makes the whole move safe
|
||
|
||
`renderByKey` falls back to the in-code seed whenever the row is absent or its `blocks` will not parse:
|
||
before the first seed runs, after a restore that dropped the table, on a row hand-edited in the
|
||
database. Without it, moving a password-reset body into a table would have made every failure mode of
|
||
that table a failure mode of account recovery. `protected = 1` stops the last of those from being
|
||
reachable through the API at all.
|
||
|
||
##### The seed guard is in the SQL, not in a read-then-write
|
||
|
||
`INSERT IGNORE`, then `UPDATE … WHERE seed_key = ? AND customized = 0 AND seed_version < ?`. A check in
|
||
JavaScript followed by an UPDATE leaves a window in which a concurrent boot overwrites an edit an
|
||
operator made a moment earlier, and a deployment can start two app processes at once. MariaDB's
|
||
`ON DUPLICATE KEY UPDATE` cannot carry a WHERE, which is why this is two statements rather than the
|
||
upsert §4.6.1 sketches.
|
||
|
||
Related, and recorded because [Phase 4a](#as-built--4a-2026-08-29) was bitten by the same thing: the
|
||
connector defaults **`foundRows: true`**, so `affectedRows` on an UPDATE counts *matched* rows. For the
|
||
seeder that is harmless (its WHERE only matches a row that will change); for an operator's save it is
|
||
the semantics wanted — re-saving a template unchanged is a success, not a 404. Both are now stated in
|
||
the code rather than relied on.
|
||
|
||
##### Two behaviour changes an operator will notice
|
||
|
||
1. **Mail is multipart.** A client that prefers HTML now shows a branded body where it used to show
|
||
plain text. Nothing a text-only reader sees has changed.
|
||
2. **Subjects resolve the deployment's own name.** They interpolate `{{siteName}}`, which is
|
||
`settings.getInstanceName()` — the admin-set `site_title` falling back to `BRAND_NAME`, rather than
|
||
`BRAND_NAME` alone. On an instance that never set a site title nothing changes; on one that did, the
|
||
subject finally says what the site calls itself.
|
||
|
||
Also inherited rather than introduced, and now visible: `admin.contact-message` declares **both**
|
||
`fromLabel` and `fromName` — the same missing name with the two different fallbacks the literal used
|
||
('a visitor' in the subject, 'unknown' in the body). Kept exactly, asserted, and now editable by whoever
|
||
wants one word.
|
||
|
||
##### One defect this phase found in a Phase 1 check
|
||
|
||
`npm run check:hosts` (§3.2 rule 4) read the template key **`auth.email-verify`** as the hostname
|
||
`auth.email`. `.email` is a real TLD and the pattern's trailing `\b` matches between `l` and `-`, so any
|
||
engagement identifier whose label happens to end in a TLD tripped it — and §4.6.1 names that key. Fixed
|
||
with a `(?![-\w])` after the TLD: a real hostname's TLD is its last label, so a following `-` or word
|
||
character means the match is a truncation of a longer identifier. Everything a host *is* followed by (a
|
||
quote, `/`, `:`, `?`) still matches, and the checker's own suite gained both the identifiers it must now
|
||
accept and two real `.email` hosts it must still catch.
|
||
|
||
##### Verification
|
||
|
||
- **31 new server tests** (`test/emailTemplates.test.js`) and 3 added to the host-check suite. Full
|
||
server suite green: 1388 passing, with the one known Windows CRLF artifact
|
||
(`engagement-triggers.json` byte comparison) and `honeypot.test.js`'s 10-second pool-acquire flake
|
||
under full-suite parallelism, both of which reproduce on clean `edge`. `routes.manifest` /
|
||
`routes.guards` need `modules/uo` moved aside, as ever; 5a adds no routes.
|
||
- **The seeder's SQL against a real MariaDB**, because the unit tests stub `seedOne` and therefore prove
|
||
the loop rather than the statements — the exact shape of Phase 4a's `foundRows` trap, where a stub
|
||
agreed with a broken query. First run 9 inserted; second 9 skipped; after a `seedVersion` bump 8
|
||
updated and the customized row skipped, keeping both its words and its old version, and surfaced by
|
||
`staleCustomized`. The blocks JSON round-tripped through `MEDIUMTEXT`, and `update()` twice with
|
||
identical values returned true both times.
|
||
- **Real mail, end to end**, through nodemailer and SMTP into a mailpit catcher: all five senders, the
|
||
message arriving as `multipart/alternative; charset=utf-8` with the curly quotes correctly
|
||
quoted-printable-encoded, the button rendering with its bare URL beneath it, and the text part
|
||
matching the deleted literal.
|
||
- **The same seed rows, two deployments.** Run as *UOMysticmoon* with a gold accent and as *Vesper Isle*
|
||
with a blue one, changing only stored settings: the subjects and the button colour follow the
|
||
deployment, from identical rows. §5a's fourth acceptance criterion, proved rather than argued.
|
||
- **A contact message carrying `<script>alert(1)</script> & <img src=x onerror=…>`**, sent for real:
|
||
escaped in the HTML part, raw in the text part, no live tag in the delivered message.
|
||
- One accidental proof worth keeping: an early rig run had a `settings` table missing `updated_at`, and
|
||
`ambient()` degraded to the `BRAND_*` env values with a warning and sent the mail anyway. That path
|
||
is not otherwise easy to reach.
|
||
|
||
**Still 5b's:** the editor, the admin Templates screen, the template CRUD routes, the save-time
|
||
undeclared-variable refusal (`variablesFor` is in place and is what it will ask), the sandboxed preview
|
||
and its CSP test, and the `runicgateway.com` admin docs page §6.0b assigns the pair.
|
||
|
||
#### As built — 5b (2026-08-29)
|
||
|
||
Built as website#TBD. **Q4 was already settled** (Phase 4a), so the only thing needing a decision before
|
||
code was the scope: §Phase 5's body names the editor and nothing else, but Q4's answer and §6.2 both
|
||
promise "Triggers, Templates and the send log" in Phase 5. **All three shipped.** Triggers is a
|
||
read-only render of two endpoints Phase 2 already serves and cost no server work; the send log is one
|
||
paged route over a table that has been filling since Phase 4a and whose index was built for it. Leaving
|
||
either out would have left the nav group half-built and **G15 — "no send log, no delivery status, no
|
||
audit" — open with the rows already on disk.**
|
||
|
||
Five more decisions were settled by the org lead before any code, each because the tree contradicted
|
||
the plan or the plan contradicted itself.
|
||
|
||
| | What the tree said | Decision |
|
||
| --- | --- | --- |
|
||
| **Where the preview comes from** | The client block registry mirrors the server's, but its entry shape carries a **React `component`** — page blocks are drawn in the browser. Email blocks are drawn on the SERVER; a mail body is a string this process produces | **A server preview route**, rendered into a sandboxed iframe. A React preview would be a second renderer for one artifact, agreeing with the send path on the day it was written and drifting from the first Outlook fix onward |
|
||
| **`status` was unenforced** | `draft`/`published` shipped in 5a and **nothing read it** — `getByKey` returns any row, so an operator who saved a template as a draft kept mailing it | **`renderByKey` requires `published`** and otherwise falls back to the shipped seed, the posture 5a already built for a missing or unusable row. A draft now means what the word means |
|
||
| **Test send vs. `trigger_id NOT NULL`** | §4.6.2 wants a test send "recorded in `engagement_sends` like any other message", but every transactional template has `trigger_id` NULL and there was nothing honest to put in the column | **A synthetic `core.admin.test-send` id.** No schema change, no nullable column, and the log keeps meaning one thing. It is deliberately not a registered trigger, and the screen renders it by name so nobody goes looking for it in the catalog |
|
||
| **Deleting** | `protected` blocks deletion. Nothing stopped deleting a template a rule's `template_keys` points at | **409 while a rule uses it, naming the rules** — the answer Phase 4b already gives for a segment in use, for the same reason: the alternative is a rule that silently stops producing mail |
|
||
| **Creating** | §4.6.2 names duplicate and never mentions a blank create | **Duplicate only.** Every template on a deployment descends from a shipped one that renders, which is the whole reason 5a landed first |
|
||
|
||
##### The correction that changed the most code
|
||
|
||
§4.6.2 introduces duplicate as "how an operator customizes a `protected` template safely: duplicate,
|
||
edit, point the rule at the copy, leave the original intact". The schema comment written in 5a says the
|
||
opposite — "Editable, NOT deletable" — and **the org lead's ruling is the schema's: a default template
|
||
is edited in place.** `customized = 1` is what stops the next seed bump from taking that edit back, and
|
||
it has been in the UPDATE's own WHERE since 5a. So `protected` now blocks deletion and *nothing else*,
|
||
and duplicate is how a NEW template comes into being rather than how an existing one is customized.
|
||
|
||
##### Three things the plan did not know, found by building it
|
||
|
||
**1. The variable check cannot be a token scan, because of one block.** §4.6.2's refusal — "a template
|
||
referencing an undeclared variable is refused at save with the variable named" — reads as a scan for
|
||
`{{name}}`. It is not sufficient. `email.itemList.variable` holds a **bare name** (`items`), because the
|
||
block iterates the value rather than interpolating it; a token scan sees nothing there. A digest pointed
|
||
at `itmes` would have saved clean and arrived empty, which is the one variable mistake a reader of the
|
||
template cannot see. Blocks now optionally declare `variables(props)` in the registry — `itemList` is the
|
||
only one that does — and `emailBlocks/variables.js` walks tokens *and* declarations across the subject,
|
||
the text override and every block prop. The editor makes that field a `<select>` over the trigger's list
|
||
variables rather than a text input, so the mistake is unavailable in the first place.
|
||
|
||
**2. A duplicate that drops its seed reference cannot be saved.** The obvious shape for `create` was
|
||
`seed_key = NULL` — a copy is not a seed and must never be adopted by the seeder. But
|
||
`templates.variablesFor()` resolves the palette from the trigger *or, for the generic templates tied to
|
||
no trigger, from the seed*, so a seedless, triggerless copy has only the four ambient variables, and
|
||
**duplicating `notify.event` would have been refused for the `{{title}}` and `{{intro}}` it was copied
|
||
with** — the one action §4.6.2 offers, refusing itself. The copy inherits `seed_key`, and it is safe to
|
||
because `customized = 1` is what the seeder actually reads: `seedOne`'s UPDATE carries `AND customized =
|
||
0`, so it can only ever match the seeded row. `staleCustomized` *does* match a copy, and should — "the
|
||
default you duplicated has been improved" is worth telling someone.
|
||
|
||
**3. `validateEmailBlocks` returns `{ valid, errors }`, not an array** — and the first version of the
|
||
model tested it with `.length`, which is `undefined` on an object, so **block validation never ran at
|
||
all**. Unvalidated props would have reached both the renderer and the row. Nothing about the code looked
|
||
wrong; the test that caught it was the one asserting a preview refuses a block with an unknown prop.
|
||
`pages.model.js` destructures it, which is why that caller has never had the bug.
|
||
|
||
##### The one genuinely new CSP surface, and how it is held
|
||
|
||
§4.6.2 called the preview out as "the one genuinely new CSP surface", and it is: operator-authored HTML
|
||
reaching a rendered surface. Three things hold it, in order of what they cost to remove:
|
||
|
||
- The preview renders in `<iframe sandbox="" srcDoc={…}>` — **every restriction on, nothing granted
|
||
back**, from an opaque origin. No `allow-scripts`, no `allow-same-origin`.
|
||
- The HTML crosses as a **JSON string**, never as a document served from this origin. That is not a
|
||
convenience: served as a document it would run under the site's own CSP with access to its cookies.
|
||
- Blocks are validated then sanitized **on write**, against the `email.*` registry, so the stored row is
|
||
already through the gate before any of the above matters.
|
||
|
||
The sibling test §4.6.2 asked for is `client/test/emailTemplates.test.js`. It reads the **attribute**,
|
||
not the file — the first version searched the source for `allow-scripts` and failed on the comment above
|
||
the iframe explaining that there is no `allow-scripts`. A check a correct file fails is worse than no
|
||
check, because the way to make it pass is to delete the explanation.
|
||
|
||
##### What was verified
|
||
|
||
- **23 model tests** (`server/test/engagementTemplatesAdmin.test.js`), one per acceptance criterion plus
|
||
the two the tree made necessary. Server suite 1403, client 324, both green.
|
||
- **The drift check was verified by breaking it both ways** — a client `version: 2` against the server's
|
||
`1`, and deleting a registration outright — because a pairing check that spans a process boundary is
|
||
the kind that silently stops checking.
|
||
- **`renderByKey`'s draft arm has its own test**, asserting the seed goes out and the draft's words do
|
||
not; the 5a fixture that carried no `status` was updated rather than the rule being softened.
|
||
|
||
**Still Phase 6's:** the email channel's `deliver` on the engine, and the `teamNotify` /
|
||
`teamDigestWorker` migration onto the four `notify.*` templates this phase can now edit.
|
||
|
||
---
|
||
|
||
### Phase 6 — The email channel on the engine, and the Teams migration ✅
|
||
|
||
Email becomes a `DeliveryChannel` driven by rules. `teamNotify.js` and `teamDigestWorker.js` are rewritten
|
||
onto the generic pipeline; `engagement_digest_state` backfills from `team_notification_prefs.last_digest_at`;
|
||
`unsubscribeToken` generalizes from `(userId, teamId)` to `(userId, channel, scopeKey)` **while still
|
||
verifying old two-part tokens**, because tokens are already in people's mailboxes.
|
||
|
||
**This is the highest-risk phase.** It touches live behaviour that people receive by email.
|
||
|
||
**Acceptance:** every existing Team notification test passes against the new pipeline; a pre-existing
|
||
`email_mode='digest'` row produces exactly one daily digest with the same window clamping; an
|
||
unsubscribe link from a mail sent *before* the migration still works; `teamDigestWorker`'s three
|
||
compute-at-send-time properties are each covered by a named test, especially *a user who lost access
|
||
between the post and the send is not mailed*; `List-Unsubscribe` + `List-Unsubscribe-Post` still both
|
||
present.
|
||
**Guardrails:** route manifest (the unsubscribe routes move); a migration test asserting the backfill is
|
||
replay-safe.
|
||
|
||
---
|
||
|
||
#### As built — 6 (2026-08-29)
|
||
|
||
The email channel gained a `deliver`, and the Team pipeline stopped being its own thing. Concretely:
|
||
`teamNotify.emailImmediate` and `mailer.sendTeamNotification` are gone, `teamNotify.forumPost` emits an
|
||
event instead, and a rule decides who is mailed. One end-to-end walk on a live rig now goes
|
||
**forum write → `events.emit` → rule → outbox → worker → email channel → template → SMTP → mailbox**,
|
||
with the send recorded in `engagement_sends` like everything else the platform sends.
|
||
|
||
**Seven decisions the org lead settled before any code**, because the tree contradicted the phase body
|
||
in every one of them:
|
||
|
||
| | Question | Decision |
|
||
|---|---|---|
|
||
| Sinks | `teamNotify` has three sinks; which move onto the engine | **email only.** The tickle and the Discord bridge stay direct calls in `teamNotify.js` |
|
||
| Audience | how a Team event reaches that Team's members, when `members` resolves to nobody and a segment takes constant params | **the event carries the set** — the same access-checked list `teamNotify.recipientIds` has always computed |
|
||
| Continuity | rules default off and core seeds none, so a straight migration stops Team email silently | **seed the four rules DISABLED**, plus an admin banner and a release note. The invariant is honoured, not excepted |
|
||
| Prefs | per-Team granularity has nowhere to live in `notification_channel_prefs` | **keep `team_notification_prefs`**, consulted by the engine as a scoped preference |
|
||
| Templates | §4.6.1 property 1 promises a generic template renders any trigger, and nothing implemented it | **payload wins, a structural projection fills gaps** |
|
||
| Digest | §4.2b says compute at send time; the engine snapshots at emit time | **keep compute-at-send-time**; generalize only the state |
|
||
| Unsubscribe | a v1 token set `muted`, silencing push as well as email | **turn off the channel the token names, and nothing else** |
|
||
|
||
##### The audience problem, which is the one that shaped the phase
|
||
|
||
The engine could not express "the members of the Team this post was in", and the reason is structural
|
||
rather than an oversight. A rule names its audience two ways: a plain ceiling name resolved from core's
|
||
own tables, or a **saved segment** composing module-declared audiences with **constant** parameters. A
|
||
Team forum post needs neither — the recipient list is different for every firing, and it is the answer
|
||
to an access question (`teamAccess.forumAccess`'s two tables) that core already knows how to ask and a
|
||
segment has no way to ask at all. `audiences.js` said as much in its own comment: *"core knows no game
|
||
vocabulary and cannot guess which members were meant."*
|
||
|
||
So the **event** names it. `engagementEmit`'s envelope gained `recipientUserIds`, and a rule whose
|
||
audience is `members` resolves to exactly that set. Three properties make it a narrowing input rather
|
||
than a hole in the ceiling lattice, and all three are tested:
|
||
|
||
- the carried set is still filtered through `users.status = 'active'`, so a banned account is not
|
||
mailable by an emitter that forgot;
|
||
- the ceiling returned is still `members`, so **G24 still runs** — a rule cannot be given
|
||
`authenticated` on a trigger that ceilings at `members`, carried set or no carried set;
|
||
- the list is bounded at `MAX_AUDIENCE` (5000) *at the emit boundary*, so an emitter cannot assert an
|
||
audience larger than the engine would have loaded from a query.
|
||
|
||
**A rule with any other audience ignores it entirely.** This is not "the emitter decides who gets
|
||
mailed": it is the emitter answering the one question core cannot, and the rule deciding everything
|
||
else.
|
||
|
||
##### `scope_key` is not `subject_key`, and Phase 6 is where that stopped being theoretical
|
||
|
||
Both are per-event strings on the outbox row and they are keyed on different things:
|
||
|
||
- **`subject_key`** is what a **cooldown** counts. It comes from the trigger's declared `subjectKey`,
|
||
which for all four Team triggers is `teamName` — a display string, which is fine, because a cooldown
|
||
only ever compares it with itself.
|
||
- **`scope_key`** is what a **preference** and an **unsubscribe** are keyed on, and it has to be a
|
||
stable identifier. `team:12` survives a rename; `The Silver Anvil` does not. Signing an unsubscribe
|
||
token over a display name would orphan every link in every mailbox the first time staff renamed a
|
||
guild — and those links have no expiry, so "the first time" means "ever".
|
||
|
||
`engagement_digest_state.scope_key` already used that vocabulary in §4.2b, so the outbox column is the
|
||
same vocabulary in the same shape rather than a second one.
|
||
|
||
##### The scoped preference, and the one place the decision as phrased could not ship
|
||
|
||
Decision 4 was recorded as *"a suppression below the channel preference"*. Building it showed that
|
||
reading is the one that cannot ship, and the reason is worth stating because it is a second instance of
|
||
the G22 pattern — a migration that degrades silently:
|
||
|
||
`notification_channel_prefs` holds a row **only where a user has expressed something**; absence means
|
||
the channel's `defaultMode`; and email's is `off`. Nobody has ever expressed a stream-level opinion
|
||
about `team.forum.post` — the screen that would let them is Phase 3's and the preference predates it by
|
||
a year. So intersecting the two preferences would resolve **every existing Team-email subscriber** to
|
||
`off`, and the deploy that migrated the pipeline would be the deploy that silenced it.
|
||
|
||
What shipped is **replacement**: where a scope has an opinion, that opinion is the preference.
|
||
`engagement/scopedPrefs.js` is a small registry keyed on the scope-key prefix; `engagement/coreScopePrefs.js`
|
||
registers `team` and maps the two columns:
|
||
|
||
- `muted` → `off` **on every channel**. That is what the toggle has always meant on the account screen,
|
||
and narrowing it to email would be a behaviour change nobody asked for.
|
||
- `email_mode` → the engine's vocabulary (`immediate` → `instant`), and **only for the email channel**.
|
||
On push or in-app the provider returns no opinion and the stream-level preference decides.
|
||
- **absence of a row means `off` for email**, which is why the provider answers for every user in the
|
||
set rather than only the rows it finds. The column defaults to `'off'` and both recipient queries
|
||
COALESCE to it: no row has always meant "has not asked for Team email". Deferring to the stream-level
|
||
preference instead would mean somebody who once switched on `team.forum.post` email in the channels
|
||
screen starts receiving mail from **every Team on the deployment** — a widening produced by a
|
||
migration, of a preference expressed about something else.
|
||
|
||
The cost, stated so nobody rediscovers it: a user cannot turn Team email off for every Team at once
|
||
from the channels screen. That control lives on the per-Team screen, which is where it has always lived
|
||
and where the unsubscribe link points.
|
||
|
||
The provider **fails open** — a lookup that throws leaves the stream-level preference in charge, which
|
||
for every core channel is `off`. So a failure means nothing is sent rather than that everybody is
|
||
mailed, and it does not drop an unrelated IDOC warning because a Team preference query timed out.
|
||
|
||
##### The projection: what §4.6.1 property 1 actually required
|
||
|
||
Property 1 says a new trigger renders through `notify.event` **with no authoring at all**. Nothing
|
||
implemented it, and building the channel is what made the hole visible: trigger payloads are
|
||
domain-named (`teamName`, `threadTitle`, `postUrl`) and the generic seeds are structural (`title`,
|
||
`intro`, `items`, `actionUrl`). The two vocabularies never met.
|
||
|
||
`engagement/projection.js` is the meeting, and its rule is one line: **a name the payload already
|
||
carries is left exactly as emitted; only a name it does not carry is supplied.** `news.post` and
|
||
`team.announcement` both declare their own `title`, and a projection that overwrote it would replace a
|
||
real headline with a category label — on the one variable every generic template puts in the subject.
|
||
|
||
What it fills comes from the **declaration**, never from a table of domain synonyms:
|
||
|
||
| | Filled from | Why not something cleverer |
|
||
|---|---|---|
|
||
| `title` | the trigger's `label` | a mapping of `threadTitle` → `title` is one game's vocabulary compiled into core, and it is wrong on the first module that names the same thing differently |
|
||
| `intro` | the trigger's `description` | same |
|
||
| `actionUrl` | the first declared `url`-typed variable holding a value | it is the only structural fact available: the declaration says which variables are links |
|
||
| `items` | `[]` | set rather than left absent, so a generic mail does not report `items` as a *missing variable* in the editor's preview |
|
||
|
||
The consequence, seen on the live rig and accepted: an unauthored `team.forum.post` mail is titled
|
||
"Team — new forum post" rather than the thread's title. That is a **plain** mail, not a wrong one, and
|
||
the operator's answer is the bespoke template shipped beside it. A projection clever enough to do
|
||
better is a projection that is confidently wrong on the first module that does not follow core's naming.
|
||
|
||
##### The digest: the state generalizes, the design does not
|
||
|
||
§4.2b's instruction was to keep `teamDigestWorker`'s compute-at-send-time design and generalize its
|
||
state. Doing that meant **correcting the engine**, not just the worker: Phase 4a's `subscribedTo`
|
||
enqueued `digest`-mode recipients with a comment reading *"what changes in Phase 6 is who drains it"*,
|
||
and what changed in Phase 6 is that **nothing drains it**. A digest is re-derived from the source tables
|
||
when it goes out; an outbox row carries a snapshot taken at emit time, and a snapshot has none of the
|
||
three properties the design exists for. So the engine now enqueues **`instant` only**, and each of the
|
||
three properties has its own named test:
|
||
|
||
1. a two-day outage sends ONE digest, not two days of replay;
|
||
2. a post a moderator hid after it was written is not in the query, so it is not in the mail;
|
||
3. **a user who lost forum access between the post and the send is not mailed** — the security one.
|
||
|
||
`engagement_digest_state` replaces `team_notification_prefs.last_digest_at`, keyed
|
||
`(user_id, channel, scope_key)` so a second digest — on another channel, or over another scope — needs
|
||
no second column on somebody's preferences row. The backfill is an `INSERT IGNORE … SELECT` in
|
||
`schema.sql`, and it is replay-safe **by construction rather than by a flag**: the primary key rejects
|
||
the second run, so a window the new worker has since moved forward is never dragged backwards by a
|
||
restart. Verified on the rig by restarting the server after a digest had gone out — the stamped row kept
|
||
its stamp, and the unstamped row was added.
|
||
|
||
**The digest is gated on an enabled email rule**, which is the part that was not in the phase body.
|
||
Without it, disabling the rule would stop the instant mail and leave a daily summary arriving
|
||
indefinitely, which reads to an operator as the switch being broken.
|
||
|
||
##### The unsubscribe: a narrowing, and a path that can never move
|
||
|
||
The token is now `(userId, channel, scopeKey)`, signed as `2.<uid>.<channel>.<scope>.<mac>`. **v1
|
||
tokens still verify, permanently**, and read as `{ channel: 'email', scopeKey: 'team:<id>' }` — which is
|
||
a reading of what they always meant, since a v1 token could only ever have arrived in an email.
|
||
|
||
Two things about it are deliberate and easy to undo by accident:
|
||
|
||
- **The route did not move, even though the phase body said the unsubscribe routes would.** The
|
||
canonical pair is now `/api/v1/public/engagement/unsubscribe/:token`, and
|
||
`/api/v1/public/teams/unsubscribe/:token` **stays forever**, handing straight to the same handlers.
|
||
Mail sent before this phase carries the old path in its `List-Unsubscribe` header and in its body;
|
||
mail is not editable once sent, so a route that moves is a person who cannot unsubscribe. Both paths
|
||
are in the route manifest.
|
||
- **A v1 token now turns off email and no longer mutes push.** That is the live behaviour change
|
||
decision 7 accepted: a link labelled "stop these emails" was quietly stopping notifications on
|
||
somebody's phone. Verified on the rig — POSTing a v1 token to the old path set `email_mode = 'off'`
|
||
and left `muted = 0`.
|
||
|
||
A channel id may legally contain a dot (`discord.dm` is §3.1's own example) and the token's separator is
|
||
a dot, so `sign` **refuses** such a channel rather than producing a token that verifies as a different
|
||
one. A scope the format cannot carry costs the mail its unsubscribe link, not the mail.
|
||
|
||
##### Two defects the phase found in code it did not write
|
||
|
||
1. **`email.button` never absolutized its href.** `email.image` and `email.itemList` both call
|
||
`ctx.absolute`; the button called `ctx.safeHref` alone. It had never mattered, because every caller
|
||
before this phase passed an absolute URL. A trigger's `url` variables are validated **site-relative
|
||
by construction** (`engagementEmit.RELATIVE_URL` exists so a variable cannot carry a recipient
|
||
off-site), so every rule-driven call-to-action would have interpolated to `/guilds/x` — a path a mail
|
||
client has no origin to resolve, i.e. a dead link in every notification the engine sends. Fixed in
|
||
both parts, with a test either side of the relative/absolute split.
|
||
2. **The digest's send-log row had no `address_hash`.** The instant path wrote one and the digest path
|
||
did not, so half the deployment's mail would have been uncorrelatable when Phase 9's bounce handling
|
||
arrives. Found by reading `engagement_sends` on the live rig, where the two rows sat next to each
|
||
other. Both paths now hash through the same function, lower-cased and trimmed — a bounce reported for
|
||
`Darrow@` has to match the row written for `darrow@`.
|
||
|
||
##### A naming inconsistency left alone, on purpose
|
||
|
||
`team.forum.post` declares its title as `threadTitle` and `team.announcement` declares it as `title`,
|
||
though both describe a thread in a Team forum. A template can only name one of them, so
|
||
`notify.team-post`'s `{{threadTitle}}` renders empty for an announcement — which is why the seeded
|
||
announcement rule points at `notify.event` instead, where the projection gets it right.
|
||
|
||
Reconciling the two declarations is a **variable rename, which §4.3 makes a version bump**, and this
|
||
phase did not take that on its own authority. A test pins the current shape so that reconciling it is a
|
||
deliberate act rather than a silent rename that empties somebody's subject line.
|
||
|
||
##### What an operator sees at cutover, which is the whole of decision 3
|
||
|
||
Core seeds four rules — one per Team trigger — all `enabled = 0`, all `audience: 'members'`, all on the
|
||
email channel. **Team email is off until an operator switches one on.** Three things make that
|
||
survivable rather than a silent regression:
|
||
|
||
- **Admin → Engagement → Rules carries a banner** whenever every Team rule is off, saying that these
|
||
used to send automatically, that they arrived switched off so nothing starts mailing on its own, and
|
||
that per-member preferences and unsubscribe links still work above them. It reads the rules rather
|
||
than a flag, so it disappears the moment one is enabled and returns if they are all switched off
|
||
again. A deployment that deleted them sees nothing, which is right — they made that choice.
|
||
- **The release note names it**, alongside the v1-token narrowing.
|
||
- **The push tickle and the Discord bridge are unaffected.** Only email moved, so the app and the
|
||
Discord channel keep working exactly as before while the rules are off.
|
||
|
||
The rules are seeded **once**, guarded by a settings key rather than ensured on every boot: an operator
|
||
who deletes a rule must not find it back after a restart, and one they enabled must not be reset to
|
||
off. The guard is stamped even after a partial run — re-running would duplicate the rules that did
|
||
insert, and a duplicate rule is two mails per event, which is worse than one missing rule an operator
|
||
can add from the screen.
|
||
|
||
##### What was verified
|
||
|
||
- **97 tests**: 35 in `teamNotifyDispatch.test.js` (rewritten — the mail-body assertions moved down to
|
||
the channel and what is asserted here is now the envelope), 31 in the new `engagementEmail.test.js`,
|
||
23 in `teamNotify.test.js`, plus the mailer, engine and block-renderer additions. Server suite **1447**
|
||
green, client **324** green.
|
||
- **A live rig**, which is where three of the findings above came from: MariaDB + Mailpit + a booted
|
||
server + a real Team with three members. The walk covered an instant mail with both
|
||
`List-Unsubscribe` headers, a digest gathering two posts and stamping `engagement_digest_state`, the
|
||
generic `notify.event` path rendering an announcement with no authoring, a pre-migration v1 token
|
||
unsubscribing through the old path, and a restart proving the backfill replay-safe.
|
||
- **The route manifest and swagger** carry the two new routes; the two legacy ones are unchanged.
|
||
|
||
**Still later phases':** the push and in-app channels' `deliver` (Phase 7), whether an unverified
|
||
address may receive opt-in mail (§7.1 Q1's narrower half, Phase 9), and the `address_hash` this phase
|
||
started writing, which is the column Phase 9's bounce correlation reads.
|
||
|
||
---
|
||
|
||
### Phase 7 — The in-app channel (core + web) ✅
|
||
|
||
`user_notifications`, the in-app `DeliveryChannel`, `GET /auth/me/notifications` + mark-read, and the web
|
||
surface (bell + list). Push tickles gain a `ref` that deep-links into the inbox.
|
||
|
||
**Acceptance:** one event delivered to `inapp` produces exactly one row; a duplicate `dedupeKey` is a
|
||
no-op; mark-read is idempotent; a user cannot read another user's row (asserted at the route, not only
|
||
the model); `url` is relative-only, validated by the same character-class rule `pageUrlTemplate` uses.
|
||
**Guardrails:** swagger + route manifest; the sanitize path for `body`.
|
||
|
||
#### As built — 7 (2026-08-31)
|
||
|
||
The third channel gets behaviour, the oldest one gets a `deliver` at last, and the preferences endpoint
|
||
Phase 3 shipped with no surface gets one. **Four decisions were settled by the org lead before any
|
||
code**, two of them widening the phase past its own acceptance line.
|
||
|
||
##### Decision 1 — `inapp` defaults to `instant`, and it is the only channel that does
|
||
|
||
`coreChannels.js` deferred this in as many words: "whether the inbox is opt-out once it is real is a
|
||
Phase 7 decision with a live surface to look at." The surface exists now, and the answer is opt-OUT.
|
||
|
||
The argument for opt-IN was never about in-app. §7.1 Q1 is standard marketing-email practice and Phase
|
||
3's `push` default is about a device somebody is holding; **an inbox item wakes nothing and leaves
|
||
nothing** — it is a row on a page the user chose to open, on this deployment, costing one glance. Left
|
||
at `off` the channel would ship dead: no rule could reach anybody until every user found a toggle for a
|
||
channel they had never seen deliver anything. The backlog Phase 3 worried about cannot happen either —
|
||
the table is empty at cutover, rules default to `enabled = 0`, and every rule carries a per-hour
|
||
ceiling.
|
||
|
||
##### Decision 2 — the phase takes the two pieces its acceptance line omitted
|
||
|
||
Two earlier phases assigned work here that Phase 7's own bullets never mention, and both were taken:
|
||
|
||
- **`push` gets its `deliver`** (§2603's "the push and in-app channels' `deliver`"). Without it a rule
|
||
naming push still finished `failed` in the send log — the oldest sink in the system, unreachable from
|
||
the engine. It is the channel that got behaviour last because until the inbox existed there was
|
||
nothing for a content-free tickle to point at.
|
||
- **The web per-channel preferences screen** (Phase 3's as-built: "the screens are Phase 7 (web) and
|
||
Phase 8 (app)"). The endpoint had shipped with no consumer on either platform.
|
||
|
||
##### Decision 3 — the inbox takes `/notifications`; the preferences move under it
|
||
|
||
`/auth/me/notifications/*` was already the preferences namespace — `streams`, `subscriptions`,
|
||
`channels`, `teams` — and `/account/notifications` was already the preferences *page*, with a bell icon
|
||
in the portal nav. Content and settings are different kinds of thing, and **the plain word belongs to
|
||
the content**: it is what a person means when they say "notifications", and what the bell opens.
|
||
|
||
So the inbox is `GET /auth/me/notifications` and the page is `/account/notifications`; the preferences
|
||
screen moved to `/account/notifications/settings` and gained its own nav row. Route order is not
|
||
incidental and is commented as such: the four named preference sub-paths are declared above, and the one
|
||
parameterised path added below them is a **POST** whose `:id` is digits-only, so nothing can shadow
|
||
`streams` or `channels`.
|
||
|
||
##### Decision 4 — `ctx.inbox.push` respects a preference where one exists
|
||
|
||
The rule-less sink has no trigger declaration to project from, no rule to pick a template and no
|
||
audience to resolve. It now writes the inbox directly **unless** `triggerId` names a *registered*
|
||
trigger and that user's effective `inapp` mode is not `instant`: a toggle somebody switched off must not
|
||
be walkable around by the module that owns the trigger behind it. An id nothing has registered has no
|
||
toggle on any screen, so there is no preference to protect and the item is written.
|
||
|
||
Scoped preferences are deliberately not consulted — a scope is a property of an *event* (`team:12`), and
|
||
a caller with no declaration has no scope to name. The engine's path, which does, still applies them.
|
||
|
||
##### The block → column mapping, which is the whole of how a template becomes a row
|
||
|
||
`user_notifications` has `title` / `body` / `url` where email has a subject and a document. The in-app
|
||
renderer (`templates.renderInappByKey`) maps by block **role**: the first `email.heading` is the title,
|
||
the first `email.button` is the url, everything else is the body. A second heading or button is ordinary
|
||
body content, which is what an operator who added one meant.
|
||
|
||
**The body is TEXT, not the email HTML**, and that is load-bearing rather than a shortcut. The `email.*`
|
||
renderer produces markup built for mail clients — table rows, inline hex colours, a light-only palette
|
||
declared with `color-scheme` — which dropped into a page that follows the viewer's theme renders as a
|
||
pale card floating in a dark one. `toText` is the same content with none of that, and it is the part
|
||
every block already promises. The consequence worth stating: **there is no operator markup on this
|
||
surface to sanitize, and no way for one to appear.** The phase's "sanitize path for `body`" guardrail is
|
||
discharged by the column never holding markup in the first place, which is a stronger guarantee than a
|
||
sanitizer.
|
||
|
||
##### Five things the tree contradicted, or the build found
|
||
|
||
- **The shipped `inapp.event` seed named variables nothing supplies.** Phase 5a wrote it before the
|
||
channel that renders it existed, declaring `body` and `url` — but a trigger declares domain names
|
||
(`teamName`, `threadTitle`) and `projection.project` fills the gaps with the *structural* ones
|
||
(`title`, `intro`, `actionUrl`). Every rendering would have produced a title and nothing else.
|
||
Renamed to `notify.event`'s vocabulary at **`seedVersion` 2**, which is §4.6.1 property 1 restated for
|
||
this channel: a new trigger must render with no authoring at all.
|
||
- **The dedupe index is scoped to the USER, which is narrower than the outbox's.** `engagement_outbox`
|
||
scopes to `(rule, user, channel)` because one event legitimately becomes one row per channel; an inbox
|
||
has no channel dimension, so two rows for one event would be **one item shown twice**. Same family of
|
||
defect as the global index Phase 4a found in §4.2a, in the opposite direction.
|
||
- **The push tickle's `ref` needed an ordering to be worth anything.** A rule spanning `inapp` and
|
||
`push` enqueues two independent rows and the outbox sweeps `ORDER BY due_at, id`, so the ref only
|
||
resolves if the in-app row was enqueued first. `engine.liveChannels` now sorts `inapp` ahead of the
|
||
rest (`CHANNEL_ORDER`) — an ordering, not a dependency: the ref is a **hint**, null when there is no
|
||
row, and the app's contract stays wake-and-pull.
|
||
- **There was no retention policy for this table at all**, and neither the outbox nor the send log
|
||
bounds it (both hold one row per *delivery*; an inbox item outlives its delivery by design).
|
||
`utils/userNotificationsPrune.js` is `teamActivityPrune`'s shape with one policy difference:
|
||
**read items only.** Age alone would delete the evidence for "I was never told", which is the
|
||
complaint this table answers. The horizon is `settings.user_notifications_retain_days`, default 90.
|
||
- **Staff had no reachable inbox, and only the live rig could see it.** `/auth/me/notifications` is
|
||
role-agnostic — behind `requireAuth` only, like every `/auth/me` route — so the server, the tests
|
||
and the API all agreed a staff member had an inbox. On the web they did not: `RequirePlayer` sends
|
||
anyone who is not a player out of `/account` (staff manage their own account under `/admin/account`),
|
||
so the bell pointed at a page that redirects. **Signed in as an admin, the feature was unreachable.**
|
||
Fixed by mounting the same two components at `/admin/notifications` and
|
||
`/admin/notifications/settings`, adding the bell to the admin header, and putting the one mapping in
|
||
`client/src/lib/notificationPaths.js` with its own test. One trap inside the fix worth keeping:
|
||
`allowedPathsFor` turns an `end: true` nav row into an EXACT match, so marking the admin row exact
|
||
left `/admin/notifications/settings` outside the allowlist and bounced staff off their own
|
||
preferences screen — the row has to cover its sub-routes.
|
||
|
||
##### What an operator and a user actually see
|
||
|
||
- **The bell** sits in the public site header and in the player portal's own header, renders nothing
|
||
when signed out, and **polls** its badge once a minute — pausing while the tab is hidden and
|
||
refreshing the moment it comes back. There is nothing to push over: the site's two SSE streams are
|
||
the shard's, neither is per-user, and a third authenticated stream carrying one integer would mean an
|
||
open connection per signed-in tab forever.
|
||
- **The preferences screen is now a matrix**, not a checkbox list. The push-only stream list it replaced
|
||
was a strict subset: `/notifications/channels` already returns every push stream *and* every event
|
||
trigger with the effective mode on each channel that applies, so a trigger-only id simply has no push
|
||
cell and core never has to explain which kind of id a row is. The two legacy whole-set endpoints are
|
||
untouched and are that surface's push projection, so **the shipped Android app keeps its wire shape**.
|
||
|
||
##### What was verified
|
||
|
||
- **28 new tests**: 23 in `engagementInapp.test.js` (the five acceptance criteria, the role mapping, the
|
||
four `ctx.inbox.push` cases, the tickle's exact key set, and the route-level ownership check) and 5 in
|
||
`userNotificationsSql.test.js` — a throwaway MariaDB, because three properties here are a *server*
|
||
contract rather than a reading of this code: a UNIQUE index admitting many NULLs, `INSERT IGNORE`
|
||
reporting `affectedRows = 0` on a duplicate, and `read_at IS NULL` making mark-read idempotent.
|
||
- **Two existing tests moved with the behaviour, and both moves are the point.**
|
||
`engagementEngine`'s "a channel with no `deliver()` finishes failed" named `inapp` (and `email` before
|
||
it) and so was rewritten by every phase that gave a channel behaviour; it now registers a throwaway
|
||
channel, because the property was never about a particular one. `notificationChannelPrefs`'s defaults
|
||
assertion carries decision 1.
|
||
- **Swagger and the route manifest** carry the four new routes, with four new component schemas.
|
||
- **A live rig**: MariaDB + a booted server + the real outbox worker + a browser. The walk is where
|
||
the staff-reachability defect came from, and it also proved the three things unit tests cannot —
|
||
that `liveChannels`' ordering really does put the in-app row first (a rule stored as
|
||
`["push","inapp"]` enqueued outbox 2 = inapp before outbox 3 = push, and the tickle carried
|
||
`ref: "notification:2"`); that **two rules on one event produce three outbox rows and exactly ONE
|
||
inbox item**, with the send log saying "already in this inbox (duplicate dedupe key)" rather than
|
||
claiming a second delivery; and that the retention worker drops an aged READ row while leaving an
|
||
equally aged UNREAD one. The preferences matrix wrote exactly one row for the one cell that changed.
|
||
|
||
**One thing this phase did NOT wire, and it is worth knowing before Phase 11.** `news.post` is a
|
||
declared trigger that **nothing emits through the engine** — `coreTriggers.js` says so in as many words
|
||
("these declare; nothing here emits yet") and Phase 6 migrated only the four `team.*` ones, so the
|
||
admin publish path still fires a raw `pushDispatch.publish` beside the engine rather than through it.
|
||
The consequence for this phase: on a real deployment the only in-app items a rule can produce today
|
||
come from the four Team triggers. Wiring the news emitter is a one-line `ctx.events.emit`-shaped change
|
||
that belongs with whoever owns that decision, not smuggled into the channel's own phase. **Written up as
|
||
§7.1 Q9**, which sets out the three other things a publish already fires (an announce leg, a module's
|
||
post hook, the raw tickle), which of them the engine replaces and which it must not touch, and the
|
||
continuity question that has to be answered before anyone writes the line. Recommended home: Phase 11.
|
||
|
||
**Still later phases':** the app's inbox screen and the tickle → pull → inbox path (Phase 8), and the
|
||
suppression list, which this channel has no equivalent of — there is no address to suppress.
|
||
|
||
---
|
||
|
||
### Phase 8 — The in-app channel (Android) ✅
|
||
|
||
Inbox screen, unread badge, and the tickle → pull → inbox path. App-store cadence, separate repo,
|
||
separate release.
|
||
|
||
**Acceptance:** a tickle wakes the app, which pulls and shows the item; the inbox works offline from
|
||
cache; the existing preferences screen gains the per-channel toggles from Phase 3.
|
||
**Note:** `android-app`'s `pr-checks.yml` triggers only on PRs into `main`, so phase PRs onto a working
|
||
branch get **no CI** — the cutover PR is the first real run. Plan for that. **Resolved: §7.1 Q8 was
|
||
answered "fix it" and the trigger now runs on PRs into `edge` too, as this phase's first commit.**
|
||
|
||
#### As built — 8 (2026-08-31)
|
||
|
||
The four inbox routes Phase 7 shipped had no consumer on either platform; this is the Android one.
|
||
**Four decisions were settled by the org lead before any code**, and one of them resolves a
|
||
contradiction the phase carried from the day it was written.
|
||
|
||
1. **§7.1 Q8 — fix the CI trigger, as the first commit.** `pr-checks.yml` now runs on PRs into
|
||
`edge` as well as `main`. All nine M12 phase PRs merged with no CI at all and engagement Phase 8
|
||
was about to repeat it; the alternative was finding a Kotlin compile error inside the cutover
|
||
window with a whole workstream's diff to bisect. `sonarqube.yml` is untouched — it is a
|
||
push-on-`main` analysis, not a PR gate, so no phase PR was ever expected to run it.
|
||
2. **The offline snapshot is a JSON blob, not Room.** This phase's acceptance line ("the inbox works
|
||
offline from cache") **contradicted a decision already recorded in `docs/android/PLAN.md` §7**:
|
||
*"Offline caching is not a v1 requirement (decided)… does not ship a Room cache in v1. No `Room`
|
||
dependency in the initial build."* §7's decision stands and this is its one named exception. An
|
||
inbox is a short, read-only, newest-first list with a server-side cursor and no joins, so what
|
||
"works offline" needs is the newest page and the badge — one blob in the DataStore the push code
|
||
already uses, capped at the server's own default page size.
|
||
3. **The drawer's "Notifications" is the inbox; the settings are behind its gear.** Exactly the
|
||
arrangement Phase 7 shipped on the web (the bare path is the content, `…/settings` the
|
||
preferences), and what a person means when they tap the word.
|
||
4. **The settings screen moved onto `/notifications/channels`**, rendering a control per channel that
|
||
applies to each id and per mode that channel accepts.
|
||
|
||
**The one thing the app must do that no plan said: resolve the item's `url`.** Phase 7's acceptance
|
||
specifies `url` is **relative-only** and validates it as such — right for a browser already on the
|
||
site, and a dead link on a phone. The live rig caught it: every item came back as
|
||
`/guilds/the-silver-anvil/forum/403`, and the app's first cut only opened `http(s)`-prefixed strings,
|
||
so **every link in the inbox did nothing**. It now resolves against the configured base with OkHttp's
|
||
`HttpUrl.resolve`, which absolutises the path *and* returns null for anything that would not end up
|
||
http(s) — so a `javascript:` or `intent:` url in a notification body opens nothing at all. The
|
||
relative-only contract did not change; the client half of it was simply never written down.
|
||
|
||
**Three further things the build settled:**
|
||
|
||
- **The tickle's `ref` is what routes it, not its stream.** An engagement rule's tickle carries the
|
||
TRIGGER id as `stream` (§7.2's one namespace), and the app's shipped `forStream` map knows only the
|
||
eight push streams — so `team.forum.post` would have landed on Home. `Routes.forTickle(stream, ref)`
|
||
sends anything whose ref starts with `notification:` to the inbox and leaves every other tickle on
|
||
the route it has always had. The ref is not decoded past that prefix and is never rendered: it is a
|
||
hint that a row exists, and the contract stays wake-and-pull, exactly as `pushChannel.js` says.
|
||
- **The snapshot is scoped to (base URL, user id), and that is the security property** — not the
|
||
clear-on-logout beside the push deregistration. A cache is only ever handed back to the pair that
|
||
wrote it, so the teardown paths that never reach a logout at all (a dead refresh token, a server
|
||
switch) cannot surface one person's notifications under another's session.
|
||
- **Reads are optimistic and are not rolled back.** The row flips locally, the server's post-write
|
||
`unread` replaces the local guess, and a failure is left alone: un-reading a row under someone's
|
||
finger looks like a bug, and the next refresh corrects it. A local read also rewrites the snapshot,
|
||
without which going offline right after reading everything would bring the badge back on the next
|
||
cold open.
|
||
|
||
**Verified on the live rig** (throwaway emulator + the real server on `edge` + real rows emitted
|
||
through `ctx.events.emit` → engine → outbox → `inappChannel`, never hand-written):
|
||
|
||
- the drawer badge showed **3**, the inbox listed the three items newest-first with unread dots and
|
||
local-zone timestamps, and a tap marked one read (3 → 2, server-side row updated);
|
||
- tapping an item opened its resolved link in a Custom Tab at the configured shard;
|
||
- one email chip wrote **exactly one row** — `(35, news.post, email, instant)` — and disturbed nothing
|
||
else, which is the sparse PUT's whole point;
|
||
- the push controls were **absent**, correctly, because the rig's shard advertises no relay, with the
|
||
reason in a note beside the list rather than replacing the screen (email and on-site preferences are
|
||
still worth setting on a shard with no push);
|
||
- with the network cut, the inbox rendered all five items from the snapshot under **"Offline — showing
|
||
what was saved on this device."**, unread count and read flags intact;
|
||
- a simulated tapped tickle (`STREAM=team.forum.post`, `REF=notification:5` — the exact extras
|
||
`PushNotifier` builds) landed on the inbox rather than Home.
|
||
|
||
**One finding this phase did NOT fix, deliberately.** *Offline works in a running app, not on a cold
|
||
start* — and the reason is the app shell, not the inbox. `MainActivity` gates the whole of `RunicApp`
|
||
on loading the site's appearance (M12), so an offline launch shows "Can't reach the site / Retry" and
|
||
never reaches the drawer at all. Widening this phase into the shell's startup model is the kind of
|
||
out-of-scope blocker that is a reason to ask, not a licence to widen the PR, so it is written up here
|
||
for the org lead. The acceptance line holds for the case the cache exists to serve — the app you were
|
||
just using, on a train — and fails for the case where the app was killed first.
|
||
|
||
**Still later phases':** nothing of this channel. The app now consumes every route Phase 7 built.
|
||
|
||
---
|
||
|
||
### Phase 9 — Deliverability: suppression and bounces ✅
|
||
|
||
`engagement_suppressions` and bounce/complaint capture per transport (SMTP has none — this is where the
|
||
API-based providers earn their place).
|
||
|
||
**The verification flow is no longer part of this phase.** §7.1 Q1's answer moved it forward into
|
||
**Phase 1b**, along with the admin gate setting that decides whether an unverified address is excluded.
|
||
Phase 9 therefore *consumes* `email_verified` rather than introducing a writer for it, and — the reason
|
||
this reshuffle is worth it — **Phase 9 no longer blocks Phase 11.** The first real rule can ship on the
|
||
verification mechanism 1b already built, with bounces following.
|
||
|
||
**Acceptance:** a suppressed address is skipped with `status='suppressed'` in `engagement_sends` and no
|
||
transport call; a hard bounce suppresses the address; with the Phase 1b gate `on`, an unverified address
|
||
is excluded from engagement rules but still receives password resets; with it `off`, it receives both.
|
||
|
||
#### As built (2026-08-31)
|
||
|
||
**Four decisions, settled by the org lead before any code:**
|
||
|
||
1. **Mechanism plus SMTP's real signal; no API transport.** The phase's own text says "SMTP has none —
|
||
this is where the API-based providers earn their place", and that is too strong. SMTP has no
|
||
*asynchronous* bounce or complaint feed, but a single-recipient send refused at `RCPT TO` throws
|
||
synchronously with the reply code intact — the highest-value deliverability signal there is, and
|
||
`mailer.js` was already catching it as a `PERMANENT_CODE` and throwing it away. So this phase reads
|
||
it. A transport MAY declare a bounce handler; none does, and **no webhook route ships** — a route
|
||
with no producer is §7.1 Q9's problem in a different costume.
|
||
2. **Suppression scopes to engagement rules only.** Password resets, invites, verification mail and
|
||
the contact form still attempt. This is the posture `passwordReset.controller.js` already stated
|
||
for the verification gate, and the argument carries: user-initiated mail must not be blocked by a
|
||
background system's opinion, and one reset to a dead mailbox is not a reputation problem.
|
||
3. **`address_masked` is added to §4.5's DDL.** The hash-only table cannot be operated — an operator
|
||
staring at sha256 digests cannot tell three typos from a whole domain refusing mail, and
|
||
un-suppressing somebody who fixed their mailbox is the one action the table must support.
|
||
4. **The verification gate filters at ENQUEUE, not at delivery.**
|
||
|
||
**The defect this phase exists to have avoided: `PERMANENT_CODES` is not a bounce classifier.**
|
||
The obvious implementation is "the mailer already tells us a failure is terminal, so suppress on
|
||
that". `mailer.PERMANENT_CODES` is `{550, 553, 554, EENVELOPE, EAUTH}`, and it answers a different
|
||
question — *is retrying pointless?* `EAUTH` is the operator's password being wrong and `554` is a
|
||
relay-wide policy refusal; neither says anything about the recipient. Under that implementation **one
|
||
stale SMTP credential suppresses every address the outbox worker touches**, with a clean send log, no
|
||
warning, and a mailing list that has to be rebuilt by hand. So `bounceClassify.js` is its own judge:
|
||
|
||
- the **RFC 3463 enhanced status** decides on its own where there is one — `5.1.1`, `5.1.2`, `5.1.3`,
|
||
`5.1.6`, `5.1.10` and `5.2.1` suppress, and `5.3.x`, `5.5.x` and `5.7.x` explicitly never do, being
|
||
about the server or about our standing with it;
|
||
- without one, a phrase match applies **only** after `550`/`551`/`553` has already narrowed the
|
||
failure to the recipient address, and only past a veto list — `552` and `554` are excluded from even
|
||
that, because a full mailbox gets emptied and "transaction failed" is what a relay says when it does
|
||
not want to say why;
|
||
- **anything it is unsure about is not suppressed.** A false negative costs one retry next month; a
|
||
false positive costs a person who silently stops hearing from the deployment and cannot find out.
|
||
|
||
`sendNotification` now returns an `smtp: { code, responseCode, response }` triple so this is
|
||
answerable at all: `retry` and `detail` cannot distinguish `550 5.1.1` from `550 5.7.1`, which are an
|
||
identical `retry: false` and mean completely different things.
|
||
|
||
**The two mechanisms sit at different points, and the split is the design.** A suppression can appear
|
||
inside a rule's `delay_seconds` grace window, so the only correct check is the one taken immediately
|
||
before the transport call — which is also what produces the `status='suppressed'` row with no
|
||
transport call that the acceptance line asks for. Being unverified is a *standing* property, stable
|
||
across that window, so excluding at delivery would write an outbox row purely to throw it away — and
|
||
on a deployment that upgraded before verifying anybody, one rule firing would write thousands of
|
||
`suppressed` rows nobody can read.
|
||
|
||
**The gate is a channel hook, not an engine branch.** It hangs off a new optional
|
||
`registerDeliveryChannel({ eligible })`, and `email` is the only channel that declares one. Both
|
||
alternatives were wrong in a way the build made obvious: filtering the shared audience before the
|
||
per-channel loop silences the wrong sink — **a rule spanning email and in-app must still put an item
|
||
in an unverified user's inbox**, since being unverified is a reason not to mail somebody and no reason
|
||
at all to hide their notifications — and an `if (channel === 'email')` in `engine.js` puts one
|
||
channel's rule inside the generic engine. Its excluded counts flow into `summary.ineligible` and into
|
||
the admin reach preview, which until now reported an audience size that was never the number of people
|
||
who would get a mail.
|
||
|
||
**Both new checks fail OPEN**, and the `try/catch` in `eligible` is load-bearing rather than habit:
|
||
`applyRule` awaits it *before* the per-user loop, so an uncaught throw abandons the whole rule for
|
||
every channel it names — a rule that silently sent nothing, with a clean log and an empty outbox. That
|
||
is G22's shape again, and the recoverable mistake is mail going out.
|
||
|
||
**The live rig found the one defect the stubs could not.** Against a real MariaDB, a real SMTP
|
||
conversation (mailpit) and the real engine and worker, a hard bounce was recorded as `status='failed'`
|
||
— honest, but `engagement_sends.status` has carried **`bounced`** since §4.5 and nothing had ever
|
||
written it, so the Send Log's "Bounced" filter matched nothing and always would have. It is now a
|
||
distinct outcome, because "the relay would not take this" and "this mailbox does not exist" send an
|
||
operator to two different places. `engagement_outbox.status` still records it as `failed`: that ENUM
|
||
has no `bounced`, and from the queue's point of view a bounced row is one that finished
|
||
unsuccessfully. `complained` still has no writer, and cannot have one without a provider feedback loop.
|
||
|
||
**Verified on the live rig**, seven rungs, each one a real send or a real refusal:
|
||
|
||
1. baseline — three recipients, three mails in mailpit, three `sent` rows;
|
||
2. one address suppressed by hand — **two** mails, and a `suppressed` row naming the reason;
|
||
3. suppression lifted — three mails again;
|
||
4. gate `on` — two mails, `ineligible: { unverified: 1 }`, and **no send-log row at all** for the
|
||
excluded user, which is what enqueue-time exclusion means;
|
||
5. a `550 5.1.1` from the relay — two mails, one `bounced` row, and `b***@example.test` written to the
|
||
suppression list with `detail: hard bounce: 5.1.1`;
|
||
6. the same rule again — that address `suppressed`, with no transport call;
|
||
7. **a password reset to the suppressed address still arrived**, which is decision 2 proved rather
|
||
than asserted.
|
||
|
||
The three admin routes were then driven over real HTTP: `created_by` records the admin on a manual row
|
||
and stays **NULL** on the worker's automatic one, which is what separates them in the list; a repeat
|
||
POST answers `200 {created:false}` rather than a 409; DELETE matches case-insensitively, which is the
|
||
whole point of hashing a folded address; no response contains `address_hash`; and all three refuse a
|
||
signed-out caller.
|
||
|
||
**Still later phases':** an API-based transport with a signed webhook receiver, which is what would
|
||
bring asynchronous bounces and `complained` to life. Nothing here blocks it — a transport may declare
|
||
a bounce handler today.
|
||
|
||
---
|
||
|
||
### Phase 10 — Protocol bump: three wire enrichments ✅ *(parallel from day one)*
|
||
|
||
**Scope widened 2026-08-31, by the org lead, before any code.** The phase was written as the
|
||
`house.decay` enrichment alone. It now carries **three** enrichments, for one reason that is a property
|
||
of this phase and of no other: **a protocol bump costs a sidecar release, a republished bundle and an
|
||
operator update on every shard.** A field left out here does not cost a follow-up commit — it costs a
|
||
*second* bump with the same three-part lead time, and an operator population split across two protocol
|
||
versions in the meantime. So everything the engagement workstream is known to need from the wire is
|
||
decided **now** and rides the single v4 → v5 bump.
|
||
|
||
The two additions come from Phase 11's expanded trigger set (below). Each was verified against the
|
||
emitters rather than assumed, and each closes a gap that §8.6 did not know it had:
|
||
|
||
- **Player-vendor fee state.** `uo.vendor.expiring` is the same shape as the flagship — an owned asset
|
||
at risk, with a real deadline — and is the second-strongest driver in the catalogue. But
|
||
`vendor.listing` carries `ownerSerial` and `ownerName` and **no `ownerAcct` and no fee state at
|
||
all**, so the trigger can today neither be addressed to a person nor know that anything is expiring.
|
||
- **A login *outcome*.** `EventSink.AccountLogin` is a pre-decision veto hook — the emitter's own
|
||
comment says so ("Fires before the auth decision, so this is an attempt, not a result"). It therefore
|
||
fires on every **successful** login too, and `uo.account.login_attempt` as §8.6 describes it
|
||
("someone tried to log into your game account") cannot be built on it: it would mail a security alert
|
||
every time the player themselves logged in. This is the one addition that is a new emitter rather
|
||
than new fields on an existing one.
|
||
|
||
**Explicitly NOT in this bump:** moving `vendor.sale` out of the opt-in patch tier. It stays where it
|
||
is (see §8.6's corrected row), which leaves `uo.vendor.sale` dormant on a shard that declined the
|
||
patches — a documentation obligation, not a protocol one.
|
||
|
||
| Repo | Change |
|
||
| --- | --- |
|
||
| `servuo-plugins/` | **(a)** `BridgeSweeps.WriteDecay` gains `ownerName` (`house.Owner.Name`), `nextStage` (`BaseHouse.NextDecayStage`), `decayPeriod`, and `estimatedCollapse` **only when `to == "IDOC"`** (§0.3) · **(b)** `BridgeMarket`'s `vendor.listing` gains `ownerAcct`, `holdGold`, `chargePerDay`, `daysRemaining` · **(c)** a new post-decision `account.login.result` (`acct`, `accepted`, `rejectReason`, `ip`) beside the existing pre-decision attempt |
|
||
| `servuo-plugins/overlay.toml` | `protocol = 4` → `5`, **in the same PR as the emitters** |
|
||
| `link/` | `PROTOCOL_VERSION: u32 = 4` → `5` (`sidecar/src/main.rs:55`); the houses board carries the new decay fields; `account.login.result` is a forwarded kind like any other — the sidecar stays a dumb forwarder |
|
||
| `website/` (`module-uo`) | ingest maps the new fields; `uoLinkConfig` protocol version; **`shardVisibility.js`'s `KIND_FEATURE` gains `account.login.result`** — rule 2 fails an unmapped kind closed to admin-only, which is the right answer for a frame carrying an IP, but it must be *chosen* here rather than inherited by accident |
|
||
| `docs/` | `docs/link/INTEGRATION.md` §Housing and §Market tables + examples, `docs/link/PLAN.md` §5/§7 |
|
||
|
||
**Two things the emitter work must not get wrong**, both following from §0.3's finding about how
|
||
ServUO actually decays:
|
||
|
||
- `estimatedCollapse` is exact **only** at IDOC, because dynamic decay draws each stage's duration at
|
||
random when the stage is entered. Emitting it at an earlier stage would publish a guess as a fact.
|
||
- `daysRemaining` for a vendor **is** exact (`HoldGold / ChargePerDay`), unlike the house. The two
|
||
fields must not be documented as though they carried the same confidence — a template that says
|
||
"your house collapses on the 4th" and one that says "your vendor is dismissed in 2 days" are making
|
||
very different promises.
|
||
|
||
**Acceptance:** a live run on the local rig (`C:\Users\colby\Desktop\ServUO` + the Rust sidecar, not
|
||
the PowerShell stub) shows **(a)** a real decay transition carrying the new fields, **(b)** a seeded
|
||
player vendor whose listing carries a fee state that falls as its held gold is drawn down, and **(c)** a
|
||
*failed* game login producing `account.login.result accepted:false` where a successful one produces
|
||
`accepted:true`; a v4 overlay paired with a v5 sidecar is **refused by the installer**, not mis-parsed;
|
||
CI publishes a bundle whose manifest pairs v5 with v5; and the five-rung shard visibility walk still
|
||
shows no leak, with `account.login.result` reaching **admin only**.
|
||
**Note:** this phase's lead time is a release plus a bundle plus an operator update, which is why it
|
||
starts early and lands independently.
|
||
|
||
---
|
||
|
||
#### Phase 10 as built (2026-08-31)
|
||
|
||
**Design of record: [`../link/v5.md`](../link/v5.md).** All three enrichments shipped as scoped,
|
||
and the phase found nothing that changed its shape — which is itself worth noting, because the two
|
||
*additions* to this phase existed only because reading the emitters had already contradicted §8.6
|
||
twice.
|
||
|
||
**Two design decisions taken during the build, neither of them in the phase text:**
|
||
|
||
- **Both new field groups are NESTED** (`house.decay.schedule`, `vendor.listing.fees`) rather than
|
||
eleven sibling keys. The visibility projection matches literal JSON keys, so a nested group is
|
||
ONE admin rule that governs the whole group — and, more importantly, a field added inside it
|
||
later inherits the gate instead of defaulting to visible. Eleven flat keys would have been eleven
|
||
rules to keep in step. There is a test that adds an imaginary future fee field and asserts it is
|
||
still gated.
|
||
- **There is no `daysRemaining`.** The obvious name is wrong: under ServUO's old vendor system a pay
|
||
period is a UO day (`Clock.MinutesPerUODay`, ≈2 real hours), so the field would have been off by
|
||
a factor of twelve on exactly the shards least likely to notice. The wire carries
|
||
`periodsRemaining` plus the interval, and resolves the arithmetic into `dismissalAt` — an instant,
|
||
which needs no units.
|
||
|
||
**Three visibility classifications, each chosen rather than inherited** (v5.md §4): the decay
|
||
`schedule` is `anonymous` (the countdown is the public IDOC page's content), `fees` is **`admin`** —
|
||
the only default in the `market` feature that does not reproduce prior behaviour, because a vendor's
|
||
held gold is owner-only in game and publishing it is both a new disclosure and a targeting aid — and
|
||
`account.login.result` is admin-only **by omission**, since `KIND_FEATURE` is the map of kinds an
|
||
admin may widen and there is no rung below admin an IP-plus-auth-verdict belongs on.
|
||
|
||
**The REST reads deliberately did not change.** `shard_vendors`' eight new columns are in no read
|
||
model's column list: they exist for Phase 11's server-side trigger and reach no client at all.
|
||
|
||
**One improvement to the pin, which is the protocol-4 bug seen from the other side.** `module-uo`'s
|
||
schema test asserted `DEFAULT 4` at each declaration site — so when the emitters moved and one site
|
||
was left behind, every site still agreed with itself and the test passed. It now reads
|
||
`DEFAULT_PROTOCOL` from the model, making the assertion "the declarations **agree**" rather than
|
||
"they all say 4", and the one-shot migration test is written once against the current version
|
||
instead of being hand-copied per bump.
|
||
|
||
**Acceptance, all met.** 470 tests in `module-uo/server` (16 new) and 39 in the sidecar; the C#
|
||
compiles against real ServUO 57.4 assemblies. On the live rig — real ServUO, seeded world, the
|
||
**release** sidecar — a house walked Fairly → Greatly → IDOC produced `estimatedCollapse` on the
|
||
IDOC frame **and only there**; every observed vendor's `periodsRemaining` matched
|
||
`funds / chargePerPeriod`, including one at `0` whose `dismissalAt` equals its next tick; and a real
|
||
socket login produced `accepted:false reason:BadPass` for a wrong password and `accepted:true` for
|
||
the right one. **The installer needed no change** — its pairing refusal is version-agnostic and
|
||
already tested (`bundle.rs::a_protocol_disagreement_inside_one_bundle_is_refused`).
|
||
|
||
**Two rig traps, both of which fake a broken emitter by producing silence** (v5.md §6.1): an
|
||
in-process login probe can never produce `accepted:true`, because `AccountHandler` calls
|
||
`HasAccess(e.State)` before it checks the password and a null `NetState` fails it; and forcing a
|
||
decay stage on a house that cannot decay emits **nothing at all**, because an `AutoRefresh` house —
|
||
and the owner's newest house is always `AutoRefresh` — resets the forced stage on the next read.
|
||
Both are recorded in `servuo-plugins/tools/scaffolding/README.md` beside the new
|
||
`BridgeProtocol5Probe.cs`.
|
||
|
||
**Deferred, deliberately:** `runicgateway.com`'s `platform.json.protocol` → 5 stays Phase 12's, since
|
||
`checkFacts.mjs` fetches from `main` and setting it during the `edge` period turns that repo red at
|
||
once.
|
||
|
||
---
|
||
|
||
### Phase 11 — module-uo's triggers: the full catalogue, and core's `news.post` ✅
|
||
|
||
**Scope widened 2026-08-31, by the org lead, before any code.** This phase used to say `module-uo`
|
||
registers `uo.house.idoc_warning` "(and siblings)", and Part 8's preamble used to say "Phase 11 ships
|
||
exactly one rule". Both are now wrong on purpose: **Phase 11 ships every ✅ row in §8.6**, with the
|
||
carve-outs named below. §8.6 was written as "a catalogue, not a commitment" — it is now the commitment,
|
||
and the burden of proof has inverted: a row that does *not* ship needs a reason recorded here. **Four
|
||
rows do not ship, and all four reasons are recorded** — one in this section (`uo.market.item_listed`)
|
||
and three in the decision block below.
|
||
|
||
**What that is worth, and it is not "more mail".** One trigger proves that a module can register a
|
||
trigger. Twenty-odd triggers spanning five audience kinds are the first real test of the things Parts
|
||
3–5 asserted and nothing has yet stressed: that the ceiling lattice holds when most triggers are *not*
|
||
public, that `check:modules` still finds no UO identifier in core when the module is this large, and
|
||
that a player's preferences screen stays legible when the catalogue is real rather than a demo. Every
|
||
rule ships `enabled = 0` (Q3), so this is a catalogue an operator turns on, not a switch that floods
|
||
anyone.
|
||
|
||
**The carve-out: `uo.market.item_listed` does not ship here**, for the reason §8.6 already gives — it
|
||
is a *saved search*, not a trigger. Its audience is "users whose stored query matches this listing",
|
||
and no per-user query store exists anywhere in the tree. It is its own workstream on top of this one.
|
||
`vendor.listing` remains ingested; nothing is lost by waiting.
|
||
|
||
#### Six decisions settled by the org lead before any code (2026-08-31)
|
||
|
||
Phase 2's precedent, and for the same reason: each of these is cheap now and expensive to retrofit,
|
||
and two of them are security boundaries. Three came out of checking §8.6's rows against what the
|
||
*website* can address rather than against what the wire carries — a different question, and the one
|
||
nobody had asked.
|
||
|
||
| | Question | Decision |
|
||
|---|---|---|
|
||
| 1 | the operator-facing family ceilings at `admin`, and there is no `admin` in the lattice | **add it**, as a child of `staff`; `MODULE_API_VERSION` → **1.8.0** (§5.1a) |
|
||
| 2 | `uo.guild.joined`/`left` overlap core's `team.member.joined` | **ship `left` + `disbanded` only**; joins stay core's (§8.6) |
|
||
| 3 | `uo.points.rank_changed` cannot address an owner | **`subscribers` only**; the personal half carved out (§8.6) |
|
||
| 4 | `uo.link.requested` has no recipient and a 5-minute TTL | **carved out** (§8.6) |
|
||
| 5 | §7.1 Q9 — `news.post` continuity | **replace the raw tickle, seed the rule DISABLED**; news push stops on upgrade, deliberately (§7.1 Q9) |
|
||
| 6 | the 11a / 11b split this phase's last paragraph asks to confirm | **confirmed**, and split as below |
|
||
|
||
**Decision 1 is the one with a version number attached, so state what it costs.** `ceilings.js` gains
|
||
one `PARENT` entry, one label, and a recipients query; `permits`, `meet` and `meetAll` are untouched,
|
||
because `admin` is the only pair in the whole tree that is a genuine refinement — every admin is
|
||
staff. The bump to 1.8.0 is real and both halves take it. **Decision 5 is the one with a migration
|
||
note attached**: `EngagementRules.jsx`'s existing "Team email resumes when you switch a rule on"
|
||
banner must name news too, and Phase 13's release note must carry it as an upgrade step, or a
|
||
deployment loses news push silently.
|
||
|
||
**Two mechanisms this phase needs that no earlier phase built, neither of which is a decision so much
|
||
as a consequence.** `uo.vendor.expiring` and `uo.economy.milestone` are *threshold* events, and rule
|
||
conditions compare a declared variable against a **literal** only (`conditions.js` — no arithmetic, no
|
||
relative time), so "within 24 hours of dismissal" is not expressible as a condition. Nor can the
|
||
trigger simply fire per frame: `vendor.listing` is a sweep frame re-emitted on any price change, and
|
||
`economy.supply` arrives every five minutes. So **the threshold crossing is tracked module-side**, in
|
||
the in-process tracker `shardStreams.js` already uses for `champActive` and `cityGovernor`, and the
|
||
trigger declares `periodsRemaining` / `hoursRemaining` as ints so an operator can still narrow with
|
||
`lte`. The seeded rule carries a `subjectKey` cooldown keyed per vendor. Same shape for the economy
|
||
milestone, whose thresholds are the module's.
|
||
|
||
**Two rows ship only because Phase 10 does.** `uo.house.idoc_warning`'s enrichment and
|
||
`uo.vendor.expiring` both need v5 fields; `uo.account.login_attempt` needs v5's new
|
||
`account.login.result` emitter and is renamed **`uo.account.login_failed`** to say what it actually
|
||
is. Everything else is mapping-only and can ship whether or not the bump has landed — the flagship's
|
||
`Greatly` mapping included, which **needs no protocol change** and can ship with Phase 6 if Phase 10
|
||
is still in flight, the trigger simply omitting `nextStage`/`estimatedCollapse` until the v5 overlay is
|
||
deployed, exactly as the `required: false` declaration already permits.
|
||
|
||
**The set, grouped by the audience kind each family exercises** — which is the point of grouping them
|
||
this way, since the audience kind is the thing being tested:
|
||
|
||
| Family | Triggers | Audience kind | Notes |
|
||
| --- | --- | --- | --- |
|
||
| **Owned asset at risk** | `uo.house.idoc_warning`, `uo.house.refreshed`, `uo.house.collapsed`, `uo.vendor.expiring` | `owner` (linked account) | The flagship family. All four resolve through `ownerAcct` → `shard_links`. `uo.house.refreshed` was added by decision 11 during the live walk: it is what CANCELS the warning's delayed row, and without it a fifteen-minute delay was just a late mail |
|
||
| **Passive income** | `uo.vendor.sale` | `owner` | **Patch-tier only** — see §8.6's corrected row; dormant on a shard that declined the patches, and the seeded rule's description must say so |
|
||
| **Personal security** | `uo.account.login_failed`, `uo.account.unlinked` | `owner`, ceiling `owner` | `uo.link.requested` was a third here and is carved out (decision 4) |
|
||
| **Personal milestone** | `uo.skill.capped`, `uo.quest.complete`, `uo.character.death`, `uo.character.murdered` | `owner`, opt-in | The two death triggers are a killfeed some players want and most do not — both ship `enabled = 0` and default `off` per channel |
|
||
| **Social / civic** | `uo.guild.left`, `uo.guild.disbanded`, `uo.governor.elected`, `uo.election.opened` | `members`, `subscribers` | `uo.guild.joined` is core's `team.member.joined` (decision 2). `uo.election.opened` carries `autoPickAt` — a real deadline, so it is the first trigger whose template has a genuine call to action with an expiry |
|
||
| **Come online now** | `uo.champ.started`, `uo.champ.boss_up`, `uo.server.up`, `uo.server.down` | `subscribers` | **`uo.server.up`/`down` is the cooldown table's stress test** — a flapping shard emits both repeatedly. Hard per-rule cooldown, not a per-send one |
|
||
| **Leaderboard** | `uo.points.rank_changed` | `subscribers` | The board's top N changed. The `owner` half is carved out — `top[]` names a serial, and links are keyed by account (decision 3) |
|
||
| **Staff-facing** | `uo.page.new`, `uo.cheat.detected` | ceiling `staff` | These are why the ceiling exists. Phase 3 already filters a `staff`-ceiling trigger out of a player's catalogue *and* gates it on write, so this family is the production proof of that work rather than new mechanism |
|
||
| **Operator-facing** | `uo.audit.staff_action`, `uo.economy.milestone`, `uo.world.saved` | ceiling `admin` (new — decision 1) | Digest-shaped by nature; neither of the last two should ever be instant. `uo.audit.staff_action` moved down from `staff`: a digest of what moderators did is not for moderators |
|
||
|
||
**Also core's own `news.post` emitter, which this phase's title has always understated** (§7.1 Q9).
|
||
`news.post` is declared with no caller, so a rule naming it can never fire; wiring it is one
|
||
`ctx.events.emit`-shaped call in `announceIfNewlyPublished`, gated on the same job-id transition signal
|
||
the push already uses. **The announce legs and the post hooks are untouched** — a module's town-crier
|
||
leg is a delivery to a channel of the deployment and its news-gump hook is state mirroring, neither of
|
||
which is a per-person notification. What the emit replaces is the raw `pushDispatch.publish` beside
|
||
them — **and it does replace it** (decision 5), with core seeding a `news.post` rule `enabled = 0`
|
||
beside the four Team ones. News push therefore stops on upgrade until an operator switches that rule
|
||
on. That is the deliberate answer rather than the comfortable one; §7.1 Q9 records why, and the two
|
||
obligations it creates — the admin banner and the release note — are 11a's and 11b's respectively.
|
||
|
||
**Also `module-uo`'s first registered audiences** (§5.1a: "`module-uo`'s first real audiences come in
|
||
Phase 11"). These are the *operator-composable* lists — named sets with constant parameters that an
|
||
admin points a rule or a segment at — and they are a different mechanism from the `members` audience
|
||
the guild triggers use. A guild event resolves to "the members of *this* guild", which is different
|
||
for every firing and therefore travels on the envelope as `recipientUserIds`, exactly as the Team
|
||
fan-out does (Phase 6, decision 2); a registered audience answers a question with the same answer
|
||
every time it is asked. Three ship: **`uo.guild.members`** (param `guildId`, ceiling `members`),
|
||
**`uo.governors`** (no params, ceiling `members`) and **`uo.linked.accounts`** (no params, ceiling
|
||
`members`) — every website user with a linked game account, which is the set an operator reaches for
|
||
first and the one a `not` composes against.
|
||
|
||
**On the size of this phase, and the split — confirmed by the org lead (decision 6).** It is the
|
||
largest in the workstream: 24 trigger declarations after the three carve-outs, their audiences and
|
||
ceilings, their `shardIngest` mappings, three registered audiences, and a seeded rule and template
|
||
each — against Phase 5a's nine seeded bodies in total. It lands as two PRs per repo, on the 4a/4b
|
||
and 5a/5b precedent:
|
||
|
||
- **11a — declarations, mappings, ceilings. Server only, and no template or rule is seeded.**
|
||
`module-uo`: `server/config/shardTriggers.js` (the 24 declarations, every variable carrying its
|
||
required `example`), `server/utils/shardEngagement.js` (the wire-kind → `ctx.events.emit` mapper and
|
||
its transition/threshold trackers), hung off `shardIngest.ingest` beside the existing broadcast and
|
||
push fan-outs and fire-and-forget like both; owner resolution through
|
||
`shardLinks.getByAccount` → `ownerUserId` on the envelope, the `shardPush.js` precedent unchanged;
|
||
`server/core.js` gains `events`; `registerEventTriggers` + `registerAudiences` in `server/index.js`.
|
||
`website`: the `admin` ceiling (decision 1), the `news.post` emitter and its seeded-disabled rule
|
||
(decision 5), the banner text, `MODULE_API_VERSION` → 1.8.0, and `engagement-triggers.json`
|
||
regenerated. The acceptance that belongs here is the catalogue-shaped half: **no staff- or
|
||
admin-ceiling trigger appears by name in a player's preferences catalogue**, and `check:modules`
|
||
proves core gained no UO identifier.
|
||
- **11b — the seeded templates, the seeded rules, and the live walk.** **26** rules, all
|
||
`enabled = 0` — the twenty-fifth is `uo.governor.appointed` (decision 10 below) and the
|
||
twenty-sixth is `uo.house.refreshed`, which the walk itself added (decision 11). Seventeen
|
||
in-universe bodies on two channels each (decisions 8 and 9), and core's `notify.event` +
|
||
`inapp.event` + the structural projection for the nine that stay plain — §4.6.1 property 1 is
|
||
precisely the claim that a trigger needs no authoring, and the nine are the test of it that
|
||
means something. All of it reaches the database through **`registerEngagementSeeds`**, which
|
||
decision 7 adds because no mechanism for a module-supplied template or rule existed. Everything
|
||
in the acceptance list below that needs a running shard is 11b's.
|
||
|
||
#### Four more decisions, settled by the org lead before any 11b code (2026-09-01)
|
||
|
||
Phase 11a shipped the declarations and stopped exactly where the plan said it would. Standing it up
|
||
raised four questions the plan had not asked, two of which are mechanism and two of which are voice.
|
||
All four were settled the same day, before any code, and three of them changed what 11b builds.
|
||
|
||
| | Question | Decision |
|
||
|---|---|---|
|
||
| 7 | a module cannot ship a template or a rule — core's seeders are core's | **add `api.registerEngagementSeeds({ templates, ruleGroups })`**; `MODULE_API_VERSION` → **1.9.0** |
|
||
| 8 | the register the game-powered bodies are written in | **in-universe, with a per-family in-fiction sender** — the crown writes about the crown's business, and nothing else |
|
||
| 9 | which bodies stay plain | **the nine security and operational ones**; the other sixteen go in-universe |
|
||
| 10 | the newly-elected governor is addressable and receives no letter | **add `uo.governor.appointed`** — a 25th trigger, `owner`-audienced, beside the existing broadcast |
|
||
|
||
**Decision 7 is the one 11a could not have discovered, because 11a seeded nothing.** `templateSeeds.js`
|
||
and `coreRules.js` are core files with core arrays in them, and there is no `registerTemplates` or
|
||
`registerRules` anywhere in `registries.js`. So the sentence above — "24 rules, all `enabled = 0`;
|
||
bespoke template bodies" — described work with **no mechanism to land in**. A module could have
|
||
reached `engagement_templates` through the `ctx.query` it already holds, and that was rejected for the
|
||
reason every raw-SQL-into-core-tables shortcut is rejected: it would re-implement `seedOne`'s
|
||
`customized` skip, its `seed_version` comparison and its block validation outside the file that owns
|
||
them, and the copy would drift the first time core improved the original. The registry keeps every one
|
||
of those in core and lets a module supply data — which is the same split `registerEventTriggers`
|
||
already makes, and the same reason it makes it.
|
||
|
||
> **Templates are re-ensured every boot; rule groups are one-shot.** That asymmetry is not an
|
||
> oversight, it is the two mechanisms this system already has, applied unchanged. A template row
|
||
> carries `seed_key` + `seed_version` + `customized`, so re-ensuring is how a better default reaches a
|
||
> deployment without stealing an operator's edit (§4.6.1 property 3) — and a template added in a later
|
||
> module version reaches every deployment, not only fresh ones. A rule is the opposite: re-ensuring one
|
||
> would resurrect a rule an operator deleted and reset one they enabled. So rules arrive in **named
|
||
> groups**, each with its own one-shot settings guard, which is 11a's seed-key finding stated as an API
|
||
> rather than as a warning. A rule added to an existing group reaches fresh installs only; a rule that
|
||
> must reach existing deployments takes a new group. The module names its groups, so the module owns
|
||
> that choice, and gets a docs paragraph telling it which one it is making.
|
||
|
||
**Where it runs is load-bearing, and it is not `seedDefaults()`.** `server.js` calls `seedDefaults()`
|
||
**before** it requires `app.js`, and requiring `app.js` is what scans the volume and runs the loader —
|
||
so at the moment core seeds its own templates, no module has registered anything. Module seeds
|
||
therefore run in `modules/lifecycle.js` `boot()`, after the `installed_modules` reconcile and **before**
|
||
the `onBoot` dispatch: a module whose `onBoot` warms a cache may assume its rules exist, and a module
|
||
the operator has disabled or that failed to load is skipped rather than seeded. Failure is `safe()`'d
|
||
like every other step there — a template that would not seed costs the shipped default, never the boot.
|
||
|
||
**Decision 8 is why the bodies are the module's and could not have been core's even with a mechanism.**
|
||
"A letter from Lord Blackthorn" is Britannian canon; §5.2's rule is that core stays ignorant of every
|
||
word of a module's domain. `check:modules` would not have caught it — it reads identifiers, imports and
|
||
route literals, never prose (`checkModuleIdentifiers.js` says so in its own header, and says why) — and
|
||
that is precisely the argument for taking the boundary seriously here rather than leaning on the
|
||
checker. A `templateSeeds.js` in core containing `uo.house.idoc_warning` and the word *Britannia* would
|
||
defeat the extraction while passing every check in the repo.
|
||
|
||
**The sender is per family, not one voice across all sixteen.** The crown was the tempting answer and
|
||
it is the wrong one: a shard where Blackthorn writes to you personally about a champion spawn is a
|
||
shard where the letter about your governorship means nothing. So the court writes about the crown's
|
||
business — the governorship, the elections — and everything else has the sender its own subject
|
||
implies:
|
||
|
||
| Family | In-fiction sender |
|
||
| --- | --- |
|
||
| `uo.governor.appointed`, `uo.governor.elected`, `uo.election.opened` | **Lord Blackthorn's court** — the seat, the ballot, the realm's offices |
|
||
| `uo.house.idoc_warning`, `uo.house.collapsed` | the **Office of Deeds** — a clerk with a ledger and a duty to warn |
|
||
| `uo.vendor.expiring`, `uo.vendor.sale` | the **Merchants' Guild** — a factor rendering accounts |
|
||
| `uo.guild.left`, `uo.guild.disbanded` | a **guild herald** |
|
||
| `uo.champ.started`, `uo.champ.boss_up` | the **town crier** — which is also the leg a module already owns |
|
||
| `uo.skill.capped`, `uo.quest.complete` | the **guildmaster of that craft** |
|
||
| `uo.character.death`, `uo.character.murdered` | the **Chronicler of the Dead** |
|
||
| `uo.points.rank_changed` | the **keeper of the rolls** |
|
||
|
||
**Decision 9 draws the line at the two places fiction costs something real.** A security notice must be
|
||
distinguishable from flavour — a failed-login mail written as *"a stranger sought entry to thy
|
||
account"* is indistinguishable in register from the phishing mail it is warning about, and that is not
|
||
a trade worth making for atmosphere. And an operator reading `uo.cheat.detected` at two in the morning
|
||
wants a name, a rule, a location and a timestamp, not a scroll. So nine stay plain and use core's
|
||
generic bodies:
|
||
|
||
```
|
||
PLAIN (9, core's notify.event / inapp.event)
|
||
uo.account.login_failed uo.account.unlinked ← security
|
||
uo.server.up uo.server.down ← infrastructure
|
||
uo.page.new uo.cheat.detected ← staff ceiling
|
||
uo.audit.staff_action uo.economy.milestone
|
||
uo.world.saved ← admin ceiling
|
||
|
||
IN-UNIVERSE (16, module-uo's own bodies)
|
||
house ×2 vendor ×2 skill/quest/death ×4
|
||
guild ×2 civic ×3 champ ×2 points ×1
|
||
```
|
||
|
||
**Both channels, and the digest is deliberately not one of them.** Each in-universe trigger ships an
|
||
`email` body — the letter — and an `inapp` body in the same voice, because a rule fires on both
|
||
channels at once and a player who reads the inbox item and then the mail must not find two different
|
||
narrators. Thirty-two rows, which is data rather than design. The **digest** stays core's generic
|
||
`notify.digest`: a day of events rolled into one list is not a letter from anyone, and dressing a
|
||
bulleted summary as correspondence is where this device stops being charming.
|
||
|
||
**Decision 10 exists because a fact in the plan was wrong by omission.** §8.6 records that
|
||
`uo.points.rank_changed` cannot address a person, because `points.board.top[]` names a mobile serial
|
||
and links are keyed by account — and the same reasoning was silently assumed to cover the governor.
|
||
It does not. `city.update`'s `governor` field is written by `BridgeJson.Actor()`, which emits
|
||
`serial`, `name`, **`acct`** and even `webId` when the account is linked. The new governor is
|
||
addressable **today**, with no protocol change and no new wire field. So the letter the whole voice
|
||
decision was chosen to make possible has a recipient, and 11b ships it:
|
||
|
||
- **`uo.governor.elected`** stays exactly as 11a declared it — `subscribers`, ceiling
|
||
`authenticated`, the town-square announcement that a city has a new governor.
|
||
- **`uo.governor.appointed`** is new — `audience: 'owner'`, `ceiling: 'owner'`, `subjectKey: 'city'`,
|
||
resolved through `ownerAccount: ev.governor.acct` on the same `city.update` frame, in the same
|
||
transition branch, guarded by the same never-on-first-sight rule. It is the letter.
|
||
|
||
Widening the existing trigger to two audiences was considered and refused for a reason worth keeping:
|
||
one trigger means one rule means **one template**, and the town's bulletin and the governor's letter
|
||
are not the same text. Two triggers is also what lets an operator run the announcement and leave the
|
||
letter off, or the reverse.
|
||
|
||
**This takes the phase to 25 triggers**, and the count in §8.6 and in 11a's summary should be read
|
||
that way from here — **26 after decision 11 below added `uo.house.refreshed`**. It does not take another `MODULE_API` bump: a trigger declaration is module data,
|
||
and 1.9.0 is decision 7's.
|
||
|
||
#### What building it found
|
||
|
||
Four things, and the first two are the ones a reader of this section should not have to re-derive.
|
||
|
||
**A template has no conditionals, and an in-universe body is a sentence.** That combination is the
|
||
whole engineering content of decision 8, and it is not obvious until a body is written. An unset
|
||
optional interpolates to the **empty string** (`interpolate.js` — no filters, no ternaries, by
|
||
design), which is exactly right for `notify.event`, whose variables are structural and each sit on
|
||
their own line. It is wrong for a letter: *"Be it known that , recorded to thy name, is this day
|
||
found ."* So the ternary stays at the call site and its result arrives as a declared optional, which
|
||
is Phase 5a's `forWhom` precedent unchanged — and the module now declares two shapes of them:
|
||
|
||
- a **LABEL** always has a value and can carry a sentence's spine. `houseLabel` is the house's name
|
||
and region, falling back to its seal number, because a warning has to name *something* the owner
|
||
can act on and a seal number is worse prose and better than a blank.
|
||
- a **TRAILING FRAGMENT** may be empty and leads with **its own space**, so the sentence closes
|
||
cleanly without it: `{{slainBy}}.` renders "has fallen." or "has fallen at the hands of a lich
|
||
lord."
|
||
|
||
The labels are declared `required: false` deliberately, and that is a real trade rather than an
|
||
oversight: a required variable missing **refuses the emit**, and a dropped notification is worse than
|
||
a cosmetic hole. Which leaves nothing at runtime to notice a mapper that forgot one — so
|
||
`engagementSeeds.test.js` asserts every label is supplied on every path that emits its trigger. That
|
||
test is the guarantee the declaration deliberately does not make.
|
||
|
||
**A render sweep over all 32 bodies is what found the defects, and none of them were visible in the
|
||
source.** Every body was rendered twice — once with every declared `example`, once with **required
|
||
variables only** — and scanned for the punctuation a vanished token leaves behind. Three real
|
||
defects, none of which any existing check could see:
|
||
|
||
1. **An optional variable in a SUBJECT LINE.** `A notice concerning thy house at {{region}}` renders
|
||
*"…thy house at "* for a house outside a named region. A subject is the one place a hole is
|
||
unmissable, and it is also the one place no block validator looks.
|
||
2. **Multi-optional detail lines.** *"On hand: gold. Charged each period: gold. Periods remaining:
|
||
."* on a pre-v5 vendor frame, where the whole fee block is absent. Fixed the way `place()` already
|
||
argues for coordinates: the line is **assembled in the mapper** from the parts actually present,
|
||
and a frame with nothing to report produces no line rather than a punctuated empty one.
|
||
3. **A trailing fragment used as a leading one**, opening an in-app body with a stray space.
|
||
|
||
**A trigger-bound template could not reference its own unsubscribe link.** `emailChannel.deliver`
|
||
computes an unsubscribe token per recipient and merges it last over the projection, so
|
||
`{{unsubscribeUrl}}` always *rendered* — but `variablesFor` takes a trigger-bound template's variable
|
||
list from the **trigger's** declaration, and a trigger has no business declaring a fact about how the
|
||
mail was delivered. So the token was undeclared, and the save-time undeclared-variable check would
|
||
have refused the first operator who tried to **edit** one of the sixteen bodies. Rendering right and
|
||
then refusing the edit is the worst of both. Core gains `DELIVERY_VARIABLES` beside
|
||
`AMBIENT_VARIABLES` — declared separately because they apply to a different set: ambient facts are
|
||
about the deployment and reach every body, delivery facts are about the send and reach the
|
||
trigger-bound ones, which is precisely the set that is engagement mail. **Nothing had ever taken this
|
||
path**, because core's `notify.event` declares the variable in its own seed and is bound to no
|
||
trigger.
|
||
|
||
**A trigger id and a template key have different grammars, and one of these ids uses both
|
||
characters.** `uo.champ.boss_up` is a legal trigger id (§7.2 relaxed the id grammar to admit `_`
|
||
inside a segment, precisely so `uo.house.idoc_warning` could exist) and an **illegal** template key —
|
||
`engagement_templates.key` is matched by `templates.KEY_RE`, which admits `.` and `-` and not `_`. So
|
||
the body for that trigger is keyed `uo.champ.boss-up`. Caught at registration as a boot failure,
|
||
which is the right place, and asserted in the module's tests so the next person meets it as a test
|
||
name rather than as a puzzle.
|
||
|
||
**Acceptance:** the five-rung shard visibility walk still shows no leak, and **no staff- or
|
||
admin-ceiling trigger appears by name in a player's preferences catalogue**; a house transitioning to
|
||
`Greatly` on the live rig produces one email to the linked owner and nothing to anyone else; a second
|
||
transition inside the cooldown produces nothing; a refresh back to `LikeNew` inside the delay window
|
||
cancels the pending mail; a player vendor drawn down to its last day produces one `uo.vendor.expiring`
|
||
to its owner; a failed game login produces one `uo.account.login_failed` and a **successful** one
|
||
produces none; a shard restarted three times in a minute produces **one** downtime mail, not six; and
|
||
**a news post published on the rig reaches a rule, with the town-crier leg and every registered post
|
||
hook still firing exactly as they did** — and, decision 5's other half, **with the seeded `news.post`
|
||
rule switched on, a publish tickles the same subscribers it tickled before this phase; with it off,
|
||
it tickles nobody and the admin screen says why.** Two more the new ceiling owes: **a rule cannot be
|
||
saved giving an `admin`-ceiling trigger a `staff` audience**, and **an editor is not a recipient of
|
||
`uo.audit.staff_action`** while an admin is.
|
||
Four more the 2026-09-01 decisions owe: **a module's seeded rule group is seeded once and only
|
||
once** — an operator who deletes one does not find it back after a restart, and one they enabled
|
||
is still enabled — while **a module template whose `seedVersion` moved is re-ensured unless the
|
||
operator edited it**; **an election on the live rig produces a letter to the new governor AND the
|
||
announcement to subscribers, from two rules either of which can be switched off alone**; and
|
||
**`check:modules` is still clean with sixteen Britannian bodies in the tree**, which is the
|
||
decision-8 boundary stated as a check rather than as a promise.
|
||
**Guardrails:** `check:modules` proves core gained no UO identifier across every phase to this point —
|
||
which is a materially stronger claim now that the module registers two dozen UO-named triggers.
|
||
|
||
#### Three decisions taken during the live walk (2026-09-01), and what it found
|
||
|
||
The walk is the last piece of 11b and it was not a formality: it found **six defects, four of them in
|
||
code shipped by earlier phases**, and three of the fixes were decisions rather than repairs. None of
|
||
the six was visible in a unit test, and the reason is the same every time — each is a disagreement
|
||
between two things that agree with each other in a fixture.
|
||
|
||
| | Question the walk raised | Decision |
|
||
|---|---|---|
|
||
| 11 | `uo.house.idoc_warning` ships `delay_seconds: 900` and nothing could cancel it | **add `uo.house.refreshed`** — a 26th trigger, with a body and a seeded rule of its own |
|
||
| 12 | a rule with a cooldown delivered on exactly ONE of its channels | **`channel` joins the cooldown key** — a cooldown is per delivery, not per occasion |
|
||
| 13 | `uo.vendor.expiring` could not fire, because the market sweep does not diff fees | **widen `BridgeMarket.Signature()`** with `exempt` + `periodsRemaining` |
|
||
|
||
**Decision 11 — the delay had no cancel path for the GOOD outcome.** §4.2a is explicit that a delay
|
||
without a cancellation is just a late mail, and the fifteen minutes on the warning exist so a player
|
||
who repairs the house is never told it is in peril. The rule's `cancel_on` named only
|
||
`uo.house.collapsed` — the outcome where the warning is pointless — and nothing named the outcome
|
||
where it is *wrong*, because the mapper deliberately returned early on every transition that was not
|
||
a late stage, so a refresh reached the engine as silence. The wire had carried it all along. It ships
|
||
with a body as well as the cancellation (the org lead's call): an office that only ever writes when
|
||
something is wrong teaches its reader to dread the seal, and a notice that the ledger is set right is
|
||
the cheapest possible way not to. The cancellation is read off the WARNING's rule, so an operator who
|
||
wants it without the reassurance simply leaves the new rule off — which, every seeded rule shipping
|
||
disabled, is the default.
|
||
|
||
> **`Ageless`, not only `LikeNew`, and this is a ServUO fact worth not re-deriving.** A condemned
|
||
> house cannot be refreshed at all — `BaseHouse.RefreshDecay()` returns false for
|
||
> `DecayType.Condemned` — so the rescue is the owner logging back in. Their NEWEST house then becomes
|
||
> `AutoRefresh` and reads **`Ageless`**; an older one becomes `ManualRefresh` and reads **`LikeNew`**.
|
||
> Both mean "out of danger", and reading only the second misses the common case. The live rig produced
|
||
> `Ageless`.
|
||
|
||
**Decision 12 is the one with the widest blast radius, and it had been latent since Phase 4a.**
|
||
`cooldownsDb.claim` is called INSIDE the engine's per-channel loop and its key was
|
||
`(rule, user, subject)`, so the first channel of a rule claimed the cooldown and every later one was
|
||
reported as `cooled`. `inapp` is ranked first on purpose (so `push` can reference the inbox row it
|
||
writes), which means a rule naming email + in-app delivered the inbox item and **silently never the
|
||
mail** — including core's own `news.post` rule, which names three channels and carries an hour's
|
||
cooldown. Decision 8 requires the letter and the inbox item to fire together, so every in-universe
|
||
body this phase wrote was unreachable behind its own in-app twin.
|
||
|
||
Claiming once per user *before* the channel loop was the alternative and was refused: it makes the
|
||
cooldown a per-occasion limit, and an operator who sets "one a day about this house" means one mail
|
||
and one inbox item, not one of the two. So `channel` joins the PRIMARY KEY, migrated in place behind
|
||
a guarded `DROP PRIMARY KEY` — MariaDB has no conditional form of a key change, so replaying the
|
||
schema file would fail on every boot after the first without the `information_schema` guard. Existing
|
||
rows keep `channel = ''` and expire on their own interval: one stale cooldown per (rule, user,
|
||
subject) is a better trade than dropping the table, which would let a storm through.
|
||
|
||
**Decision 13 is a defect in the shard plugin, and Phase 10 is where it was introduced by omission.**
|
||
`BridgeMarket.Signature()` diffs shop name, owner, map, coordinates and the item/price list — and v5
|
||
added a `fees` block that it never learned about. So a vendor quietly running out of gold changes
|
||
nothing the sweep compares, emits no frame, and the trigger whose entire subject is a vendor running
|
||
out of gold can fire only by coincidence: when somebody happens to reprice an item on a shop that is
|
||
already broke. Proved on the rig by setting a vendor's held gold to zero and watching no frame
|
||
follow. The fix carries the DERIVED values (`exempt`, `periodsRemaining`) rather than the raw ones —
|
||
an integer division moves only when the shard's own answer to "is this vendor in danger" moves, while
|
||
`holdGold` changes on every sale and `nextPayAt` on every tick, either of which would re-emit a fat
|
||
listing frame for a shop whose listings did not change. No new wire field and **no `PROTOCOL_VERSION`
|
||
bump**: it changes when an existing frame is sent, not its shape.
|
||
|
||
**Four more defects, each fixed without a decision because the right answer was a fact rather than a
|
||
choice:**
|
||
|
||
- **Every module-seeded rule failed to insert.** `checkSeedRule` validated `max_sends_per_hour` and
|
||
then dropped it from the normalized rule; the column is NOT NULL, so all 25 rules failed at boot —
|
||
loudly, on a real database, and on no fixture. The registry test asserted the REJECTION of a bad
|
||
ceiling and never that a good one survives. It now asserts the normalized rule against the insert's
|
||
own column list, so the next field added is covered the day it is added.
|
||
- **Every `owner`-audienced trigger resolved to nobody.** `resolveTarget` read `link.user_id` where
|
||
the model's `toSafe` returns `userId`, so the flagship family — houses, vendors, logins, unlinks,
|
||
deaths, the governor's letter — reached no one, and looked exactly like the ordinary
|
||
unlinked-account case, which the code treats as normal and deliberately does not log. **The test
|
||
fake returned `user_id` and therefore agreed with the bug**, while `shardStreams.test.js`'s fake
|
||
next door — same model, and the path `shardEngagement.js` says it copies — returned `userId`. The
|
||
fake is now built by running the real `toSafe` over a stubbed row.
|
||
- **Every call-to-action in every in-universe body was a dead link**, from two independent mistakes
|
||
with one symptom. The client router prefixes a module's routes with its **ID** (`/uo/houses`), not
|
||
with `module.json`'s `mounts` (`/shard/…`), so every declared `example` and every hard-coded href
|
||
was a 404 — and not one of the eight `url` variables was ever populated by the mapper, so the
|
||
buttons rendered with an empty href and vanished from the text part altogether. Both now read
|
||
`config/clientPaths.js`, and two tests close it: every `url` example must name a route the module
|
||
mounts, and every declared `url` variable must actually be supplied on the path that emits its
|
||
trigger.
|
||
- **A raw wire timestamp was signing off the Merchants' Guild's letter** —
|
||
`2026-09-02T04:06:43.8397548Z`, mid-sentence. Core has no interpolation filters by design, so the
|
||
readable form is assembled in the mapper and arrives as its own variable; the machine value stays
|
||
exactly as it was, because an operator writes `is at most` conditions against it.
|
||
|
||
**What the walk proved, rung by rung, against real ServUO + the release sidecar + a live SMTP
|
||
catcher.** A `Greatly` transition produced **one** notification to the linked owner on both channels
|
||
and nothing to a second linked player who was subscribed; the next transition inside the day was
|
||
cooled; the owner logging back in **cancelled both pending rows** and sent the reassurance letter; a
|
||
vendor drawn to its last period warned its owner once and was cooled on the second crossing; a failed
|
||
game login over a real socket produced one notice and a **successful** one produced none; three
|
||
genuine shutdown/boot cycles produced **one** downtime fan-out and two fully-cooled ones; a governor
|
||
seated on a live `city.update` produced **both** the town's bulletin (six subscribers) and the crown's
|
||
letter (the winner alone); a real staff action driven through the sidecar reached the admin and
|
||
**neither the editor nor the moderator**, both of whom were subscribed; and the rule editor refused
|
||
`staff`, `authenticated` and `everyone` on an `admin`-ceiling trigger while accepting `admin`.
|
||
|
||
> The three cycles took about three minutes rather than the acceptance's "one minute", because a real
|
||
> ServUO boot is roughly fifty seconds. The property is unchanged: the rule's cooldown is an hour.
|
||
|
||
The walk is repeatable. `servuo-plugins/tools/scaffolding/BridgeRigDriver.cs` drives the shard from
|
||
outside the game over a polled command file, which is what a walk needs and what no existing probe
|
||
gave — every other one runs a fixed script at boot or from an in-game client, and ServUO's console
|
||
takes a fixed verb set, so `[p5probe` cannot be typed at a headless shard at all.
|
||
|
||
---
|
||
|
||
### Phase 12 — `runicgateway.com`: the public site and the docs journey ✅
|
||
|
||
**This phase is not optional polish, and it is not "update the marketing copy". Two of its checks
|
||
will fail the build on their own**, and they read from `main`, which fixes exactly when this has to
|
||
land (see the timing note below).
|
||
|
||
**The mechanical half — the site's own checks go red at cutover.** `scripts/checkFacts.mjs` fetches
|
||
each value's authority from the **`main` branch of the source repo** and fails the build on any
|
||
disagreement (`link main:sidecar/src/main.rs`, `servuo-plugins main:overlay.toml`,
|
||
`website main:server/src/modules/version.js`, `Module-uo main:module.json`, and the `bundles` branch).
|
||
So `src/data/platform.json` needs:
|
||
|
||
- `protocol: 4` → **5** (Phase 10) — asserted three times over: the sidecar, the overlay, the bundle.
|
||
- `moduleApi: "1.6.0"` → **"1.7.0"** (Phase 2).
|
||
- `bundle.tag` / `bundle.sidecar` / `bundle.overlay` — whatever Phase 10's republished bundle carries.
|
||
- `verifiedOn` moved, and the page that quotes each value re-read rather than the JSON edited to make
|
||
the check pass — which is what the file's own header comment tells you in as many words.
|
||
|
||
**The content half — the site currently documents the opposite of what Phase 1 ships.**
|
||
`src/content/docs/docs/administration/notifications-and-email.mdx` carries an
|
||
`<Aside type="note" title="There is no SMTP option">` and the sentence *"Gmail over OAuth2 is the only
|
||
supported delivery path today."* Also `configuration.mdx:62` ("it is Gmail over OAuth2, it reuses the
|
||
Google authentication client"), `troubleshooting.mdx:101` ("Connect Gmail in **Settings → Email
|
||
delivery**"), and `system-architecture.mdx:117` ("the Gmail refresh token"). All four become false the
|
||
day Phase 1 merges to `main`.
|
||
|
||
**The capability claim to fix while here.** `src/data/capabilities.mjs`'s Notifications item already
|
||
says *"Web, push and email, chosen per stream by each person"* — **"Web" is not true today** (§0.1) and
|
||
becomes true at Phase 7. One list feeds `/`, `/features/` and `/modules/`, so this is one edit, and
|
||
`assertDetailCoverage()` will make sure the detail line comes with it.
|
||
|
||
**Legal and Play Data Safety.** `PLAY_DATA_SAFETY.md` and `/privacy` are generated from **one
|
||
inventory** (`scripts/playDataSafety.mjs`, `src/data/legal.mjs`), and an engagement mailer changes what
|
||
that inventory has to say: an email address is now used for **more than account function**, there is a
|
||
send log (`engagement_sends`, address hashes) and a suppression list. Play's Data Safety form
|
||
distinguishes "app functionality" from "communications/marketing", and getting that wrong is a store
|
||
review problem rather than a doc nit. Phase 9's decisions are the input, so this lands with or after
|
||
Phase 9 — **and it is the one part of Phase 12 with an external deadline attached to it.**
|
||
|
||
**New docs pages** for the admin docs journey: the engagement rules screen, the template editor, and
|
||
per-channel notification preferences. `scripts/checkSidebar.mjs` and `checkQuickstart.mjs` both hold
|
||
this to the actual site, and `checkQuickstart` is a **two-way** drift check against the website repo's
|
||
own setup — Phase 1 changes email setup, so expect it to have an opinion.
|
||
|
||
**Timing — this is the sequencing trap.** Because `checkFacts` reads `main`, the site stays **green for
|
||
the entire `edge` period** and goes red **the instant the cutover lands**. So Phase 12's work must be
|
||
**written and reviewed before the cutover and merged inside the same window** — not "after we ship".
|
||
Left until afterwards, the public site is broken and publishing false statements about the product at
|
||
exactly the moment anyone would look at it.
|
||
|
||
**Acceptance:** `node scripts/checkFacts.mjs`, `checkLinks`, `checkSidebar`, `checkQuickstart`,
|
||
`checkReference`, `checkA11y`, `checkCsp`, `checkBrand` and `npm test` all green **against the
|
||
post-cutover `main` of every source repo**; `grep -ri "gmail" src/` returns only historical/legal
|
||
references; `/privacy` and `PLAY_DATA_SAFETY.md` regenerate from the amended inventory with no manual
|
||
edit; the Notifications capability line is true of the shipped system.
|
||
|
||
#### As built (2026-09-01) — split 12a / 12b, because the facts cannot be green and correct at once
|
||
|
||
**The org lead's decision: split the phase.** The timing note above says Phase 12's work is "written
|
||
and reviewed before the cutover and merged inside the same window", and building it made the reason
|
||
sharper than the sentence. `checkFacts.mjs` fetches from `main`, so the moment `platform.json` says
|
||
`protocol: 5` the repo is red — and it stays red for the whole `edge` window. Worse, **three of its
|
||
values cannot be written at all today**: `bundle.tag` / `bundle.sidecar` / `bundle.overlay` name an
|
||
artefact that does not exist until Phase 13 has landed `link` + `servuo-plugins` on `main` and CI has
|
||
republished the bundle. So:
|
||
|
||
- **12a — `runicgateway.com#26`, mergeable on `edge` today.** The `Engagement rules` page, the
|
||
Troubleshooting symptom that points at it, the cross-links, and the `/privacy` + Play Data Safety
|
||
inventory. Every check green.
|
||
- **12b — `runicgateway.com#27`, a DRAFT, deliberately red.** `platform.json` `protocol` → 5 and
|
||
`moduleApi` → 1.9.0, plus the protocol page. Held for the Phase 13 window; the fill-in step is in
|
||
Phase 13's order below.
|
||
|
||
**Two of this section's own statements were already stale**, both because Phase 11 moved after it was
|
||
written: `moduleApi` is **1.9.0**, not 1.7.0 (1.8.0 for the `admin` ceiling, 1.9.0 for the module seed
|
||
API), and the `capabilities.mjs` "Web" claim was already corrected in Phase 7's rg.com#24. The Gmail
|
||
sweep is likewise already done — `grep -ri gmail src/` returns only the migration note Phase 1 wrote
|
||
and the historical reference in Troubleshooting, which is what the acceptance line asks for.
|
||
|
||
**The page the phase actually owed.** "New docs pages … the engagement rules screen, the template
|
||
editor, and per-channel notification preferences" — the second and third landed in Phases 5b and 7. The
|
||
first had not, and it was the largest hole on the site: there was a page for *where* a message goes and
|
||
a page for *what it says*, and nothing for **what makes one get sent**, which is the entire admin
|
||
surface this workstream built. One page — Rules, Audiences, the trigger catalog and the send log —
|
||
between the two it joins up. Suppressions stays in Troubleshooting (Phase 9) and Templates keeps its
|
||
own page (Phase 5b).
|
||
|
||
Two things it exists to state, because both are things a reader gets wrong by reasoning:
|
||
|
||
- **Every rule ships disabled, including the ones a module brings.** "Installed" is not "on".
|
||
- **The ceiling is a tree, not a ladder** (§7.2's lattice). The ladder reading says a staff-only event
|
||
could obviously also go to one person; "one person" for `uo.cheat.detected` is *the player it was
|
||
detected on*.
|
||
|
||
#### Three findings, none of them in the phase's own text
|
||
|
||
**1. `/privacy` had been publishing a false sentence since Phase 8.** The `app-content` row claimed
|
||
"Nothing is cached for offline use and nothing is duplicated anywhere else", and Phase 8 shipped a
|
||
DataStore snapshot of the inbox. That row is an *input to the generated Play Data Safety answers*, so
|
||
it is a store-review matter and not a doc nit. The snapshot now has its own row and its own Play
|
||
mapping (Messages / Other in-app messages; not collected by us, stored on the device), and the old
|
||
claim is narrowed rather than softened.
|
||
|
||
**2. Nothing in the engagement schema has a retention sweep.** `engagementCooldowns.db.prune()` exists
|
||
and **has no caller**; a finished `engagement_outbox` row is never deleted; the send log and the
|
||
suppression list grow forever. The `deploy-engagement` privacy row therefore says "kept until the
|
||
operator removes them; nothing here expires on its own", which is the true answer rather than the one
|
||
a reader assumes. **Recorded, not fixed** — a sweep is a `website` change and outside this phase.
|
||
|
||
**3. The one page whose whole subject is the protocol number was the one page that hardcoded it.**
|
||
`platform.json`'s header says no version number is ever hardcoded in prose; `protocol-versions.mdx`
|
||
wrote `4` three times. It now imports the JSON like every other page that quotes a fact. Note for
|
||
whoever does this again: **MDX does not evaluate an expression inside backticks**, so the interpolation
|
||
has to sit outside the code span or the braces reach the published page.
|
||
|
||
---
|
||
|
||
### Phase 13 — The cutover: `edge` → `main`, in order ✅
|
||
|
||
One PR per repo, all in one window. **The order is not cosmetic** — a `main` holding a v5 sidecar and a
|
||
v4 overlay is a shard that cannot pair, and the installer refuses it by design.
|
||
|
||
1. **`docs`** — the design of record and every doc the phases produced, so the reference exists before
|
||
the code that needs it.
|
||
2. **`servuo-plugins`** and **`link`** together — the emitter and `overlay.toml` and
|
||
`PROTOCOL_VERSION` are one protocol bump with three declaration sites (CLAUDE.md). Then CI
|
||
publishes the paired bundle.
|
||
3. **`website`** — core: the transport abstraction, the trigger registry, `MODULE_API_VERSION` **1.9.0**,
|
||
the engine, the templates, the in-app channel, the Teams migration.
|
||
4. **`module-uo`** — its `coreApi` range and its triggers, after the core it declares against.
|
||
5. **`Integration-kit`** — `ci/core-ref.json` to the new website `main` sha. Its equality check is red
|
||
until this lands, **on purpose**; moving the pin is the acknowledgement that someone re-read the
|
||
chapters.
|
||
6. **`android-app`** — the in-app inbox. First real CI run (§6.0a).
|
||
7. **`runicgateway.com`** — last, because every fact it fetches has to be true on `main` first.
|
||
Two merges, in this order: **12a (#26)** into `edge` at any time, then `edge` → `main`, then
|
||
**12b (#27)** with its three unknowable values filled in. Filling them in is mechanical, because
|
||
the check names them: run `GITEA_TOKEN=<token> node scripts/checkFacts.mjs` and copy what it prints
|
||
in the "says" column — `bundle.tag`, `bundle.sidecar`, `bundle.overlay` from
|
||
`installer bundles:current.json`, `releases.Module-uo` if the cutover cut a new module release, and
|
||
`verifiedOn` as the date it was run. Then re-read the two pages that quote the bundle
|
||
(`getting-started/connect-a-game-server`, `administration/the-shard-connection`) before merging —
|
||
§12's own rule is that a value is never edited to make the check pass.
|
||
8. **`.profile/README.md`** — only if this is a headline capability.
|
||
|
||
**Acceptance:** a clean install from `main` alone stands the whole stack up — installer pairs a v5
|
||
sidecar with a v5 overlay, the site boots, SMTP is configured, a `Greatly` house transition on the
|
||
local rig produces exactly one email and one in-app item to the linked owner and nothing to anyone
|
||
else, and `runicgateway.com` builds green with no fact disagreeing with its authority. Every `edge`
|
||
is then fast-forwarded to `main` again so the next workstream starts from a clean one — the step
|
||
§6.0a found had been skipped after every previous cutover.
|
||
|
||
#### As built (2026-09-01)
|
||
|
||
**The window ran in the order above and missed two steps.** Steps 1 (docs#200), 2a
|
||
(servuo-plugins#20), 2b (link#35), 3 (website#180), 4 (Module-uo#26) and 6 (Android-app#43) merged
|
||
together; **5 and 7 did not**, and were done afterwards as **Integration-kit#9** and
|
||
**runicgateway.com#28**, with **Module-uo#27** clearing a leftover from step 4. What the window cut:
|
||
`link` **v2.1.0**, the plugin overlay **v1.1.0**, `module-uo` **v1.1.0**, and the paired bundle
|
||
**2026.09.01** (protocol 5). `website` releases nothing by design.
|
||
|
||
**Every `edge` was deleted rather than fast-forwarded**, in seven repos. The acceptance line above
|
||
asks for the fast-forward so the next workstream starts from a clean branch; deleting reaches the
|
||
same place more bluntly, since §6.0a's Phase -1 cuts `edge` fresh anyway — and a branch that does not
|
||
exist cannot be the stale one somebody branches from, which was the actual failure. Only
|
||
`Integration-kit` ended up in the state §6.0a warns about, one commit behind `main` once its own step
|
||
landed late; fast-forwarded on the day. **The convention is now delete-on-merge**, and Phase -1 cuts
|
||
`edge` rather than trusting one it finds.
|
||
|
||
**1. Gitea's `raw` API route is CDN-cached for six hours, and a stale read fails BOTH ways.**
|
||
`checkFacts.mjs` failed `runicgateway.com` for a `moduleApi` that was correct: it had been served
|
||
`website`'s `version.js` from a fortnight earlier (`Cache-Control: public, max-age=21600`,
|
||
`cf-cache-status: HIT`, `Age: 15713`), and no edit in that repository could have made it pass. The
|
||
false red is the cheap half. **The same run reported the already-republished bundle triple as still
|
||
current** — a stale read is just as able to say "nothing has moved", and that repo's whole bargain is
|
||
that it goes red when the platform moves. All three cross-repository checks now read the `contents`
|
||
endpoint, which answers `private, must-revalidate` and is not cached; a request `Cache-Control:
|
||
no-cache` header does **not** bust the CDN, and a cache-busting query param was rejected as
|
||
papering over the mechanism rather than choosing the right one.
|
||
|
||
**2. The Integration-kit equality check never goes red on its own, and step 5 is the only thing that
|
||
makes anyone look.** It clones the ref **the kit itself pins**, so a core that moves past that pin
|
||
changes nothing there. The kit sat three minor versions behind the platform for the whole
|
||
workstream — teaching a 1.6.0 contract with no triggers, audiences or seeds in it — and was green
|
||
throughout. Between cutovers the kit is not wrong, it is **dated**, and `ci/core-ref.json` is where
|
||
the date is written down. Read step 5's obligation as *someone re-reads the chapters*, never as
|
||
*CI will tell us*.
|
||
|
||
**3. `registerEngagementSeeds` does not validate the body it seeds, and the kit's own example was
|
||
malformed.** The call checks that `blocks` is a non-empty array and stops; the body is validated by
|
||
the block registry, which runs in the template editor and in the renderer and nowhere else. The
|
||
draft template shipped a heading `level: 2` — the registry takes `'h1' | 'h2' | 'h3'` — and no block
|
||
`id`s at all, so it would have registered cleanly, seeded cleanly, and failed the first time an
|
||
operator opened it. It was found by running the template's `register()` through **core's real
|
||
`stage()` validators** at the pinned ref, which the kit's CI does not do and cannot: it runs the
|
||
template against `test/_fakes.js`, and a fake accepts what core refuses. Two smaller corrections came
|
||
out of the same run — core **does** validate `subjectKey` against the declared variables and refuses
|
||
the module, and `ctx.events.emit` **throws outside production** rather than only dropping and
|
||
logging. Both had been written into the chapter the other way round.
|
||
|
||
The general lesson is §5.3's, arriving from the other side: a registration surface is only as
|
||
teachable as the thing that checks it. Where the registry validates, the book can describe the
|
||
error; where it does not — and `blocks` is the one place it does not — the book has to say so, and
|
||
the template needs a test of its own. It has one now, verified by breaking it.
|
||
|
||
**4. `runicgateway.com`'s `pr-checks.yml` triggers on PRs into `main` only**, so not one of the seven
|
||
site PRs in this workstream was gated, and the cutover was their first CI run in eight phases. Same
|
||
shape as `android-app`'s trigger before Q8 fixed it (§6.0a), and left alone rather than fixed: that
|
||
repo's checks read the *source repos' `main`*, so running them on an `edge` PR would have been red
|
||
for the whole window by design. Phase 12's split exists for the same reason. What it costs is real
|
||
though — 12b's fill-in was verified locally and merged on that evidence alone.
|
||
|
||
#### The acceptance walk — run 2026-09-01, PASSED
|
||
|
||
Every clause of the acceptance line above, on a rig built the way an operator would build one rather
|
||
than the way a developer would.
|
||
|
||
**A clean install from `main` alone.** The **released** installer v0.1.1, downloaded and checked
|
||
against its `SHA256SUMS` — the trust anchor the docs name, since releases are unsigned — resolved
|
||
bundle **2026.09.01** and reported the pair it was about to deploy: overlay **v1.1.0** protocol 5,
|
||
sidecar **v2.1.0** protocol 5. `--verify` first (`add=23 change=1`, nothing written), then the real
|
||
run. `dotnet build` on the synced tree: **0 errors** — the released overlay compiles against stock
|
||
ServUO 57.4. The shard booted, printed `[Bridge] enabled=True`, and dialled the sidecar; the site
|
||
reported `status=connected pluginConnected=true`. The website was a **fresh `main` clone** and
|
||
`module-uo` **v1.1.0 installed from its release manifest** through the `MODULES` declaration — sha
|
||
verified and unpacked by core, never copied in by hand. SMTP was configured against a mail catcher
|
||
and its test send arrived.
|
||
|
||
**One email and one in-app item, to the linked owner, and nothing to anyone else.** Forcing
|
||
`0x400143DB` from IDOC to **Greatly** produced `event dispatched {rules:1, enqueued:2, deduped:0,
|
||
cooled:0, capped:0, cancelled:0}`, and after the rule's grace window both rows sent on the first
|
||
attempt. **Every row in every delivery table names the linked user and no other** — send log 2,
|
||
outbox 2, inbox 1, and nothing for the admin or for the second, unlinked player. The mail was the
|
||
in-universe body, correctly rendered: *"A notice concerning “Yewcross”"*, the stage in words, the
|
||
detail line, an absolute link to `/uo/houses` and a working unsubscribe link — no empty href, which
|
||
is the Phase 11b defect class staying fixed.
|
||
|
||
**Two defects it found, both in shipped core, neither visible to any test:**
|
||
|
||
1. **The one-shot rule-group guard is not atomic.** `moduleSeeds.js` `seedRuleGroup()` reads its
|
||
settings stamp, inserts the whole group, and writes the stamp **after the loop**. Two processes
|
||
booting within the same moment both read "not seeded" and both insert: the rig ended up with
|
||
**52 UO rules where the module ships 26**, under one stamp. The function's own comment states the
|
||
cost — *"a duplicate rule is two mails per event"* — which is exactly what its guard fails to
|
||
prevent. It was reached here by two app processes starting milliseconds apart, which is what a
|
||
`--scale app=2` or a rolling restart does on purpose. The fix is to claim before inserting: an
|
||
atomic conditional write of the stamp, or an insert idempotent per (owner, group, trigger).
|
||
Same shape as Phase 4a's `foundRows` finding — a guard that reads before it claims.
|
||
2. **A module-seeded rule that ships a `digest` body cannot be saved from the Rules screen.**
|
||
`checkSeedRule` permits `digest` explicitly, in as many words — *"a template slot rather than a
|
||
channel … allowed here and absent from `channels`"* — and sixteen in-universe UO rules ship one.
|
||
`engagementRules.model.js` then rejects any `templateKeys` entry that is not one of the rule's
|
||
channels, so an operator who opens **House — decay warning** and presses Save gets a 400 naming a
|
||
key they never typed. The only way to save is to delete the digest body, which silently removes
|
||
digest support from that rule. The two validators have to agree about what `digest` is.
|
||
|
||
**Both are fixed in website#181.** `settings.db` gained `claim(key, value)` — the same
|
||
`INSERT IGNORE` as `seedDefault`, reporting its own `affectedRows`, so exactly one caller can win a
|
||
key — and both seeders now claim before inserting rather than stamping after. The trade the code
|
||
already documented is unchanged, only its order: a process that dies mid-loop leaves the group
|
||
stamped and partly seeded, which both functions already called the better failure. The stubs are
|
||
`claim`-shaped now and each suite gained a test that runs two seeders under `Promise.all`; reverting
|
||
the fix makes the module one report every rule inserted twice, which is how the fix was checked. And
|
||
`engagementRules.model.js` now admits `digest` as the one key that is a slot rather than a channel,
|
||
so the two validators agree; anything else is still refused, with a test in each direction.
|
||
|
||
**Also still open:** nothing else. `.profile/README.md` (step 8) landed — the org lead called the
|
||
engagement system a headline capability — and Phase 12's second finding became **Phase 14**.
|
||
|
||
---
|
||
|
||
### Phase 14 — Retention: the engagement schema has no sweep
|
||
|
||
**Status: BUILT, 2026-09-01.** Phase 12 found this and recorded it without fixing it (finding 2 of
|
||
its as-built), on the grounds that a sweep is a `website` change and outside a documentation phase.
|
||
Nothing after it picked the finding up, so it was stated here as the phase it always was — and this
|
||
is that phase. Six decisions were settled by the org lead before any code; the as-built is at the end
|
||
of this section.
|
||
|
||
**Four tables grow without bound**, and they are not one problem with one horizon:
|
||
|
||
| Table | What accumulates | The constraint on a horizon |
|
||
| --- | --- | --- |
|
||
| `engagement_cooldowns` | one row per (rule, user, subject, channel) per fire, read once per fire | **`engagementCooldowns.db.prune(olderThan)` already exists and has no caller.** The horizon must exceed the longest `cooldown_seconds` on any enabled rule, or a pruned row makes the next fire a *first* fire — the model says so and leaves it to the caller |
|
||
| `engagement_outbox` | every enqueued delivery, including terminal `sent` / `failed` / `cancelled` / `suppressed` rows | only terminal rows are eligible; a `scheduled` row may be days out by design (`delay_seconds`) |
|
||
| `engagement_sends` (the send log) | one row per delivery attempt | **two live readers.** The per-rule hourly cap reads `idx_engs_rule_window (rule_id, created_at)`, so a horizon under an hour breaks Q3's ceiling; Admin → Engagement → Send Log is the operator's only answer to "was this person told", so a short one blinds the screen that exists to be looked at |
|
||
| `engagement_suppressions` | one row per suppressed address, forever | **the odd one out, and probably correct as it is.** A suppression is a standing decision; ageing out a `bounce` row means the next send re-mails an address that already hard-bounced, which is exactly how a sender loses a domain's reputation. If anything expires here it is `unverified`, and that is a decision to take rather than a default to assume |
|
||
|
||
**The shape is already in the tree, twice.** `utils/teamActivityPrune.js` and
|
||
`utils/userNotificationsPrune.js` are the same worker — `setInterval` + `unref` + `stop()`, a horizon
|
||
read from a `*_retain_days` setting, started and stopped in `server.js`. Phase 7 wrote the second one
|
||
for `user_notifications` after finding that table had no policy either, with one policy difference
|
||
worth carrying: it deletes **read items only**, because age alone would destroy the evidence for "I
|
||
was never told". The same question has to be answered per table here rather than assumed.
|
||
|
||
**Documentation this owes.** `runicgateway.com` again, and for Phase 12's reason: `/privacy` and the
|
||
generated **Play Data Safety** answers currently publish, for the `deploy-engagement` row, *"kept
|
||
until the operator removes them; nothing here expires on its own"*. That is the true answer today and
|
||
becomes false the moment this lands, and it is generated from one inventory (`src/data/collection.mjs`)
|
||
rather than written twice. Also `BACKEND_DESIGN.md`'s table inventory, and an operator-facing note for
|
||
whichever settings keys this adds.
|
||
|
||
**For the org lead, before any code:** the four horizons (or three, plus "suppressions do not
|
||
expire"), and whether the send log's retention is allowed to be shorter than the operator screen's
|
||
usefulness — the alternative is that the screen learns to say "older entries have been swept" rather
|
||
than showing a silently truncated history.
|
||
|
||
**Acceptance:** every one of the four tables has a stated policy — a sweep with a horizon, or a
|
||
recorded decision that it does not expire and why; the cooldown horizon is checked against the
|
||
longest enabled rule's cooldown rather than picked; `/privacy` and the Play answers are regenerated
|
||
from the inventory; and a rig run shows the sweep deleting terminal rows while leaving a `scheduled`
|
||
outbox row and an in-window send-log row alone.
|
||
|
||
#### Phase 14 as built
|
||
|
||
**The six decisions, settled by the org lead before any code.** Four horizons were put with the facts
|
||
the tree supplied, and two of the four answers went past what was offered:
|
||
|
||
1. **`engagement_cooldowns` — 30 days**, plus the runtime guard. (`MAX_COOLDOWN_SECONDS` is a
|
||
validated 86 400, so even 2 days is provably safe; 30 was chosen for headroom.)
|
||
2. **`engagement_outbox` — terminal rows only, 30 days, plus a stuck-`sending` reaper.** The reaper
|
||
half turned out to be half-built already — see finding 2.
|
||
3. **`engagement_sends` — admin-configurable, 90 / 180 / 365 / custom.** This is the answer that
|
||
widened the phase: none of the three offered options had a UI, and the other two retention workers
|
||
in this codebase (`teamActivityPrune`, `userNotificationsPrune`) keep their horizons in invisible
|
||
`settings` rows. It bought the phase a screen, and the screen took the other two horizons with it —
|
||
"what does this deployment keep" is one question.
|
||
4. **`engagement_suppressions` — never expires, plus an admin purge.** The purge half also went past
|
||
the offer, and cost a Phase 9 decision — see finding 1.
|
||
|
||
**As built.** `utils/engagementRetentionPrune.js` (one worker, three sweeps, batched 1000 × 50, each
|
||
table's failure caught on its own), `model/engagement/engagementRetention.model.js` (the policy, its
|
||
bounds and the cooldown check), two `.db` additions (`outbox.pruneTerminal`, `sends.prune`) and a
|
||
`limit` on the cooldown prune that already existed and had never had a caller. Three routes
|
||
(`GET`/`PUT /admin/engagement/retention`, `DELETE /admin/engagement/suppressions/by-hash/:hash`), a
|
||
new Admin → Engagement → **Retention** screen, a per-row **Lift** button on Suppressions, and one line
|
||
under the Send Log's pager. Two indexes (`idx_engo_sweep`, `idx_engs_sweep`) — every existing index on
|
||
those tables has `created_at` in *second* position, which serves a per-rule window and is useless to a
|
||
whole-table horizon. Reference: `BACKEND_DESIGN.md` § *Engagement retention*.
|
||
|
||
**Three findings.**
|
||
|
||
1. **The per-row purge cost a Phase 9 decision, and there was no way to have it otherwise.** Phase 9
|
||
deliberately stripped `address_hash` from every listed row — a sha256 of every address on the
|
||
deployment is an offline dictionary attack waiting to be run — and the consequence, which its own
|
||
screen shows, is that the only way out of the list was a `window.prompt` asking the operator to
|
||
retype the full address. **They do not have it**: the screen stores and shows a mask. So the row
|
||
had no handle at all, and a per-row button was not a UI change but a reversal. Taken knowingly:
|
||
the route is admin-only and an admin can already suppress and unsuppress any address they can
|
||
name, so the hash grants no capability they lack. `GET /sends` still strips its own.
|
||
|
||
2. **`reclaimStale` could not give up, so the outbox sweep would not have been a bound.** The
|
||
stuck-`sending` reaper decision 2 asked for already existed and already ran every worker tick —
|
||
but it returned *every* stale row to `scheduled`, and `MAX_ATTEMPTS` is consulted only on a
|
||
graceful `retry` outcome. A send that killed the process mid-flight therefore cycled
|
||
sending → scheduled → sending **forever**: never terminal, therefore never eligible for the
|
||
terminal-only sweep this phase was adding. Fixed by failing an exhausted row *before* reclaiming
|
||
the rest; the order is the fix, and `engagementRetentionSql.test.js` runs the wrong order
|
||
deliberately to show what it avoids.
|
||
|
||
3. **The cooldown warning is unreachable through the UI, by construction.** `MAX_COOLDOWN_SECONDS`
|
||
(86 400) is smaller than the cooldown horizon's floor (2 days), so no rule that can be *saved* can
|
||
trip the guard. It is not therefore dead: it fires for a hand-edited row, and it is the thing that
|
||
catches a future raise of `MAX_COOLDOWN_SECONDS` that forgets this floor. Proved live by setting a
|
||
5-day cooldown by hand — the screen and the sweep both warned, and the sweep ran anyway.
|
||
|
||
**The acceptance line, discharged.** All four tables have a stated policy (three sweeps and one
|
||
recorded refusal). The cooldown horizon is checked rather than picked. `/privacy` is regenerated from
|
||
`collection.mjs` — **and the Play answers correctly did not move**, which the phase text did not
|
||
expect: `deploy-engagement` is `scope: 'deployment'`, and `PLAY_DATA_SAFETY.md` is generated from the
|
||
`app`-scoped entries only, because Play asks what the *app* collects rather than what a self-hosted
|
||
deployment keeps. `npm run play:datasafety` rewrites the file byte-identically and
|
||
`check:datasafety` stays green. The rig run happened twice: `engagementRetentionSql.test.js`
|
||
against a throwaway database (7 tests), and the live stack, where a 90-day-old `cancelled` row was
|
||
swept and a 90-day-old `scheduled` row survived.
|
||
|
||
**One rig trap worth not re-deriving.** The first live run appeared to delete the `scheduled` row —
|
||
the acceptance case, apparently failing. It had not: the outbox **worker** was running in the same
|
||
server, the planted row's `due_at` was 90 days in the past, so the worker legitimately claimed and
|
||
sent it before the sweep saw it. **Plant the scheduled row with a FUTURE `due_at`**, or the worker,
|
||
not the sweep, is what the test is measuring.
|
||
|
||
---
|
||
|
||
### Sequencing
|
||
|
||
```
|
||
Phase -1 fast-forward every edge to main; create edge in android-app,
|
||
runicgateway.com and Integration-kit ← blocking, §6.0a
|
||
|
||
Stage A 1 ── 1b ── 2
|
||
Stage B └─ 3 ── 4 ── 5a ── 5b ── 6
|
||
Stage C └─ 7 ── 8 (8 = app-store cadence)
|
||
Stage D └─ 9
|
||
Stage E 10 ──────────────────────────────────── 11a ── 11b (10 parallel from day one; 11 needs 6 + 10)
|
||
Stage F 12 ── 13 (12 written before 13, merged in its window)
|
||
Stage G └─ 14 (retention; after the cutover, so straight to `main`)
|
||
|
||
── all of the above onto `edge` ──
|
||
13 is the only thing that touches `main`
|
||
...and 14, which comes after it
|
||
```
|
||
|
||
**Phase -1 is blocking and takes minutes.** Every `edge` is 0 ahead / 3–16 behind `main` (§6.0a), so
|
||
the fast-forward is lossless; skipping it means the cutover diff carries other workstreams' leftovers.
|
||
|
||
Phases 1, 1b and 10 can start immediately and in parallel; **Phase 2 now follows 1b** rather than 1,
|
||
because a trigger's audience resolves to users and the identity those users are mailed at should be
|
||
unique and verifiable before anything resolves an audience over it. **Phase 1 and Phase 6 are the two that touch
|
||
mail people actually receive** and should each land alone: Phase 1 because it can silently stop email
|
||
for the live deployment (§1.2a), Phase 6 because it rewrites the pipeline behind notifications going
|
||
out today. Both get the local rig exercised before merge, not only tests.
|
||
|
||
5a can land before 4 if that sequencing is more convenient — the seeded templates and renderer have no
|
||
dependency on the engine, only on Phase 1's channel interface. The order above simply keeps the engine
|
||
provable before anything renders through it.
|
||
|
||
**Phase 12 is the one with a deadline rather than a dependency.** `runicgateway.com`'s checks read the
|
||
source repos' `main`, so the site stays green through the whole `edge` period and breaks at the cutover.
|
||
Its work therefore has to be finished *before* Phase 13 and merged *inside* the same window — the
|
||
failure mode of leaving it until after is a public site making false claims about the product on the
|
||
day it ships.
|
||
|
||
---
|
||
|
||
## Part 7 — Open questions and forward-compat notes
|
||
|
||
### 7.1 Questions for the org lead — all nine answered
|
||
|
||
1. ✅ **ANSWERED — may unverified addresses receive engagement mail?** *"Emails need to be unique and
|
||
verification blocking sending is an admin setting."* Combined with the opt-in answer, this settles
|
||
three things at once and creates **Phase 1b**:
|
||
- **Opt-in only**, following standard marketing-email practice — explicit consent, working
|
||
unsubscribe, suppression. This matches the `team_notification_prefs` pattern already in use.
|
||
- **`users.email` becomes UNIQUE.** Not a schema tweak — see §0.6 for the three code paths it breaks
|
||
and the boot it can stop, and Phase 1b for the work. Duplicates are resolved **oldest-wins**: the
|
||
earliest account keeps the address, later ones are nulled and listed in an admin report.
|
||
- **The verification gate is an admin setting**, default **on for fresh installs, off for upgrades**,
|
||
so a running deployment does not silently stop mailing its opted-in users at cutover.
|
||
- The collision error stays **generic and anti-enumeration**, rate-limited but **not bot-scored**.
|
||
|
||
*Consequence for ordering:* Phase 9 no longer blocks Phase 11 — the verification *mechanism* moves
|
||
forward into 1b, and Phase 9 keeps only bounces and suppression.
|
||
|
||
**The narrower half is settled (2026-08-31, at the start of Phase 9): an unverified address is
|
||
excluded at ENQUEUE, and only from the email channel.** With the gate on, `emailChannel.eligible`
|
||
drops the user before an outbox row is written, so nothing is queued only to be thrown away and
|
||
the admin reach preview can report the exclusion. It is email-only because a rule spanning
|
||
channels must still put an item in that person's in-app inbox. Transactional mail — resets,
|
||
invites, verification itself — is unaffected either way.
|
||
2. ✅ **ANSWERED — multi-instance.** Neither of the two the question offered, and the third is
|
||
better than both: the outbox sweep **claims each row with a compare-and-set** —
|
||
`UPDATE … SET status='sending' WHERE id=? AND status='scheduled'` — and the instance the server
|
||
reports `affectedRows = 1` to owns it. §4.2a's status ENUM already carried a `sending` state that
|
||
nothing else needed, so this is what the schema was shaped for; it needs no open transaction (which
|
||
nothing else in this codebase's workers does) and no MariaDB version floor. It makes the **outbox**
|
||
safe for two app instances and does not, on its own, make the deployment multi-instance — the four
|
||
existing workers are still written for one. Built in Phase 4a.
|
||
3. ✅ **ANSWERED — rules as data vs. rules as code.** Data, as recommended: rules are operator-editable
|
||
rows, `enabled` defaults to `0`, and every rule carries a **hard per-rule hourly send ceiling**. The
|
||
ceiling is not a nicety — it is the thing that keeps a misconfigured rule from becoming a mail storm,
|
||
and it is what makes "data" safe enough to choose over "code". Phase 4 builds both.
|
||
4. ✅ **ANSWERED — the engagement admin surface gets its own top-level nav group**, "Engagement",
|
||
beside Content / Moderation / System. Four screens is too much to bury: Settings is already one long
|
||
page of sections, and a send log is a paged table rather than a settings section. Rules lands with
|
||
Phase 4b; Triggers, Templates and the send log join it in Phase 5. **Email Delivery stays a section
|
||
of Settings** for now — moving it is not part of either phase.
|
||
5. ✅ **ANSWERED — which SMTP posture is the documented default?** Document all three; **lead with a
|
||
relay** (Mailgun/SES/Postmark); name Gmail-with-an-app-password (`smtp.gmail.com:587`) explicitly as
|
||
the migration path off OAuth2 for the existing deployment (§1.2a). Phase 1 owes all three in
|
||
`runicgateway.com`'s `notifications-and-email.mdx` as well as `BACKEND_DESIGN.md` §7.
|
||
6. ✅ **ANSWERED — time-based triggers (G25), design now, build when?** As recommended: the
|
||
declaration lands in Phase 2 so `kind: 'scheduled'` is in the contract, the manifest and every
|
||
stored declaration from day one; the evaluator is built after Phase 9. Registration accepts
|
||
`scheduled` today and `ctx.events.emit` **refuses** to fire one — an evaluator's trigger is not a
|
||
caller's — so `kind` means something from the moment it is declarable rather than from the moment
|
||
it is honoured. The lifecycle uses in §8.5 are the highest-value non-game triggers on the list and
|
||
the first thing anyone will ask for after the IDOC mail works.
|
||
7. ✅ **ANSWERED — manual/operator-authored sends.** *"There is no campaign in the normal sense of email
|
||
marketing. But admins can create all sorts of trigger conditions"*, and separately: *"lists can be
|
||
built if they are powered by game data — say team X members or governors or whatever — thru the uo
|
||
module; same surface will be exposed to all modules."*
|
||
|
||
So: **no campaigns surface, no operator-authored send screen, no free-form list building.** An
|
||
admin's expressive power lives in trigger conditions and rules. `kind: 'scheduled'` therefore stays in
|
||
the Phase 2 contract as an admin-definable *condition*, not as a campaign.
|
||
|
||
**Lists do exist, module-declared.** A module registers named audiences over its own data on a surface
|
||
core exposes to every module, and an operator may compose them with and/or/not into a saved segment.
|
||
Composition must **narrow, never widen** — the segment takes the tightest ceiling it contains and is
|
||
still checked against the trigger's G24 ceiling. §5.1a is the design; Phase 2 owns the surface,
|
||
Phase 4 the composition UI, Phase 11 `module-uo`'s first real audiences.
|
||
8. **Android CI on `edge`** (§6.0a). **ANSWERED 2026-08-31 (org lead), at the start of Phase 8: fix
|
||
it, as the phase's first commit.** `pr-checks.yml` triggered only on PRs into `main`, so Phase 8
|
||
would have landed with zero CI and Phase 13 would have been its first real build — as happened to
|
||
all nine M12 phase PRs. `pull_request.branches` is now `[main, edge]`. **`sonarqube.yml` was
|
||
deliberately left alone**: it is a push-on-`main` analysis rather than a PR gate, so no phase PR
|
||
was ever expected to run it.
|
||
9. ✅ **ANSWERED (2026-08-31, at the start of Phase 11) — core's own `news.post` emitter, and the
|
||
three other things a publish already fires.** The three sub-questions below are all settled:
|
||
**the emit REPLACES `pushDispatch.publish`, and core seeds a `news.post` rule DISABLED** — the
|
||
Phase 6 Team precedent, applied unchanged. **It gates on `enqueueIfNeeded`'s job id**, the existing
|
||
transition signal, re-read nowhere. **Phase 11 owns it**, in the 11a half.
|
||
|
||
**State the consequence plainly, because it is the cost of the answer and it is deliberate: news
|
||
push stops on upgrade.** A deployment that ships this and does nothing else tickles nobody on the
|
||
next publish, until an operator opens Admin → Engagement → Rules and switches the seeded rule on.
|
||
The alternative — keeping the raw tickle beside the emit for one release — was considered and
|
||
refused for the same reason Phase 6 refused seeding enabled: an exception carved for the one
|
||
pipeline that predates the engine is an exception that has to be re-argued every time somebody
|
||
reads the invariant, and "the tickle stays for one release" is an exception with a deadline nobody
|
||
owns. The mitigations are the ones Phase 6 already built and they are obligations of Phase 11, not
|
||
nice-to-haves: **`EngagementRules.jsx`'s banner must name news alongside Teams**, and **Phase 13's
|
||
release note must name it as an upgrade step**.
|
||
|
||
The original question and its analysis follow, because the table below is still the map of what
|
||
must *not* be disturbed.
|
||
|
||
`config/coreTriggers.js` declares core's five triggers and says in as many words that **nothing here
|
||
emits yet**; Phase 6 migrated only the four `team.*` ones onto the engine. So `news.post` is a
|
||
declared payload contract with **no caller**, and on a real deployment the only in-app or email items
|
||
a rule can produce today come from Teams. Phase 7 found this and deliberately did not wire it, on the
|
||
grounds that a channel's own phase is not the place to give another phase's trigger an emitter.
|
||
|
||
**The call itself is one line. The care is entirely in what it must not disturb.**
|
||
`admin.controller.js`'s `announceIfNewlyPublished` already fans one publish four ways, and they are
|
||
different in kind — three of them are *not* the engagement engine's business:
|
||
|
||
| What fires on a publish | Whose | What kind of thing it is | The engine's? |
|
||
| --- | --- | --- | --- |
|
||
| `announceJobs.enqueueIfNeeded` → `announce_job_legs` | core, with legs registered by modules — `module-uo` owns `towncrier` | a one-shot **delivery to a channel of the deployment**, with retry and classification | **No** |
|
||
| `registries.dispatchPostHook('onSaved')` | modules, via `registerPostHook` (API 1.1.0) | idempotent **state mirroring** — it also runs on delete, and refreshes silently on an edit | **No** |
|
||
| `pushDispatch.publish('news.post', { ref })` | core | a **per-person notification**, to whoever subscribed | **Yes — this is the one that should become an emit** |
|
||
| *(missing)* `engagementEmit.emit('core', 'news.post', …)` | core | rules → email / in-app / push | **Yes** |
|
||
|
||
`registries.js` already states the first two apart and why they were not folded together ("a leg is a
|
||
one-shot DELIVERY with retry and classification; a post hook maintains idempotent STATE, has to run on
|
||
delete as well as save, and refreshes silently on an edit"). **Adding the engine makes a third
|
||
distinction of the same kind, not a replacement for either.** A module's town-crier leg and a module's
|
||
news gump must keep firing exactly as they do; what changes is only that the raw tickle stops being
|
||
the one person-facing sink and becomes one channel of a rule.
|
||
|
||
**What a module gets out of this, stated so nobody widens it by accident.** A module already has two
|
||
doors onto a news publish — the announce leg and the post hook — and it keeps both. What it does *not*
|
||
get is the ability to fire `news.post` itself: the id's owner is core, `ctx.events.emit` binds the
|
||
owner at the call and never reads it from the arguments, and §7.2's one namespace means an id has
|
||
exactly one owner across both facets. A module that wants a person-facing notification of its own
|
||
declares its own trigger through `registerEventTriggers`. That is the whole of "modules can use it".
|
||
|
||
**Three things to settle before anyone writes the line — all three settled above:**
|
||
- **Continuity, and it is the same shape as G22 and Phase 6's decision 3.** Today publishing news
|
||
tickles every `news.post` subscriber directly. If the emit *replaces* that call, push stops the
|
||
moment this lands and stays stopped until an operator enables a rule — silently, because `enabled`
|
||
defaults to `0`. Either core seeds a `news.post` rule (and then: enabled, against the standing
|
||
default, or disabled with a banner as the Team rules got?), or the raw tickle stays beside the emit
|
||
for one release and is removed once a rule is known to exist.
|
||
- **The transition signal must be reused, not re-derived.** `enqueueIfNeeded` returning a truthy job
|
||
id is the single "newly published news" test, and the push call already piggybacks on it
|
||
deliberately so an edit or a re-publish does not re-fire. The emit must gate on the same value; a
|
||
second reading of the transition is a second chance to disagree with the first.
|
||
- **Which phase owns it.** Recommendation: **Phase 11**, which already ships "the first real rule" and
|
||
is where a declared trigger first gets a caller. It is core work rather than `module-uo`'s, so that
|
||
phase's title understates it — say so there rather than inventing a phase for one call site.
|
||
|
||
### 7.2 One namespace, or two?
|
||
|
||
`notification_channel_prefs.stream_id` above holds *either* a push stream id *or* a trigger id. Streams
|
||
and triggers are close enough to collide conceptually — `idoc.warning` (a stream) and
|
||
`uo.house.idoc_warning` (a trigger) would describe the same game moment. Two options:
|
||
|
||
- **One namespace**: a trigger *is* a stream with a payload contract; the seven grandfathered ids stay as
|
||
contract-free triggers. Cleanest long-term, but touches the shipped app's catalog.
|
||
- **Two namespaces**: streams stay exactly as they are (push only), triggers are new and separate.
|
||
Zero risk now, permanent duplication later.
|
||
|
||
Recommendation was **two namespaces through Phase 8**, on the grounds that the shipped Android client
|
||
reads the stream catalog.
|
||
|
||
✅ **ANSWERED 2026-08-29 by the org lead: ONE namespace.** A trigger is a payload contract attached to
|
||
an id that may also carry a subscription toggle; `news.post` names one event whichever question is
|
||
being asked of it. They stay two REGISTRATIONS with two shapes — a stream entry is a subscription
|
||
toggle and says nothing about payload — but **an id has exactly one owner across both facets**, and
|
||
that is checked in both directions.
|
||
|
||
The risk the recommendation was hedging against does not materialise, and that is worth recording
|
||
because it is the part that looked expensive: **the push catalog is untouched.** `allStreams()` still
|
||
serves the stream facet only, so the shipped app sees exactly the seven-plus-five ids it saw before,
|
||
and a trigger-only id simply has no push toggle. What one namespace actually costs is the ownership
|
||
check; what it buys is `notification_channel_prefs.stream_id` staying single-keyed, where two
|
||
namespaces would have forced a `kind` discriminator into its primary key and left `news.post` naming
|
||
two things forever. See Phase 2's as-built for the two knock-on effects (a relaxed id grammar and a
|
||
shared legacy allowlist) that only appeared once it was implemented.
|
||
|
||
### 7.3 Forward-compat: the note the brief asked for, corrected
|
||
|
||
Event-name collision handling is **already implemented** (§0.4), so the brief's §6 concern is closed.
|
||
The real forward-compat note is different, and belongs in `MODULE_API.md`:
|
||
|
||
> **A trigger id is namespaced by its owner and collision-checked at registration, exactly as a
|
||
> notification stream is. What is *not* yet expressed is a rule or template referring to a trigger whose
|
||
> module has been uninstalled.** `engagement_rules.trigger_id` is a plain `VARCHAR`, deliberately, so a
|
||
> module can be removed and reinstalled without the operator's rules being destroyed — the same decision
|
||
> `announce_job_legs` took for a leg whose module is gone ("Leave it alone: failing it would make the job
|
||
> roll up terminal on the strength of a leg that no longer exists, and reinstalling the module should
|
||
> resume it"). A rule whose trigger is unregistered must therefore show as **dormant** in the admin UI,
|
||
> never as an error and never auto-deleted.
|
||
|
||
### 7.4 Documentation obligations — the whole surface
|
||
|
||
**§6.0b assigns each of these to the phase that causes it; this is the inventory it draws from.**
|
||
Nothing here is "documentation to do at the end" — a phase is not done until its rows have landed.
|
||
|
||
- `docs/website/ENGAGEMENT.md` — this document, as the design of record.
|
||
- `docs/website/BACKEND_DESIGN.md` — §7 (Email) **rewritten, not amended**: it currently documents Gmail
|
||
OAuth2 as the mechanism, and every mention of "Connect Gmail", the borrowed `google` client and the
|
||
`mail.google.com` scope goes with it (§1.2a). The table inventory gains eight tables; the route tables
|
||
gain the engagement and channel-preference surfaces and lose the two `/admin/email/connect/*` routes.
|
||
- **Operator-facing, for the Gmail removal** — a release note naming the required action, and SMTP setup
|
||
guidance wherever `.env.example` / `website/README.md` currently point an operator at the Connect Gmail
|
||
flow. This is the one documentation obligation with a live deployment depending on it.
|
||
- `docs/website/MODULE_API.md` — §1.1 (1.7.0 + the stale-`edge` correction), §2.3 (`ctx.events`,
|
||
`ctx.inbox`), §2.4 (`registerEventTriggers`), and §7.3's dormant-rule note.
|
||
- `docs/website/TEAMS.md` — §6.3/§6.4 rewritten once Phase 6 migrates the Team pipeline.
|
||
- `docs/link/INTEGRATION.md` + `docs/link/PLAN.md` — Phase 10's wire fields.
|
||
- `docs/android/PLAN.md` — §11 gains the in-app inbox and the per-channel preferences.
|
||
- `docs/modules/uo/API.md` + `README.md` — Phase 11's triggers.
|
||
- `integration-kit/` — `ci/core-ref.json` moved to the new pin; chapter 2 gains a section on registering
|
||
a trigger, since the kit currently teaches none of the registries. The kit **teaches and never
|
||
re-specifies**, so it links to `MODULE_API.md` rather than restating the contract.
|
||
- **`runicgateway.com`** — the largest single obligation and the only one whose checks *fetch* their
|
||
facts rather than being told them, so it fails on its own. Full detail in Phase 12; in summary:
|
||
`src/data/platform.json` (`protocol` 4→5, `moduleApi` 1.6.0→1.7.0, the bundle triple) ·
|
||
`administration/notifications-and-email.mdx` (whose "There is no SMTP option" aside becomes false) ·
|
||
`configuration.mdx`, `troubleshooting.mdx`, `architecture/system-architecture.mdx`,
|
||
`architecture/protocol-versions.mdx` · `src/data/capabilities.mjs` (the Notifications line claims a
|
||
web channel that does not exist until Phase 7) · new admin pages for rules, templates and per-channel
|
||
preferences · **`PLAY_DATA_SAFETY.md` + `/privacy`**, both generated from one inventory that an
|
||
engagement mailer materially changes.
|
||
- `.profile/README.md` — only if this lands as a headline capability. **It did:** the org lead
|
||
called it one, and the org landing page names the engagement system in the Phase 13 window.
|
||
|
||
---
|
||
|
||
## Part 8 — What the system could be used for
|
||
|
||
Written as a catalogue, not a commitment — to check that the design in Parts 3–5 is general enough, and
|
||
to show what an operator gets for the phases they pay for. **§8.6 is no longer that.** On 2026-08-31 the
|
||
org lead scheduled **every ✅ row of §8.6 into Phase 11**, so for the `module-uo` section the burden of
|
||
proof has inverted: a row that does not ship needs a reason recorded in the phase. The rest of Part 8 —
|
||
§8.1–8.5 and §8.7 — remains unscheduled and stays a catalogue.
|
||
|
||
Read the availability column as: **✅** the data exists today and a rule could be written the day the
|
||
engine lands · **⚠** the data exists but needs a mapper or a resolver · **❌** needs new plumbing
|
||
(named in the row).
|
||
|
||
### 8.1 Core — account, access and security
|
||
|
||
| Use | Trigger source | Data | Channels | Timing |
|
||
| --- | --- | --- | --- | --- |
|
||
| Password reset | `password_resets` | ✅ (a sender exists today) | email | instant |
|
||
| Account invite | `user_invites` | ✅ | email | instant |
|
||
| Email verification | Phase 9 | ❌ no verification flow yet | email | instant |
|
||
| **New sign-in from an unrecognised device** | `trusted_devices` | ⚠ the table exists; nothing notifies | email, in-app | instant |
|
||
| **A trusted device was added / all devices revoked** | same | ⚠ | email, in-app | instant |
|
||
| **2FA enabled / disabled, recovery codes regenerated** | `users.totp_enabled`, `recovery_codes` | ⚠ | email, in-app | instant |
|
||
| **Password or email changed** | `/auth/me` account routes | ⚠ | email | instant |
|
||
| **Your account was locked by the bot-score / IP-ban layer** | `middleware/botScore` | ⚠ | email | instant, hard cooldown |
|
||
| **SSO identity linked or unlinked** | `user_identities` | ⚠ | email, in-app | instant |
|
||
| **Your session was revoked everywhere** | `revoked_sessions` | ⚠ | in-app | instant |
|
||
|
||
The security family is the strongest argument for the in-app channel: every one of these is something a
|
||
user should be able to *review a history of*, not merely be pinged about once.
|
||
|
||
### 8.2 Core — content and community
|
||
|
||
| Use | Trigger source | Data | Channels | Timing |
|
||
| --- | --- | --- | --- | --- |
|
||
| New news post / Five-on-Friday | `posts` + `news.post` stream | ✅ | email, push, in-app | instant or digest |
|
||
| Team forum reply / new thread | `team_forum_posts` | ✅ (shipping today) | email, push, in-app | instant or digest |
|
||
| Team announcement from a leader | `team.announcement` stream | ✅ | email, push, in-app | instant |
|
||
| Joined / removed from a Team | `team.member.*` streams | ✅ | in-app, email | instant |
|
||
| Team leadership changed | `team.leadership.changed` | ✅ | in-app | instant |
|
||
| **Someone replied to *your* thread** | `team_forum_posts.parent` | ⚠ author resolution exists; no per-author rule | email, in-app | instant |
|
||
| **You were mentioned in a post** | forum body | ❌ no mention parsing | in-app, push | instant |
|
||
| **A wiki page you edited changed** | `wiki_revisions` | ⚠ revisions exist; no watch list | email | digest |
|
||
| **A wiki page you watch changed** | — | ❌ needs a watch table | email | digest |
|
||
| **Weekly "what happened" digest** | posts + forum + activity | ⚠ digest machinery lands in Phase 6 | email | weekly |
|
||
| **Your uploaded image was removed by the sweep** | `team_forum_uploads` | ⚠ | in-app | instant |
|
||
|
||
### 8.3 Core — moderation, appeals and reports
|
||
|
||
| Use | Trigger source | Data | Channels | Timing |
|
||
| --- | --- | --- | --- | --- |
|
||
| **You received a warning** | `warnings` | ⚠ | email, in-app | instant |
|
||
| **You were muted / banned, and why** | `mod_actions` | ⚠ | email, in-app | instant |
|
||
| **Your appeal was received / decided** | `appeals` | ⚠ | email, in-app | instant |
|
||
| **Your content was reported** *(to staff, not the author)* | `content_reports` | ⚠ | in-app, email | instant, staff audience |
|
||
| **A moderation request needs a second pair of eyes** | `team_moderation_requests` | ⚠ | in-app, email | instant, staff |
|
||
| **Report queue is over N items** | `content_reports` count | ❌ needs a threshold evaluator | email, in-app | daily digest |
|
||
|
||
Note the audience direction: several of these go to **staff**, not to the subject. `engagement_rules.audience`
|
||
has to support a role-derived audience (`'staff'`, `'admins'`) as well as `'owner'` and `'subscribers'` —
|
||
worth confirming in Phase 4 rather than discovering in Phase 11.
|
||
|
||
### 8.4 Core — operator and staff operations
|
||
|
||
| Use | Trigger source | Data | Channels | Timing |
|
||
| --- | --- | --- | --- | --- |
|
||
| **Email delivery is failing** | `email_config.status` | ⚠ — and note the bootstrap problem: this one cannot be emailed | in-app, Discord | instant |
|
||
| **A module failed to load / is in `startup_failed`** | `modules/loader.js` | ⚠ | in-app, email | instant |
|
||
| **Sidecar unreachable / protocol mismatch** | `uoLinkClient` `ok:false`, `409` | ⚠ | in-app, email, Discord | instant, cooldown |
|
||
| **Announce leg exhausted its retries** | `announce_job_legs.status='failed'` | ✅ | in-app, email | instant |
|
||
| **A new admin was created, or a role was elevated** | `activity_log` | ⚠ | email to admins | instant |
|
||
| **Weekly operator report** — signups, active users, moderation volume, send volume | `activity_log`, `engagement_sends` | ⚠ | email | weekly |
|
||
|
||
`engagement_sends` making the send volume reportable is a small thing that closes a real gap: today
|
||
"how much mail did we send" is unanswerable (G15).
|
||
|
||
### 8.5 Lifecycle and re-engagement — the "engagement drivers" the brief names
|
||
|
||
| Use | Trigger source | Data | Channels | Timing |
|
||
| --- | --- | --- | --- | --- |
|
||
| **Welcome / first steps** after registration | `users.created_at` | ⚠ needs a time-based evaluator | email | delayed (e.g. +1 h) |
|
||
| **Finish setting up** — no linked game account after N days | `shard_links` absence | ⚠ module-supplied predicate | email, in-app | delayed |
|
||
| **We miss you** — no login in N days | `users.last_login_at` | ⚠ needs a scheduled sweep | email | monthly, hard cap |
|
||
| **Come back for X** — a scheduled event is starting | operator-authored | ❌ needs a manual/scheduled trigger type | email, push, in-app | scheduled |
|
||
| **Your invite is about to expire** | `user_invites.expires_at` | ⚠ | email | delayed |
|
||
| **Account dormant, scheduled for cleanup** | policy | ❌ no dormancy policy exists | email | scheduled, staged |
|
||
|
||
**These need a trigger kind the design does not yet have: a *time-based* trigger, not an event-based one.**
|
||
Everything in Parts 3–5 is "a thing happened → maybe send". "Nothing happened for 30 days → send" is a
|
||
periodic evaluator over a query. That is a real addition — a `registerScheduledTrigger({ id, cron, evaluate })`
|
||
whose `evaluate` returns candidate (user, subject) pairs, with the cooldown table doing exactly the job
|
||
it already does. **It should be designed in Phase 2 even if it is built later**, because retrofitting a
|
||
second trigger kind into the rule model afterwards is the expensive version.
|
||
|
||
The same mechanism gives the operator a **manual send**: "announce this to everyone who opted into news",
|
||
which is the one legitimately campaign-shaped use and the one most likely to be asked for first.
|
||
|
||
### 8.6 Module events — `module-uo`, grounded in what the shard actually emits
|
||
|
||
Every kind below is real (`docs/link/INTEGRATION.md`). The trigger id is what `module-uo` registers;
|
||
core stays ignorant of every word in it. **Every ✅ row here is Phase 11 scope** except the four
|
||
struck through or marked carved out, each of which carries its reason.
|
||
|
||
**Three rows were corrected on 2026-08-31**, when the set was scheduled and each claim was checked
|
||
against the emitters rather than against this table. Two of the corrections are what widened Phase 10
|
||
from one wire enrichment to three; the third is a caveat that has to reach the operator, not a defect.
|
||
**Three more were carved out or narrowed on the same day**, at the start of Phase 11, when the same
|
||
check was run against the *website* rather than against the wire — see "Three carve-outs" below.
|
||
|
||
| Trigger | Wire source | Data | Why anyone cares |
|
||
| --- | --- | --- | --- |
|
||
| `uo.house.idoc_warning` | `house.decay` → `Greatly` | ⚠ mapping only (the enrichment is Phase 10) | **The flagship.** Your house is decaying; log in or lose it and its contents |
|
||
| `uo.house.collapsed` | `house.decay` → `Collapsed`, `house.remove` | ✅ | The bad news, so it is not a surprise |
|
||
| `uo.vendor.sale` | `vendor.sale` | ✅ (`ownerAcct` → linked user) — **but patch-tier only**, see below | Your player vendor sold something, for how much |
|
||
| `uo.vendor.expiring` | `vendor.listing` + **v5 fee fields** | ⚠ → ✅ **after Phase 10**; `vendor.listing` carries no `ownerAcct` and no fee state today | Your vendor is about to be dismissed for unpaid fees |
|
||
| `uo.account.login_failed` | **v5 `account.login.result`** | ⚠ → ✅ **after Phase 10**; the existing `account.login.attempt` is pre-decision, see below | Someone tried to log into your game account, and failed, and from where |
|
||
| `uo.character.death` / `murdered` | `player.death`, `player.murdered` | ✅ | Opt-in; a killfeed some players want and most do not |
|
||
| `uo.skill.capped` | `skill.gain` where `base == cap` | ✅ | You hit the cap in a skill — a genuine milestone |
|
||
| `uo.quest.complete` | `quest.complete` | ✅ | Milestone / achievement mail |
|
||
| ~~`uo.guild.joined`~~ | `guild.join` | **carved out — Phase 11** | Core's `team.member.joined` already fires for this; see below |
|
||
| `uo.guild.left` | `guild.leave`, `guild.roster` | ✅ | A member left. Core has no `team.member.left`, so this one has no overlap |
|
||
| `uo.guild.disbanded` | `guild.remove` | ✅ | Consequential and easy to miss |
|
||
| `uo.champ.started` | `champ.update` → active | ✅ (transition tracker exists) | A champion spawn is up — the classic "come online now" driver |
|
||
| `uo.champ.boss_up` | `champ.update` `bossUp` | ✅ | Higher-value version of the same |
|
||
| `uo.governor.elected` | `city.update` | ✅ (transition tracker exists) | Civic events; naturally digestible |
|
||
| `uo.election.opened` | `city.update` `electionPhase` | ✅ | "Voting is open in Britain" — with `autoPickAt` as a real deadline |
|
||
| `uo.server.up` / `down` | `server.hello`, `server.shutdown`, `server.crashed` | ✅ | Downtime notice; **needs a hard cooldown**, a flapping shard is a mail loop |
|
||
| `uo.page.new` | `page.new` | ✅ | A player opened a help page → **staff** audience |
|
||
| `uo.cheat.detected` | `cheat.fastwalk` | ✅ | Staff audience only — this kind is *not* on the public allowlist |
|
||
| `uo.audit.staff_action` | `audit.set`, `audit.command`, `admin.audit` | ✅ | Admin-audience digest of staff activity |
|
||
| ~~`uo.link.requested`~~ | `link.request` | **carved out — Phase 11** | No addressable recipient, and a TTL no channel can beat; see below |
|
||
| `uo.account.unlinked` | `account.unlinked` | ✅ | Someone severed the tie from in-game |
|
||
| `uo.market.item_listed` | `vendor.listing` | ✅ (protocol v3) | Saved-search hit: "a Vanquishing kryss appeared under 50k" |
|
||
| `uo.points.rank_changed` | `points.board` | ⚠ **`subscribers` only** (protocol v3) | The board's top N changed. The personal "you were pushed out" half is carved out — see below |
|
||
| `uo.economy.milestone` | `economy.supply` | ✅ | Operator-facing; economy health thresholds |
|
||
| `uo.world.saved` | `world.save.after` | ✅ | Operator-facing only; world-size trend |
|
||
|
||
**Three rows as originally written were wrong, and finding out cost one sweep of the emitters:**
|
||
|
||
- **`uo.vendor.sale` is real and does carry `ownerAcct` — but it lives in `servuo-plugins/patches/`,
|
||
the opt-in patch tier**, verified only against ServUO 57.4. It is a `PlayerVendorSale` EventSink the
|
||
patches *add* to core plus a subscriber that reads it, which is why it is absent from `overlay/`. A
|
||
shard that declined the tier, or runs another ServUO version, emits this kind never — so the rule is
|
||
silently dormant there rather than broken. Phase 10 deliberately does **not** try to move it into the
|
||
overlay; the obligation is that the seeded rule and the operator docs say which tier it needs.
|
||
- **`uo.vendor.expiring` had no data at all, not merely no mapper.** `vendor.listing` carries
|
||
`ownerSerial` and `ownerName` — no `ownerAcct`, so it cannot even be addressed to a website user —
|
||
and nothing anywhere on the wire carries a vendor's held gold or daily charge. Phase 10 adds all four.
|
||
- **`uo.account.login_attempt` could not have been built as described.** `EventSink.AccountLogin` is a
|
||
pre-decision veto hook; the emitter's own comment says "Fires before the auth decision, so this is an
|
||
attempt, not a result". A rule on it would have mailed "someone tried to log into your account" every
|
||
time the player logged in successfully — the exact inversion that makes people distrust security
|
||
mail. Phase 10 adds a post-decision `account.login.result`, and the trigger is renamed
|
||
**`uo.account.login_failed`** so its id cannot be misread again.
|
||
|
||
**Three others are worth calling out as design pressure on Parts 3–5:**
|
||
|
||
- **`uo.market.item_listed` is a *saved search*, not a plain trigger.** The audience is "users whose
|
||
stored query matches this listing", which `audience: 'computed'` covers but a per-user query store
|
||
does not exist. Scoping note only — it is the most-requested feature of every UO shard site and would
|
||
land as its own workstream on top of this one.
|
||
- **`uo.server.up`/`down` is the cooldown table's real stress test.** A flapping shard emits both kinds
|
||
repeatedly; without `engagement_cooldowns` keyed per rule this is a mail loop, which is exactly why
|
||
the brief is right that cooldowns must exist *before* the first trigger ships.
|
||
- **`uo.cheat.detected` and `uo.audit.*` must never be publicly subscribable.** `shardStreams.js`
|
||
already enforces this for push with a public-allowlist filter (`PUBLIC_KINDS`) and the comment calls it
|
||
defence in depth. **The engagement engine needs the equivalent**: a trigger declares an
|
||
`audience` it is *permitted* to have, and a rule cannot widen it. Without that, the module system's
|
||
security boundary has a second door. **This belongs in Phase 2's trigger declaration, not Phase 4.**
|
||
|
||
**Three carve-outs and one narrowing, settled by the org lead 2026-08-31 at the start of Phase 11.**
|
||
Phase 11 inverted the burden of proof — a ✅ row that does *not* ship needs a reason recorded — so
|
||
here they are. All three were found by checking each row against what the *website* can address,
|
||
which is a different question from whether the wire carries the event. In every case the event keeps
|
||
being ingested and nothing else changes.
|
||
|
||
- **`uo.guild.joined` — duplicates core's `team.member.joined`.** A UO guild *is* a Team and
|
||
`module-uo` is this deployment's Team provider, so `teamSync.notifyRoster` already emits
|
||
`team.member.joined` and `team.leadership.changed` on every roster reconcile, and core already seeds
|
||
(disabled) rules for both. Shipping a UO trigger for the same fact means an operator who enables the
|
||
obvious two rules gets two mails for one join and has to work out why. **`uo.guild.left` and
|
||
`uo.guild.disbanded` still ship** — core has no `team.member.left` and no disband event at all, so
|
||
they are the half of the family with no overlap. The trade given up is latency: `guild.join` is
|
||
real-time and the Team sweep is ~15 minutes. That is the right thing to give up, because a duplicate
|
||
mail is a defect an operator discovers in production and fifteen minutes is not.
|
||
- **`uo.link.requested` — nobody to address, and a deadline no channel can meet.** By construction the
|
||
game account is *not yet* linked, which is the entire point of the event, so an `owner` audience
|
||
resolves to nobody: `shard_account_links` has no row for it, and there is no other tie between that
|
||
account and a website user. Even if there were, `link.request` carries `ttlSec` — a default of five
|
||
minutes — and every channel this system has is slower than that in the worst case and *allowed* to
|
||
be. The player is already looking at the code on their screen when the event fires.
|
||
- **`uo.points.rank_changed` — ships `subscribers` only; the personal half is carved out.**
|
||
`points.board`'s `top[]` entries are `{rank, serial, name, points}` — a **mobile serial**, and
|
||
`shard_account_links` is keyed by game **account**. No general serial→account map exists on the
|
||
website: `shard_online` has one only for characters logged in right now, and `shard_guild_members`
|
||
only for guilded ones. So "you were pushed out of the top ten" would be addressable for some players
|
||
and silently not for others, which is worse than not offering it — a leaderboard mail that reaches
|
||
half the board reads as favouritism. The board-changed feed at `subscribers` needs no owner and
|
||
ships. Making the personal half real means a serial→account map, which nothing else in the tree
|
||
needs, so it waits for something that does.
|
||
|
||
### 8.7 Module events — the game-agnostic shape
|
||
|
||
What a *different* game's module would register, to prove nothing above is UO-specific:
|
||
|
||
| Pattern | UO instance | A survival/Rust-like instance | An MMO instance |
|
||
| --- | --- | --- | --- |
|
||
| **Owned asset is at risk** | house decaying | base decaying, upkeep expiring | plot/housing lease lapsing |
|
||
| **Owned asset changed hands** | house demolished | base raided/destroyed | item traded |
|
||
| **Passive income event** | player vendor sold | shop/market sale | auction house sold |
|
||
| **Identity/security event** | game-account login attempt | server-account login | account login |
|
||
| **Time-boxed world event opening** | champion spawn up | raid window / airdrop | world boss / dungeon rotation |
|
||
| **Civic / social structure change** | governor elected | clan leadership change | guild leadership change |
|
||
| **Personal milestone** | skill capped, quest done | level/blueprint unlocked | achievement earned |
|
||
| **Server availability** | shard up/down | server wipe/restart | realm maintenance |
|
||
| **Staff-facing signal** | help page opened, cheat flagged | report filed, anticheat flag | GM ticket |
|
||
|
||
The engine sees none of the left-hand words. It sees a trigger id, a declared variable set, an audience
|
||
kind and a subject key — which is the test the design has to pass, and the reason §4.3's variable
|
||
contract is not optional.
|
||
|
||
### 8.8 What it is deliberately not for
|
||
|
||
Worth writing down, because the failure mode of an engagement system is scope creep into a marketing
|
||
platform — which the brief explicitly rules out and which would change the security posture.
|
||
|
||
- **No list building, no imported contacts, no audiences of non-users.** Every recipient is a user row
|
||
with a preference. The system has no concept of an address that is not attached to an account.
|
||
- **No open/click tracking, no tracking pixels, no link rewriting.** Beyond the privacy position, it
|
||
would put a per-recipient beacon URL in every mail — a phone-home by construction (§3.2).
|
||
- **No A/B testing, no drip sequences, no lead scoring, no funnels.**
|
||
- **No sending on behalf of a third party.** The transport is the operator's own; there is no
|
||
multi-tenant sender.
|
||
- **No raw HTML authoring and no HTML-typed variables** (§4.6.2). A module supplies data, never markup.
|
||
- **Push stays content-free.** Whatever else this system does, the tickle invariant survives it: a
|
||
channel declares `carriesContent`, and push's is `false` structurally rather than by convention.
|