RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(website): the email channel and the Teams migration as built (engagement Phase 6)

Browse Source 
ENGAGEMENT.md gains the Phase 6 as-built: the seven decisions settled up front,
the audience problem that shaped the phase, why scope_key is not subject_key, the
one place decision 4 as phrased could not ship, the projection's rule and what it
refuses to guess, the digest correcting Phase 4a rather than only implementing
§4.2b, and the three defects the build found. §4.2b and §6.0b updated with it.

TEAMS.md §6.4 rewritten: the pipeline it described no longer exists as its own
thing. It now says which of the three sinks moved and which did not, where each
of the four properties it always claimed lives now, that Team email is OFF until
an operator turns a rule on, and what the generalized unsubscribe token does.
§6.3's `last_digest_at` is marked as no longer read.

BACKEND_DESIGN.md: engagement_digest_state, engagement_outbox.scope_key, and the
unsubscribe routes — the canonical /public/engagement pair plus the /public/teams
path kept permanently because mail is not editable once sent.

Code: RunicGateway/website#TBD

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
claude
2026-08-29 20:12:15 -05:00
parent 2161119c8c
commit c93151b352
3 changed files with 2047 additions and 1691 deletions
Download Patch File Download Diff File Expand all files Collapse all files

267
website/ENGAGEMENT.md
View File

@@ -593,7 +593,11 @@ Confirmed, and there is nothing to fix — only something to not break:
- `pushDispatch.allowedOrigins()` reads `NTFY_ALLOWED_ORIGINS` / `NTFY_BASE_URL` and **returns empty when
neither is set**. No Runic Gateway host appears anywhere in it.
- `mailer.isConfigured()` gates every sink, and `teamNotify.emailImmediate` checks it *before* the
recipient query so an unconfigured deployment pays nothing.
recipient query so an unconfigured deployment pays nothing. **As of Phase 6 that sink is the engine's**
and the gate moved with it: `mailer.sendNotification` returns a *retryable* failure when mail is
unconfigured, so an operator midway through typing SMTP credentials finds the outbox drains rather
than a backlog the worker gave up on. The digest worker still checks `isConfigured()` up front, before
any query that costs anything.
**Rules to carry into the abstraction:**
@@ -725,10 +729,20 @@ CREATE TABLE IF NOT EXISTS engagement_digest_state (
```
`team_notification_prefs.last_digest_at` backfills into this with `channel='email'`,
`scope_key = team_id`. The three properties from `teamDigestWorker`'s header comment must be preserved
verbatim by the generic worker, and the third one (a user who lost access is no longer in the recipient
set) should get its own named test, the way the Teams phase-5 work gave the leader-can't-see-reports rule
its own test.
`scope_key = CONCAT('team:', team_id)`. The three properties from `teamDigestWorker`'s header comment must
be preserved verbatim by the generic worker, and the third one (a user who lost access is no longer in the
recipient set) should get its own named test, the way the Teams phase-5 work gave the
leader-can't-see-reports rule its own test.
**Phase 6 as built, and it corrects §4.2a rather than only implementing this.** Keeping compute-at-send-time
means a digest-mode recipient must get **no outbox row at all**: Phase 4a's `subscribedTo` enqueued them
("what changes in Phase 6 is who drains it") and what changed in Phase 6 is that nothing drains it. An
outbox row holds a payload snapshotted at emit time and therefore has none of the three properties above —
including the security one. The engine now enqueues `instant` only. Two more things landed with it:
`engagement_outbox` gained a **`scope_key`** column (what a preference and an unsubscribe are keyed on,
which is not `subject_key` — see Phase 6's as-built), and the backfill's replay-safety is a property of the
PRIMARY KEY rather than of a flag, so a window the worker has since moved forward is never dragged
backwards by a restart.
### 4.3 The template variable contract — a code-declared schema, mirrored to a checked-in manifest
@@ -1008,7 +1022,7 @@ UPDATE` keyed on `seed_key`, **and it refuses to overwrite a row whose `customiz
| --- | --- | --- |
| `notify.event` | the generic single-event mail | `title`, `intro`, `items[]`, `actionUrl`, `unsubscribeUrl` |
| `notify.digest` | `teamDigestWorker`'s body | `intro`, `periodLabel`, `items[]`, `moreCount`, `scopeUrl`, `unsubscribeUrl` |
| `notify.team-post` | `mailer.sendTeamNotification` immediate | `teamName`, `authorName`, `threadTitle`, `excerpt`, `threadUrl` |
| `notify.team-post` | `mailer.sendTeamNotification` immediate | `teamName`, `authorName`, `threadTitle`, `excerpt`, `postUrl` — **`postUrl`, not `threadUrl`**: it has to be the name `team.forum.post` DECLARES, or the mail links nowhere. Renamed in Phase 6 (`seedVersion` 2) |
| `inapp.event` | *(new)* — the in-app channel's short form | `title`, `body`, `url` |
**Three properties of the seeded set that are design, not packaging:**
@@ -1319,7 +1333,7 @@ change is not complete until `docs/` reflects it" — is the floor; this table i
| **4b** Rules screen ✅ | `website/BACKEND_DESIGN.md` route table (the twelve routes, incl. the `PATCH …/enabled` argument and the count-only preview) · `website/ENGAGEMENT.md` §5.1a composition UI | Landed with the phase (docs#184) |
| **5a** Templates ✅ | `website/ENGAGEMENT.md` §4.6 as built · `BACKEND_DESIGN.md` — the `engagement_templates` table, the two block registries, the token grammar, and §7's multipart/subject changes | Landed with the phase |
| **5b** The editor | `website/ENGAGEMENT.md` §4.6.2 as built · `BACKEND_DESIGN.md` route table | **`runicgateway.com`**: a new admin docs page for the template editor |
| **6** Email channel + Teams migration | `website/TEAMS.md` §6.3/§6.4 **rewritten** — the Team pipeline it describes no longer exists as its own thing | **`runicgateway.com`**: `administration/teams.mdx` notification section |
| **6** Email channel + Teams migration ✅ | `website/TEAMS.md` §6.3/§6.4 **rewritten** — the Team pipeline it describes no longer exists as its own thing · `website/ENGAGEMENT.md` §4.2b + this phase as built · `BACKEND_DESIGN.md` route table and table inventory | **`runicgateway.com`**: `administration/teams.mdx` notification section. Landed with the phase |
| **7** In-app channel (core+web) | `website/BACKEND_DESIGN.md` routes + tables · `website/ENGAGEMENT.md` | **`runicgateway.com`**: `notifications-and-email.mdx` gains the in-app channel |
| **8** In-app (Android) | `android/PLAN.md` | `android-app/README.md` |
| **9** Deliverability | `website/BACKEND_DESIGN.md` §7 · a suppression/bounce operator section (the verification flow is Phase 1b's) | **`runicgateway.com`**: `troubleshooting.mdx` gains bounce/suppression · **`PLAY_DATA_SAFETY.md` + `/privacy`** — see Phase 12 |
@@ -2335,7 +2349,7 @@ check, because the way to make it pass is to delete the explanation.
---
### Phase 6 — The email channel on the engine, and the Teams migration
### Phase 6 — The email channel on the engine, and the Teams migration ✅
Email becomes a `DeliveryChannel` driven by rules. `teamNotify.js` and `teamDigestWorker.js` are rewritten
onto the generic pipeline; `engagement_digest_state` backfills from `team_notification_prefs.last_digest_at`;
@@ -2355,6 +2369,243 @@ replay-safe.
---
#### As built — 6 (2026-08-29)
The email channel gained a `deliver`, and the Team pipeline stopped being its own thing. Concretely:
`teamNotify.emailImmediate` and `mailer.sendTeamNotification` are gone, `teamNotify.forumPost` emits an
event instead, and a rule decides who is mailed. One end-to-end walk on a live rig now goes
**forum write → `events.emit` → rule → outbox → worker → email channel → template → SMTP → mailbox**,
with the send recorded in `engagement_sends` like everything else the platform sends.
**Seven decisions the org lead settled before any code**, because the tree contradicted the phase body
in every one of them:
| | Question | Decision |
|---|---|---|
| Sinks | `teamNotify` has three sinks; which move onto the engine | **email only.** The tickle and the Discord bridge stay direct calls in `teamNotify.js` |
| Audience | how a Team event reaches that Team's members, when `members` resolves to nobody and a segment takes constant params | **the event carries the set** — the same access-checked list `teamNotify.recipientIds` has always computed |
| Continuity | rules default off and core seeds none, so a straight migration stops Team email silently | **seed the four rules DISABLED**, plus an admin banner and a release note. The invariant is honoured, not excepted |
| Prefs | per-Team granularity has nowhere to live in `notification_channel_prefs` | **keep `team_notification_prefs`**, consulted by the engine as a scoped preference |
| Templates | §4.6.1 property 1 promises a generic template renders any trigger, and nothing implemented it | **payload wins, a structural projection fills gaps** |
| Digest | §4.2b says compute at send time; the engine snapshots at emit time | **keep compute-at-send-time**; generalize only the state |
| Unsubscribe | a v1 token set `muted`, silencing push as well as email | **turn off the channel the token names, and nothing else** |
##### The audience problem, which is the one that shaped the phase
The engine could not express "the members of the Team this post was in", and the reason is structural
rather than an oversight. A rule names its audience two ways: a plain ceiling name resolved from core's
own tables, or a **saved segment** composing module-declared audiences with **constant** parameters. A
Team forum post needs neither — the recipient list is different for every firing, and it is the answer
to an access question (`teamAccess.forumAccess`'s two tables) that core already knows how to ask and a
segment has no way to ask at all. `audiences.js` said as much in its own comment: *"core knows no game
vocabulary and cannot guess which members were meant."*
So the **event** names it. `engagementEmit`'s envelope gained `recipientUserIds`, and a rule whose
audience is `members` resolves to exactly that set. Three properties make it a narrowing input rather
than a hole in the ceiling lattice, and all three are tested:
- the carried set is still filtered through `users.status = 'active'`, so a banned account is not
mailable by an emitter that forgot;
- the ceiling returned is still `members`, so **G24 still runs** — a rule cannot be given
`authenticated` on a trigger that ceilings at `members`, carried set or no carried set;
- the list is bounded at `MAX_AUDIENCE` (5000) *at the emit boundary*, so an emitter cannot assert an
audience larger than the engine would have loaded from a query.
**A rule with any other audience ignores it entirely.** This is not "the emitter decides who gets
mailed": it is the emitter answering the one question core cannot, and the rule deciding everything
else.
##### `scope_key` is not `subject_key`, and Phase 6 is where that stopped being theoretical
Both are per-event strings on the outbox row and they are keyed on different things:
- **`subject_key`** is what a **cooldown** counts. It comes from the trigger's declared `subjectKey`,
which for all four Team triggers is `teamName` — a display string, which is fine, because a cooldown
only ever compares it with itself.
- **`scope_key`** is what a **preference** and an **unsubscribe** are keyed on, and it has to be a
stable identifier. `team:12` survives a rename; `The Silver Anvil` does not. Signing an unsubscribe
token over a display name would orphan every link in every mailbox the first time staff renamed a
guild — and those links have no expiry, so "the first time" means "ever".
`engagement_digest_state.scope_key` already used that vocabulary in §4.2b, so the outbox column is the
same vocabulary in the same shape rather than a second one.
##### The scoped preference, and the one place the decision as phrased could not ship
Decision 4 was recorded as *"a suppression below the channel preference"*. Building it showed that
reading is the one that cannot ship, and the reason is worth stating because it is a second instance of
the G22 pattern — a migration that degrades silently:
`notification_channel_prefs` holds a row **only where a user has expressed something**; absence means
the channel's `defaultMode`; and email's is `off`. Nobody has ever expressed a stream-level opinion
about `team.forum.post` — the screen that would let them is Phase 3's and the preference predates it by
a year. So intersecting the two preferences would resolve **every existing Team-email subscriber** to
`off`, and the deploy that migrated the pipeline would be the deploy that silenced it.
What shipped is **replacement**: where a scope has an opinion, that opinion is the preference.
`engagement/scopedPrefs.js` is a small registry keyed on the scope-key prefix; `engagement/coreScopePrefs.js`
registers `team` and maps the two columns:
- `muted` → `off` **on every channel**. That is what the toggle has always meant on the account screen,
and narrowing it to email would be a behaviour change nobody asked for.
- `email_mode` → the engine's vocabulary (`immediate` → `instant`), and **only for the email channel**.
On push or in-app the provider returns no opinion and the stream-level preference decides.
- **absence of a row means `off` for email**, which is why the provider answers for every user in the
set rather than only the rows it finds. The column defaults to `'off'` and both recipient queries
COALESCE to it: no row has always meant "has not asked for Team email". Deferring to the stream-level
preference instead would mean somebody who once switched on `team.forum.post` email in the channels
screen starts receiving mail from **every Team on the deployment** — a widening produced by a
migration, of a preference expressed about something else.
The cost, stated so nobody rediscovers it: a user cannot turn Team email off for every Team at once
from the channels screen. That control lives on the per-Team screen, which is where it has always lived
and where the unsubscribe link points.
The provider **fails open** — a lookup that throws leaves the stream-level preference in charge, which
for every core channel is `off`. So a failure means nothing is sent rather than that everybody is
mailed, and it does not drop an unrelated IDOC warning because a Team preference query timed out.
##### The projection: what §4.6.1 property 1 actually required
Property 1 says a new trigger renders through `notify.event` **with no authoring at all**. Nothing
implemented it, and building the channel is what made the hole visible: trigger payloads are
domain-named (`teamName`, `threadTitle`, `postUrl`) and the generic seeds are structural (`title`,
`intro`, `items`, `actionUrl`). The two vocabularies never met.
`engagement/projection.js` is the meeting, and its rule is one line: **a name the payload already
carries is left exactly as emitted; only a name it does not carry is supplied.** `news.post` and
`team.announcement` both declare their own `title`, and a projection that overwrote it would replace a
real headline with a category label — on the one variable every generic template puts in the subject.
What it fills comes from the **declaration**, never from a table of domain synonyms:
| | Filled from | Why not something cleverer |
|---|---|---|
| `title` | the trigger's `label` | a mapping of `threadTitle` → `title` is one game's vocabulary compiled into core, and it is wrong on the first module that names the same thing differently |
| `intro` | the trigger's `description` | same |
| `actionUrl` | the first declared `url`-typed variable holding a value | it is the only structural fact available: the declaration says which variables are links |
| `items` | `[]` | set rather than left absent, so a generic mail does not report `items` as a *missing variable* in the editor's preview |
The consequence, seen on the live rig and accepted: an unauthored `team.forum.post` mail is titled
"Team — new forum post" rather than the thread's title. That is a **plain** mail, not a wrong one, and
the operator's answer is the bespoke template shipped beside it. A projection clever enough to do
better is a projection that is confidently wrong on the first module that does not follow core's naming.
##### The digest: the state generalizes, the design does not
§4.2b's instruction was to keep `teamDigestWorker`'s compute-at-send-time design and generalize its
state. Doing that meant **correcting the engine**, not just the worker: Phase 4a's `subscribedTo`
enqueued `digest`-mode recipients with a comment reading *"what changes in Phase 6 is who drains it"*,
and what changed in Phase 6 is that **nothing drains it**. A digest is re-derived from the source tables
when it goes out; an outbox row carries a snapshot taken at emit time, and a snapshot has none of the
three properties the design exists for. So the engine now enqueues **`instant` only**, and each of the
three properties has its own named test:
1. a two-day outage sends ONE digest, not two days of replay;
2. a post a moderator hid after it was written is not in the query, so it is not in the mail;
3. **a user who lost forum access between the post and the send is not mailed** — the security one.
`engagement_digest_state` replaces `team_notification_prefs.last_digest_at`, keyed
`(user_id, channel, scope_key)` so a second digest — on another channel, or over another scope — needs
no second column on somebody's preferences row. The backfill is an `INSERT IGNORE … SELECT` in
`schema.sql`, and it is replay-safe **by construction rather than by a flag**: the primary key rejects
the second run, so a window the new worker has since moved forward is never dragged backwards by a
restart. Verified on the rig by restarting the server after a digest had gone out — the stamped row kept
its stamp, and the unstamped row was added.
**The digest is gated on an enabled email rule**, which is the part that was not in the phase body.
Without it, disabling the rule would stop the instant mail and leave a daily summary arriving
indefinitely, which reads to an operator as the switch being broken.
##### The unsubscribe: a narrowing, and a path that can never move
The token is now `(userId, channel, scopeKey)`, signed as `2.<uid>.<channel>.<scope>.<mac>`. **v1
tokens still verify, permanently**, and read as `{ channel: 'email', scopeKey: 'team:<id>' }` — which is
a reading of what they always meant, since a v1 token could only ever have arrived in an email.
Two things about it are deliberate and easy to undo by accident:
- **The route did not move, even though the phase body said the unsubscribe routes would.** The
canonical pair is now `/api/v1/public/engagement/unsubscribe/:token`, and
`/api/v1/public/teams/unsubscribe/:token` **stays forever**, handing straight to the same handlers.
Mail sent before this phase carries the old path in its `List-Unsubscribe` header and in its body;
mail is not editable once sent, so a route that moves is a person who cannot unsubscribe. Both paths
are in the route manifest.
- **A v1 token now turns off email and no longer mutes push.** That is the live behaviour change
decision 7 accepted: a link labelled "stop these emails" was quietly stopping notifications on
somebody's phone. Verified on the rig — POSTing a v1 token to the old path set `email_mode = 'off'`
and left `muted = 0`.
A channel id may legally contain a dot (`discord.dm` is §3.1's own example) and the token's separator is
a dot, so `sign` **refuses** such a channel rather than producing a token that verifies as a different
one. A scope the format cannot carry costs the mail its unsubscribe link, not the mail.
##### Two defects the phase found in code it did not write
1. **`email.button` never absolutized its href.** `email.image` and `email.itemList` both call
`ctx.absolute`; the button called `ctx.safeHref` alone. It had never mattered, because every caller
before this phase passed an absolute URL. A trigger's `url` variables are validated **site-relative
by construction** (`engagementEmit.RELATIVE_URL` exists so a variable cannot carry a recipient
off-site), so every rule-driven call-to-action would have interpolated to `/guilds/x` — a path a mail
client has no origin to resolve, i.e. a dead link in every notification the engine sends. Fixed in
both parts, with a test either side of the relative/absolute split.
2. **The digest's send-log row had no `address_hash`.** The instant path wrote one and the digest path
did not, so half the deployment's mail would have been uncorrelatable when Phase 9's bounce handling
arrives. Found by reading `engagement_sends` on the live rig, where the two rows sat next to each
other. Both paths now hash through the same function, lower-cased and trimmed — a bounce reported for
`Darrow@` has to match the row written for `darrow@`.
##### A naming inconsistency left alone, on purpose
`team.forum.post` declares its title as `threadTitle` and `team.announcement` declares it as `title`,
though both describe a thread in a Team forum. A template can only name one of them, so
`notify.team-post`'s `{{threadTitle}}` renders empty for an announcement — which is why the seeded
announcement rule points at `notify.event` instead, where the projection gets it right.
Reconciling the two declarations is a **variable rename, which §4.3 makes a version bump**, and this
phase did not take that on its own authority. A test pins the current shape so that reconciling it is a
deliberate act rather than a silent rename that empties somebody's subject line.
##### What an operator sees at cutover, which is the whole of decision 3
Core seeds four rules — one per Team trigger — all `enabled = 0`, all `audience: 'members'`, all on the
email channel. **Team email is off until an operator switches one on.** Three things make that
survivable rather than a silent regression:
- **Admin → Engagement → Rules carries a banner** whenever every Team rule is off, saying that these
used to send automatically, that they arrived switched off so nothing starts mailing on its own, and
that per-member preferences and unsubscribe links still work above them. It reads the rules rather
than a flag, so it disappears the moment one is enabled and returns if they are all switched off
again. A deployment that deleted them sees nothing, which is right — they made that choice.
- **The release note names it**, alongside the v1-token narrowing.
- **The push tickle and the Discord bridge are unaffected.** Only email moved, so the app and the
Discord channel keep working exactly as before while the rules are off.
The rules are seeded **once**, guarded by a settings key rather than ensured on every boot: an operator
who deletes a rule must not find it back after a restart, and one they enabled must not be reset to
off. The guard is stamped even after a partial run — re-running would duplicate the rules that did
insert, and a duplicate rule is two mails per event, which is worse than one missing rule an operator
can add from the screen.
##### What was verified
- **97 tests**: 35 in `teamNotifyDispatch.test.js` (rewritten — the mail-body assertions moved down to
the channel and what is asserted here is now the envelope), 31 in the new `engagementEmail.test.js`,
23 in `teamNotify.test.js`, plus the mailer, engine and block-renderer additions. Server suite **1447**
green, client **324** green.
- **A live rig**, which is where three of the findings above came from: MariaDB + Mailpit + a booted
server + a real Team with three members. The walk covered an instant mail with both
`List-Unsubscribe` headers, a digest gathering two posts and stamping `engagement_digest_state`, the
generic `notify.event` path rendering an announcement with no authoring, a pre-migration v1 token
unsubscribing through the old path, and a restart proving the backfill replay-safe.
- **The route manifest and swagger** carry the two new routes; the two legacy ones are unchanged.
**Still later phases':** the push and in-app channels' `deliver` (Phase 7), whether an unverified
address may receive opt-in mail (§7.1 Q1's narrower half, Phase 9), and the `address_hash` this phase
started writing, which is the column Phase 9's bounce correlation reads.
---
### Phase 7 — The in-app channel (core + web)
`user_notifications`, the in-app `DeliveryChannel`, `GET /auth/me/notifications` + mark-read, and the web