feat(engagement): the email channel on the engine, and the Teams migration (engagement Phase 6)
Email becomes a DeliveryChannel driven by rules, and the Team pipeline stops being
its own thing. `teamNotify.forumPost` now emits an event; a rule decides who is
mailed, through which template, and how often at most. One walk goes forum write
-> events.emit -> rule -> outbox -> worker -> email channel -> template -> SMTP.
Seven decisions settled by the org lead before any code:
- email only moves; the push tickle and the Discord bridge stay direct calls
- the EVENT carries its access-checked audience, and `members` resolves to it
- the four Team rules are seeded DISABLED, with an admin banner and a note
- team_notification_prefs stays, read by the engine as a scoped preference
- the payload wins and a structural projection fills the gaps
- the digest keeps computing at send time; only its state generalizes
- an unsubscribe token turns off the channel it names, and nothing else
Three defects found while building it:
- `email.button` never absolutized its href, while image and itemList both
did. Every rule-driven CTA would have been a dead relative link, because a
trigger's url variables are validated site-relative by construction.
- Phase 4a enqueued digest-mode recipients for a drain that Phase 6 decided
not to build. An outbox row snapshots the payload and so has none of the
three properties the digest design exists for, including the security one.
- the digest's send-log row carried no address_hash while the instant row
beside it did, which would have made half the mail uncorrelatable in Phase 9.
Also: engagement_digest_state + a replay-safe backfill, engagement_outbox.scope_key,
a v2 unsubscribe token that still verifies v1 forever, and the canonical
/public/engagement/unsubscribe pair with the old /public/teams path kept
permanently — mail is not editable once sent.
Verified with 1464 server tests, 324 client tests, and a live rig (MariaDB +
Mailpit + a real Team) covering the instant mail, the digest, the generic
template, a pre-migration unsubscribe link and the backfill's replay-safety.
Docs: RunicGateway/docs#TBD
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -1,13 +1,14 @@
|
||||
// ── Team forum digest worker (TEAMS.md §6.4, phase 6) ──────────────────────
|
||||
// ── Team forum digest worker (TEAMS.md §6.4; migrated in ENGAGEMENT.md Phase 6) ──
|
||||
//
|
||||
// Daily, per (user, Team): "here is what you missed". The same in-process shape as
|
||||
// utils/teamActivityPrune and utils/announceWorker — setInterval + unref + stop(),
|
||||
// wired into server.js start/shutdown. There is no cron in this stack.
|
||||
//
|
||||
// **It computes at send time and keeps no queue.** The only state is
|
||||
// `team_notification_prefs.last_digest_at`; everything else is re-derived from the
|
||||
// forum tables when the mail is about to go out. Three properties fall out of that,
|
||||
// and they are why the design chose it over a pending-items table:
|
||||
// **It computes at send time and keeps no queue, and Phase 6 deliberately did not
|
||||
// change that.** §4.2b's decision was to generalize this worker's STATE, not its
|
||||
// absence of one. Three properties fall out of computing at send time, and they
|
||||
// are why the design chose it over a pending-items table — and, now, over the
|
||||
// engine's own outbox:
|
||||
//
|
||||
// 1. A deployment that was down for two days sends ONE correct digest, not two
|
||||
// days of replay.
|
||||
@@ -18,14 +19,29 @@
|
||||
// the recipient set, so they are not emailed content they can no longer read.
|
||||
// This is the one that would have been a security bug.
|
||||
//
|
||||
// An outbox row carries a snapshot of the payload taken at emit time and has none
|
||||
// of the three. That is why `engine.subscribedTo` enqueues only `instant`
|
||||
// recipients and leaves `digest` to this file.
|
||||
//
|
||||
// **What DID change is everything around the query.** The state is
|
||||
// `engagement_digest_state` rather than a column on the preferences row; the body
|
||||
// is the `notify.digest` template an operator can edit rather than a literal in
|
||||
// `mailer`; the unsubscribe link is a v2 token naming the email channel; and the
|
||||
// whole worker is gated on an ENABLED rule, so an operator who switches Team
|
||||
// email off switches off both halves of it rather than the immediate half only.
|
||||
//
|
||||
// **The first run is delayed, for the same reason the prune's is**: a boot that is
|
||||
// crash-looping must not send mail on every loop.
|
||||
|
||||
const teamNotify = require('../model/teams/teamNotify.model')
|
||||
const forumSettings = require('../model/teams/teamForumSettings.model')
|
||||
const rulesDb = require('../model/engagement/engagementRules.db')
|
||||
const sendsDb = require('../model/engagement/engagementSends.db')
|
||||
const digestDb = require('../model/engagement/engagementDigest.db')
|
||||
const templates = require('../engagement/templates')
|
||||
const emailChannel = require('../engagement/emailChannel')
|
||||
const mailer = require('./mailer')
|
||||
const notify = require('./teamNotify')
|
||||
const brand = require('../config/brand')
|
||||
const log = require('./logger')('team-digest')
|
||||
|
||||
const INTERVAL_MS = Number(process.env.TEAM_DIGEST_INTERVAL_MS) || 24 * 60 * 60 * 1000
|
||||
@@ -41,6 +57,16 @@ const MAX_LOOKBACK_MS = 7 * 24 * 60 * 60 * 1000
|
||||
// to the Team is the better answer.
|
||||
const MAX_ITEMS = 20
|
||||
|
||||
// The two triggers a Team forum digest summarises. A rule on either, enabled and
|
||||
// naming the email channel, is what turns this worker on.
|
||||
const DIGEST_TRIGGERS = ['team.forum.post', 'team.announcement']
|
||||
|
||||
// The digest's own template key, and the rule's `template_keys.digest` overrides
|
||||
// it. A digest is not the same message as the instant mail and must not silently
|
||||
// borrow `template_keys.email`: that template is written for one event and would
|
||||
// render a day's worth of posts as a single missing `{{threadTitle}}`.
|
||||
const DEFAULT_TEMPLATE = 'notify.digest'
|
||||
|
||||
let timer = null
|
||||
let firstRun = null
|
||||
|
||||
@@ -53,6 +79,29 @@ const clampSince = (last, now) => {
|
||||
return at < floor ? floor : at
|
||||
}
|
||||
|
||||
/**
|
||||
* The enabled email rule that authorises Team forum digests, or null.
|
||||
*
|
||||
* **The gate is why this worker did not simply keep running.** Phase 6's decision
|
||||
* 3 is that Team notifications become rules an operator turns on; if the immediate
|
||||
* mail were rule-gated and the digest were not, disabling the rule would stop one
|
||||
* kind of Team mail and leave a daily summary arriving indefinitely — which reads
|
||||
* as the switch being broken.
|
||||
*
|
||||
* The FIRST matching rule wins, and the tie-break is not interesting because what
|
||||
* is read off it is one template key. Two rules disagreeing about the template of
|
||||
* a digest neither of them describes is a configuration an operator can see in the
|
||||
* rules list.
|
||||
*/
|
||||
async function digestRule() {
|
||||
for (const triggerId of DIGEST_TRIGGERS) {
|
||||
const rules = await rulesDb.enabledForTrigger(triggerId)
|
||||
const rule = rules.find((r) => (r.channels || []).includes('email'))
|
||||
if (rule) return rule
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* One recipient's digest for one Team. Returns true if a mail went out.
|
||||
*
|
||||
@@ -61,7 +110,8 @@ const clampSince = (last, now) => {
|
||||
* stamping first, silently eats a day of somebody's notifications every time the
|
||||
* mail provider has a bad minute.
|
||||
*/
|
||||
async function sendOne(team, recipient, now) {
|
||||
async function sendOne(team, recipient, rule, now) {
|
||||
const scopeKey = notify.scopeKey(team)
|
||||
const since = clampSince(recipient.last_digest_at, now)
|
||||
const posts = await teamNotify.digestPostsSince(team.id, since, MAX_ITEMS)
|
||||
// Nothing new for THIS recipient — which is not the same as nothing new for the
|
||||
@@ -70,21 +120,54 @@ async function sendOne(team, recipient, now) {
|
||||
if (posts.length === 0) return false
|
||||
|
||||
const label = notify.teamLabel(team)
|
||||
const res = await mailer.sendTeamNotification({
|
||||
to: recipient.email,
|
||||
subject: `[${brand.name}] ${label}: ${posts.length} new post${posts.length === 1 ? '' : 's'}`,
|
||||
const unsub = emailChannel.unsubscribeUrls(recipient.user_id, scopeKey)
|
||||
const key = (rule && rule.template_keys && rule.template_keys.digest) || DEFAULT_TEMPLATE
|
||||
|
||||
const rendered = await templates.renderByKey(key, {
|
||||
periodLabel: `${label}: ${posts.length} new post${posts.length === 1 ? '' : 's'}`,
|
||||
intro: `Since your last digest, ${posts.length} new post${posts.length === 1 ? '' : 's'} in ${label}:`,
|
||||
items: posts.map((p) => ({
|
||||
heading: `${p.title} — ${p.author_username || 'someone'}`,
|
||||
excerpt: notify.excerpt(p.body_html),
|
||||
url: notify.threadUrl(team, p.thread_id),
|
||||
url: notify.threadPath(team, p.thread_id),
|
||||
})),
|
||||
teamUrl: notify.teamPageUrl(team),
|
||||
unsubscribeUrl: notify.unsubscribeUrl(recipient.user_id, team.id),
|
||||
unsubscribeApiUrl: notify.unsubscribeApiUrl(recipient.user_id, team.id),
|
||||
scopeUrl: notify.teamPagePath(team),
|
||||
...unsub,
|
||||
})
|
||||
if (!res || !res.sent) return false
|
||||
await teamNotify.stampDigest(recipient.user_id, team.id, now)
|
||||
if (!rendered) {
|
||||
log.warn('digest template is missing and has no shipped default', { key })
|
||||
return false
|
||||
}
|
||||
|
||||
const result = await mailer.sendNotification({ to: recipient.email, rendered, ...unsub })
|
||||
|
||||
// Recorded in `engagement_sends` like every other message the platform sends,
|
||||
// which is G15's whole point: "did user X get the digest?" was unanswerable
|
||||
// before this phase because the digest went out through a sender that wrote
|
||||
// nothing down. `outbox_id` is null because a digest has no outbox row — see
|
||||
// this file's header — and that null is the honest record of a different path,
|
||||
// not a missing value.
|
||||
await sendsDb
|
||||
.record({
|
||||
outbox_id: null,
|
||||
rule_id: rule ? rule.id : null,
|
||||
trigger_id: DIGEST_TRIGGERS[0],
|
||||
user_id: recipient.user_id,
|
||||
channel: 'email',
|
||||
transport: result.transport ?? null,
|
||||
// The same hash the instant path writes, and it has to be the same
|
||||
// function: a bounce (Phase 9) arrives with an address and is matched
|
||||
// against this column, so a digest row without one is a delivery that
|
||||
// cannot be correlated. Found by reading the send log on the live rig,
|
||||
// where the instant row had a hash and the digest row beside it did not.
|
||||
address_hash: emailChannel.hashAddress(recipient.email),
|
||||
status: result.ok ? 'sent' : 'failed',
|
||||
detail: result.ok ? null : result.detail,
|
||||
})
|
||||
.catch((err) => log.warn('digest send not logged', { message: err.message }))
|
||||
|
||||
if (!result.ok) return false
|
||||
await digestDb.stamp(recipient.user_id, 'email', scopeKey, now)
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -97,11 +180,14 @@ async function sendOne(team, recipient, now) {
|
||||
async function tick(now = new Date()) {
|
||||
const summary = { teams: 0, sent: 0, skipped: null }
|
||||
try {
|
||||
// Two cheap gates before any query that costs anything. Forums switched off
|
||||
// Three cheap gates before any query that costs anything. Forums switched off
|
||||
// means the content this digest summarises is not readable on the site
|
||||
// either, and un-configured email means there is no sink at all (§6.4).
|
||||
// either; un-configured email means there is no sink at all (§6.4); and no
|
||||
// enabled rule means the operator has not turned Team email on.
|
||||
if (!(await forumSettings.forumsEnabled())) return { ...summary, skipped: 'forums-disabled' }
|
||||
if (!(await mailer.isConfigured())) return { ...summary, skipped: 'email-unconfigured' }
|
||||
const rule = await digestRule()
|
||||
if (!rule) return { ...summary, skipped: 'no-enabled-rule' }
|
||||
|
||||
const floor = new Date(now.getTime() - MAX_LOOKBACK_MS)
|
||||
const teams = await teamNotify.teamsWithForumActivitySince(floor)
|
||||
@@ -110,15 +196,27 @@ async function tick(now = new Date()) {
|
||||
for (const team of teams) {
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
const rows = await teamNotify.emailRecipients(team.id)
|
||||
for (const r of rows.filter((x) => x.email_mode === 'digest')) {
|
||||
const wanted = rows.filter((x) => x.email_mode === 'digest')
|
||||
if (!wanted.length) continue
|
||||
// The stamps now live in their own table, so they are read here rather than
|
||||
// arriving on the recipient row. One query per Team, not one per recipient.
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
const stamps = await digestDb.stampsFor(
|
||||
wanted.map((r) => r.user_id),
|
||||
'email',
|
||||
notify.scopeKey(team),
|
||||
)
|
||||
for (const r of wanted) {
|
||||
try {
|
||||
// Serial, like the immediate sender and for the same reason: one SMTP
|
||||
// Serial, like the instant path and for the same reason: one SMTP
|
||||
// conversation at a time against a provider with its own rate limits.
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
if (await sendOne(team, r, now)) summary.sent += 1
|
||||
const recipient = { ...r, last_digest_at: stamps.get(Number(r.user_id)) ?? null }
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
if (await sendOne(team, recipient, rule, now)) summary.sent += 1
|
||||
} catch (err) {
|
||||
// One recipient's failure must not end the sweep for the rest. The
|
||||
// unstamped preference means the next run retries this one.
|
||||
// unstamped window means the next run retries this one.
|
||||
log.warn('digest send failed', { teamId: team.id, userId: r.user_id, message: err.message })
|
||||
}
|
||||
}
|
||||
@@ -155,4 +253,15 @@ function stop() {
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { start, stop, tick, clampSince, MAX_LOOKBACK_MS, MAX_ITEMS }
|
||||
module.exports = {
|
||||
start,
|
||||
stop,
|
||||
tick,
|
||||
sendOne,
|
||||
clampSince,
|
||||
digestRule,
|
||||
MAX_LOOKBACK_MS,
|
||||
MAX_ITEMS,
|
||||
DIGEST_TRIGGERS,
|
||||
DEFAULT_TEMPLATE,
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user