RunicGateway/website
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity

feat(teams): email as the third sink, with a digest that keeps no queue

Browse Source 
A web-only user on a deployment running neither the Android app nor Discord gets
no notification that someone replied to their own thread — which is most users on
most deployments, and a forum where replies are invisible is a forum nobody
returns to. Email is a third consumer of the recipient set the previous commit
builds, not a fourth pipeline.

Unlike a push tickle, an email carries content: a mailbox is a destination the
recipient chose, not an untrusted relay reached by an unguessable topic. It
carries a title and an excerpt, never a full post.

The digest COMPUTES AT SEND TIME and keeps no pending-items queue. The only state
is `last_digest_at`. Three properties fall out, and the third is why it was chosen:
a deployment down for two days sends one correct digest rather than replaying a
backlog; a post a moderator hid after it was written is simply not in the query;
and a user who lost forum access between the post and the send is no longer in
the recipient set, so they are not emailed content they can no longer read.

`last_digest_at` is stamped only on a SUCCESSFUL send — stamping first would
quietly eat a day of somebody's notifications every time the mail provider had a
bad minute.

One-click unsubscribe is a stateless HMAC rather than a token table. Every
property that makes a password-reset token a row is absent: the link sits in a
mailbox for months so it has no useful expiry, and clicking it twice must mean
what clicking it once meant. Its whole capability is setting `muted` for one
(user, Team) pair.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
claude
2026-08-18 14:34:38 -05:00
parent 26c23bd603
commit 686a214979
5 changed files with 555 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

79
server/src/utils/mailer.js
View File

@@ -187,4 +187,81 @@ async function sendPasswordReset({ to, resetUrl, username }) {
}
}
module.exports = { isConfigured, sendContactMessage, sendTest, sendInvite, sendPasswordReset }
/**
* Send a Team notification — one event (`immediate` mode) or a day's worth
* (`digest` mode). TEAMS.md §6.4.
*
* **This one carries CONTENT, and the push tickle beside it deliberately does
* not.** A tickle goes to ntfy, an untrusted relay reachable by an unguessable
* topic, so it carries `{ stream, ref }` and the app pulls the real thing over an
* access-checked API. A mailbox is a destination the recipient chose. Same
* reasoning as the Discord bridge (§7.2), and it is why this function takes
* excerpts rather than ids.
*
* **Excerpts, never full posts.** Partly courtesy, mostly so that the blast radius
* of a mis-addressed or forwarded mail is a sentence rather than a thread. The
* caller does the truncation, because it is the caller that knows the body was
* already stripped of markup.
*
* The `List-Unsubscribe` pair is what makes a mail client's own unsubscribe button
* appear, and both halves are needed: the `mailto:`-free URL form for clients that
* open the link, and `List-Unsubscribe-Post` for RFC 8058 one-click, which POSTs
* without ever showing the user a page. Both reach the same tokened endpoint that
* writes the same per-Team mute the site shows.
*
* Never throws. A notification failing must not fail the forum write that caused
* it, and there is nobody up the stack to catch it — the digest worker runs on a
* timer and the immediate send is fired from a request that has already replied.
*/
async function sendTeamNotification({ to, subject, intro, items, teamUrl, unsubscribeUrl, unsubscribeApiUrl }) {
const built = await buildTransport()
if (!built) return { sent: false, reason: 'NOT_CONFIGURED' }
const { transport, config } = built
const lines = [intro, '']
for (const item of items || []) {
lines.push(`${item.heading}`)
if (item.excerpt) lines.push(` ${item.excerpt}`)
if (item.url) lines.push(` ${item.url}`)
lines.push('')
}
if (teamUrl) lines.push(teamUrl, '')
if (unsubscribeUrl) {
lines.push('To stop these emails for this team, use this link:', unsubscribeUrl)
}
try {
await transport.sendMail({
from: fromHeader(config),
to,
subject,
text: lines.join('\n'),
// The header carries the API url, not the one in the body: a one-click
// client POSTs to whatever is here without rendering anything, so it has to
// be an endpoint. Falls back to the body's url when no API one was passed.
headers: (unsubscribeApiUrl || unsubscribeUrl)
? {
'List-Unsubscribe': `<${unsubscribeApiUrl || unsubscribeUrl}>`,
'List-Unsubscribe-Post': 'List-Unsubscribe=One-Click',
}
: undefined,
})
return { sent: true }
} catch (err) {
// Logged and swallowed, unlike every other sender in this file. Those are
// called by a request that can report the failure to whoever caused it; this
// one is not, and recordStatus already puts the error where an admin reads it.
log.warn('team notification send failed', { message: err.message })
await emailConfig.recordStatus({ status: 'error', statusDetail: err.message }).catch(() => {})
return { sent: false, reason: 'SEND_FAILED' }
}
}
module.exports = {
isConfigured,
sendContactMessage,
sendTest,
sendInvite,
sendPasswordReset,
sendTeamNotification,
}

158
server/src/utils/teamDigestWorker.js Normal file
View File

@@ -0,0 +1,158 @@
// ── Team forum digest worker (TEAMS.md §6.4, phase 6) ──────────────────────
//
// Daily, per (user, Team): "here is what you missed". The same in-process shape as
// utils/teamActivityPrune and utils/announceWorker — setInterval + unref + stop(),
// wired into server.js start/shutdown. There is no cron in this stack.
//
// **It computes at send time and keeps no queue.** The only state is
// `team_notification_prefs.last_digest_at`; everything else is re-derived from the
// forum tables when the mail is about to go out. Three properties fall out of that,
// and they are why the design chose it over a pending-items table:
//
// 1. A deployment that was down for two days sends ONE correct digest, not two
// days of replay.
// 2. A post a moderator hid after it was written is not in the query, so it is
// not in the mail. A queue written at publish time would have to remember to
// go back and remove it.
// 3. A user who lost forum access between the post and the send is no longer in
// the recipient set, so they are not emailed content they can no longer read.
// This is the one that would have been a security bug.
//
// **The first run is delayed, for the same reason the prune's is**: a boot that is
// crash-looping must not send mail on every loop.
const teamNotify = require('../model/teams/teamNotify.model')
const forumSettings = require('../model/teams/teamForumSettings.model')
const mailer = require('./mailer')
const notify = require('./teamNotify')
const brand = require('../config/brand')
const log = require('./logger')('team-digest')
const INTERVAL_MS = Number(process.env.TEAM_DIGEST_INTERVAL_MS) || 24 * 60 * 60 * 1000
const FIRST_RUN_MS = Number(process.env.TEAM_DIGEST_DELAY_MS) || 10 * 60 * 1000
// How far back a recipient with no `last_digest_at` reaches. A first digest must
// not be the entire history of the forum, and this is also the clamp that stops a
// long outage producing one enormous mail — `since` is never older than this,
// however long ago the last send was.
const MAX_LOOKBACK_MS = 7 * 24 * 60 * 60 * 1000
// Posts per digest. Beyond this the mail is a list, not a summary, and the link
// to the Team is the better answer.
const MAX_ITEMS = 20
let timer = null
let firstRun = null
const clampSince = (last, now) => {
const floor = new Date(now.getTime() - MAX_LOOKBACK_MS)
// No previous send: reach back one interval, not to the floor. A brand-new
// subscriber's first digest should cover today, not the past week.
if (!last) return new Date(now.getTime() - INTERVAL_MS)
const at = new Date(last)
return at < floor ? floor : at
}
/**
* One recipient's digest for one Team. Returns true if a mail went out.
*
* Stamps `last_digest_at` ONLY on a successful send. A failed SMTP call leaves the
* stamp alone so the next run tries the same window again — the alternative,
* stamping first, silently eats a day of somebody's notifications every time the
* mail provider has a bad minute.
*/
async function sendOne(team, recipient, now) {
const since = clampSince(recipient.last_digest_at, now)
const posts = await teamNotify.digestPostsSince(team.id, since, MAX_ITEMS)
// Nothing new for THIS recipient — which is not the same as nothing new for the
// Team, because each recipient has their own `since`. No mail, and no stamp: a
// stamp here would move the window past posts they have not been told about.
if (posts.length === 0) return false
const label = notify.teamLabel(team)
const res = await mailer.sendTeamNotification({
to: recipient.email,
subject: `[${brand.name}] ${label}: ${posts.length} new post${posts.length === 1 ? '' : 's'}`,
intro: `Since your last digest, ${posts.length} new post${posts.length === 1 ? '' : 's'} in ${label}:`,
items: posts.map((p) => ({
heading: `${p.title} — ${p.author_username || 'someone'}`,
excerpt: notify.excerpt(p.body_html),
url: notify.threadUrl(team, p.thread_id),
})),
teamUrl: notify.teamPageUrl(team),
unsubscribeUrl: notify.unsubscribeUrl(recipient.user_id, team.id),
unsubscribeApiUrl: notify.unsubscribeApiUrl(recipient.user_id, team.id),
})
if (!res || !res.sent) return false
await teamNotify.stampDigest(recipient.user_id, team.id, now)
return true
}
/**
* One sweep. Never throws — it runs on a timer with nobody to catch it.
*
* Returns a small summary so a test (and the log line) can tell "nothing to do"
* from "did nothing".
*/
async function tick(now = new Date()) {
const summary = { teams: 0, sent: 0, skipped: null }
try {
// Two cheap gates before any query that costs anything. Forums switched off
// means the content this digest summarises is not readable on the site
// either, and un-configured email means there is no sink at all (§6.4).
if (!(await forumSettings.forumsEnabled())) return { ...summary, skipped: 'forums-disabled' }
if (!(await mailer.isConfigured())) return { ...summary, skipped: 'email-unconfigured' }
const floor = new Date(now.getTime() - MAX_LOOKBACK_MS)
const teams = await teamNotify.teamsWithForumActivitySince(floor)
summary.teams = teams.length
for (const team of teams) {
// eslint-disable-next-line no-await-in-loop
const rows = await teamNotify.emailRecipients(team.id)
for (const r of rows.filter((x) => x.email_mode === 'digest')) {
try {
// Serial, like the immediate sender and for the same reason: one SMTP
// conversation at a time against a provider with its own rate limits.
// eslint-disable-next-line no-await-in-loop
if (await sendOne(team, r, now)) summary.sent += 1
} catch (err) {
// One recipient's failure must not end the sweep for the rest. The
// unstamped preference means the next run retries this one.
log.warn('digest send failed', { teamId: team.id, userId: r.user_id, message: err.message })
}
}
}
if (summary.sent > 0) log.info('team digests sent', summary)
return summary
} catch (err) {
log.error('team digest sweep failed', { message: err.message })
return { ...summary, skipped: 'error' }
}
}
function start() {
if (timer || firstRun) return timer
firstRun = setTimeout(() => {
firstRun = null
tick()
timer = setInterval(() => { tick() }, INTERVAL_MS)
if (timer.unref) timer.unref()
}, FIRST_RUN_MS)
if (firstRun.unref) firstRun.unref()
log.info('team forum digests started', { intervalMs: INTERVAL_MS, firstRunMs: FIRST_RUN_MS })
return timer
}
function stop() {
if (firstRun) {
clearTimeout(firstRun)
firstRun = null
}
if (timer) {
clearInterval(timer)
timer = null
}
}
module.exports = { start, stop, tick, clampSince, MAX_LOOKBACK_MS, MAX_ITEMS }

225
server/src/utils/teamNotify.js Normal file
View File

@@ -0,0 +1,225 @@
// ── Team notification fan-out (TEAMS.md Part 6, phase 6) ───────────────────
//
// One event in, up to two sinks out: a content-free push tickle and — for forum
// content only — an email. The expensive part of a notification is working out
// who should get it, and that is computed once here and handed to both.
//
// **Nothing in this file ever throws.** Every entry point is called from a path
// that has already done the real work: a forum reply is written and answered
// before this runs, and the roster sync's whole job is the roster. A notification
// is a courtesy, and a courtesy that can fail the transaction behind it is a
// defect. So every export catches, logs and returns.
//
// **Push and email do not carry the same thing, on purpose.** The tickle is
// `{ stream, ref }` and goes to ntfy, an untrusted relay reached by an unguessable
// topic; the app wakes and PULLS the real content over the authenticated,
// access-checked API. The email carries a title and an excerpt, because a mailbox
// is a destination the recipient chose rather than a relay (§6.4). The asymmetry
// is the security model, not an inconsistency to tidy up.
//
// **Roster events are push-only, and forum events are the only ones that email.**
// §6.4's argument for the email sink is the web-only user who never learns that
// someone replied to their own thread. "Someone joined the guild" is not that: it
// arrives from a sweep that runs every fifteen minutes, it is already on the
// activity feed, and mailing it is how a notification feature earns a spam
// complaint. The streams exist for all four events; the SINKS differ, and this is
// the file that says so.
const pushDispatch = require('./pushDispatch')
const teamNotify = require('../model/teams/teamNotify.model')
const forumSettings = require('../model/teams/teamForumSettings.model')
const mailer = require('./mailer')
const registries = require('../modules/registries')
const unsubscribeToken = require('./unsubscribeToken')
const brand = require('../config/brand')
const log = require('./logger')('team-notify')
const STREAMS = {
MEMBER_JOINED: 'team.member.joined',
LEADERSHIP_CHANGED: 'team.leadership.changed',
FORUM_POST: 'team.forum.post',
ANNOUNCEMENT: 'team.announcement',
}
// How much of a post body an email carries. Long enough to tell whether the
// thread is worth opening, short enough that the mail is not a copy of the forum.
const EXCERPT_CHARS = 200
const baseUrl = () => (process.env.APP_BASE_URL || 'http://localhost:5173').replace(/\/+$/, '')
/**
* Where this Team's page lives, or null.
*
* Core does not own a Team page — the module that owns the vocabulary owns the
* page (Part 3) — so the only way core can write a link to one is the optional
* `pageUrlTemplate` the provider registers. A deployment whose module omits it
* gets email that names the Team and cannot link to it, which is a worse email
* and not a broken one.
*/
function teamPageUrl(team) {
const provider = registries.registeredTeamProvider()
const template = provider && provider.pageUrlTemplate
if (!template || !team) return null
const path = template
.replace('{externalId}', encodeURIComponent(team.external_id ?? team.externalId ?? ''))
.replace('{slug}', encodeURIComponent(team.slug ?? ''))
return `${baseUrl()}${path}`
}
const threadUrl = (team, threadId) => {
const page = teamPageUrl(team)
// The forum navigates by SEARCH PARAM rather than by a route, because core has
// no route on a page it does not own (TeamForumPanel.jsx). So a deep link to a
// thread is the module's page plus `?thread=`, and it works under whatever path
// the module chose.
return page ? `${page}?thread=${Number(threadId)}` : null
}
// TWO urls from one token, and they are not interchangeable.
//
// `unsubscribeUrl` is the human one that goes in the mail body: the site's own
// page, which explains what is about to happen and POSTs once a person has read
// it. `unsubscribeApiUrl` is the machine one that goes in the `List-Unsubscribe`
// header, where RFC 8058 says a client may POST without showing anybody anything —
// so it has to be an endpoint, not a page. The API route answers GET on the same
// path with a redirect to the page, which covers the clients that render the
// header as an ordinary link.
const unsubscribeUrl = (userId, teamId) =>
`${baseUrl()}/unsubscribe/${unsubscribeToken.sign(userId, teamId)}`
const unsubscribeApiUrl = (userId, teamId) =>
`${baseUrl()}/api/v1/public/teams/unsubscribe/${unsubscribeToken.sign(userId, teamId)}`
const teamLabel = (team) => (team && (team.display_name_override || team.name)) || 'your team'
/** Markup out, whitespace collapsed, truncated. The email is plain text. */
function excerpt(html) {
const text = String(html || '')
.replace(/<[^>]*>/g, ' ')
.replace(/&nbsp;/g, ' ')
.replace(/&amp;/g, '&')
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&quot;/g, '"')
.replace(/\s+/g, ' ')
.trim()
return text.length > EXCERPT_CHARS ? `${text.slice(0, EXCERPT_CHARS - 1)}…` : text
}
/** The push half. Resolves recipients, honours mutes and subscriptions, never throws. */
async function tickle(streamId, team, { ref, exclude = [] } = {}) {
const userIds = await teamNotify.recipientIds(team.id, { exclude })
if (userIds.length === 0) return 0
await pushDispatch.publishToUsers(streamId, { ref, userIds })
return userIds.length
}
// ── Roster events (push only, see the header) ──────────────────────────────
// No `memberName` argument, and that is the point: a tickle is content-free, so
// there is nothing about WHO joined for this function to carry. The name is on
// the activity feed the app pulls after waking.
async function memberJoined(team) {
try {
return await tickle(STREAMS.MEMBER_JOINED, team, { ref: `team:${team.id}` })
} catch (err) {
log.warn('member-joined notification failed', { teamId: team && team.id, message: err.message })
return 0
}
}
async function leadershipChanged(team) {
try {
return await tickle(STREAMS.LEADERSHIP_CHANGED, team, { ref: `team:${team.id}` })
} catch (err) {
log.warn('leadership notification failed', { teamId: team && team.id, message: err.message })
return 0
}
}
// ── Forum events (push + immediate email) ──────────────────────────────────
/**
* A new thread or reply.
*
* `type` picks the stream: an announcement is its own stream so a user can take
* the thing a leader wants everyone to read and mute the day-to-day chatter,
* which is the split §6.2 drew and the reason there are four streams and not two.
*
* The author is excluded from both sinks. Not as a nicety — a forum that emails
* you your own post is the first thing anyone turns off, and turning it off costs
* the deployment every other notification too.
*/
async function forumPost({ team, threadId, threadTitle, type, authorUserId, authorName, bodyHtml }) {
try {
// Belt and braces with the routes, which already 404 when forums are off. The
// digest worker has no route in front of it, so the check has to live here as
// well as there — and a switch flipped between a write and its notification
// must silence the notification.
if (!(await forumSettings.forumsEnabled())) return { push: 0, emails: 0 }
const stream = type === 'announcement' ? STREAMS.ANNOUNCEMENT : STREAMS.FORUM_POST
const exclude = authorUserId ? [authorUserId] : []
const push = await tickle(stream, team, { ref: `team:${team.id}:thread:${threadId}`, exclude })
const emails = await emailImmediate({ team, threadId, threadTitle, type, exclude, authorName, bodyHtml })
return { push, emails }
} catch (err) {
log.warn('forum notification failed', { teamId: team && team.id, message: err.message })
return { push: 0, emails: 0 }
}
}
/**
* The `immediate` email mode: one mail per event, to the people who asked for
* exactly that.
*
* Skipped entirely when no email is configured — §6.4's "off unless configured"
* — and checked BEFORE the recipient query so a deployment with no Gmail
* connected pays nothing for the sink it does not have.
*/
async function emailImmediate({ team, threadId, threadTitle, type, exclude, authorName, bodyHtml }) {
if (!(await mailer.isConfigured())) return 0
const rows = await teamNotify.emailRecipients(team.id, { exclude })
const recipients = rows.filter((r) => r.email_mode === 'immediate')
if (recipients.length === 0) return 0
const label = teamLabel(team)
const kind = type === 'announcement' ? 'announcement' : 'post'
const url = threadUrl(team, threadId)
let sent = 0
for (const r of recipients) {
// Serial rather than Promise.all: this is an SMTP conversation per recipient
// against a provider with its own rate limits, and a burst of them from a
// busy thread is how a Gmail sender gets throttled. The loop is also why the
// send below is fire-and-report rather than fire-and-throw.
// eslint-disable-next-line no-await-in-loop
const res = await mailer.sendTeamNotification({
to: r.email,
subject: `[${brand.name}] ${label}: ${threadTitle}`,
intro: `${authorName || 'Someone'} posted a new ${kind} in ${label}.`,
items: [{ heading: threadTitle, excerpt: excerpt(bodyHtml), url }],
teamUrl: teamPageUrl(team),
unsubscribeUrl: unsubscribeUrl(r.user_id, team.id),
unsubscribeApiUrl: unsubscribeApiUrl(r.user_id, team.id),
})
if (res && res.sent) sent += 1
}
return sent
}
module.exports = {
STREAMS,
memberJoined,
leadershipChanged,
forumPost,
// Exported for the digest worker and for the tests, which is the whole reason
// they are not inlined: a URL that only ever appears inside a mail body is a
// URL nothing can assert on.
teamPageUrl,
threadUrl,
unsubscribeUrl,
unsubscribeApiUrl,
excerpt,
teamLabel,
}

91
server/src/utils/unsubscribeToken.js Normal file
View File

@@ -0,0 +1,91 @@
// ── One-click unsubscribe tokens (TEAMS.md §6.4) ───────────────────────────
//
// A stateless HMAC over (userId, teamId, version), not a row in a table.
//
// **Why stateless.** The alternative is a `password_resets`-shaped token table,
// and it is the wrong shape for this: an unsubscribe link sits in a mailbox for
// months and must still work, so it has no useful expiry; it is not single-use,
// because clicking it twice must mean the same thing as clicking it once; and a
// table would need pruning for a capability that never expires. Every property
// that makes a reset token a row is absent here.
//
// **What the capability actually is.** Holding a token lets the holder set
// `muted = 1` for ONE (user, Team) pair. It cannot read anything, cannot unmute,
// cannot touch email mode, and names no other Team. So the honest threat model is:
// someone who intercepts the mail can silence one Team's notifications for that
// account, visibly and reversibly on the account screen. That is a smaller
// capability than the mail itself already carries (it contains the content).
//
// **`v` is the version prefix, and it is what makes rotation possible at all.** A
// stateless token cannot be revoked individually; bumping VERSION invalidates
// every outstanding link at once, which is the only revocation a design with no
// server-side state can offer, and it needs to exist before it is needed.
//
// The key is SECRET_ENC_KEY, derived through the same dev fallback as
// utils/secretBox — a separate label so an unsubscribe token can never be
// confused with, or replayed as, anything else keyed by the same secret.
const crypto = require('crypto')
require('dotenv').config()
const log = require('./logger')('unsub-token')
const VERSION = 1
function resolveKey() {
const explicit = process.env.SECRET_ENC_KEY
if (explicit) return crypto.createHash('sha256').update(`unsubscribe:${explicit}`).digest()
if (process.env.NODE_ENV === 'production') {
throw new Error('SECRET_ENC_KEY must be set in production')
}
const jwt = process.env.JWT_SECRET || 'dev-insecure-jwt-secret-do-not-use-in-production'
log.warn('SECRET_ENC_KEY is not set — deriving an insecure unsubscribe key from JWT_SECRET for development.')
return crypto.createHash('sha256').update(`unsubscribe:${jwt}`).digest()
}
let cachedKey = null
const key = () => {
// Lazily, not at require time. utils/secretBox resolves its key on import and
// that is fine for a module every boot loads anyway; this one is reached from a
// mail template, and a test that never sends mail should not have to set an env
// var to require the module that sends it.
if (!cachedKey) cachedKey = resolveKey()
return cachedKey
}
// base64url so the token survives being a path segment, a query value and a mail
// client's own re-wrapping of a long URL without any of the three escaping it.
const b64u = (buf) => buf.toString('base64').replace(/\+/g, '-').replace(/\//g, '_').replace(/=+$/, '')
function sign(userId, teamId) {
const body = `${VERSION}.${Number(userId)}.${Number(teamId)}`
const mac = crypto.createHmac('sha256', key()).update(body).digest()
// Truncated to 16 bytes (128 bits). Full-length would double the URL for no
// reachable gain: forging this buys one mute, and 128 bits is far past the
// point where that is worth anyone's compute.
return `${body}.${b64u(mac.subarray(0, 16))}`
}
/**
* Verify a token. Returns { userId, teamId } or null — null for every failure
* mode, deliberately, so a caller cannot accidentally report which part was wrong.
*/
function verify(token) {
const parts = String(token || '').split('.')
if (parts.length !== 4) return null
const [v, uid, tid] = parts
if (Number(v) !== VERSION) return null
const userId = Number(uid)
const teamId = Number(tid)
if (!Number.isInteger(userId) || !Number.isInteger(teamId)) return null
const expected = sign(userId, teamId)
const a = Buffer.from(expected)
const b = Buffer.from(String(token))
// Length-check first: timingSafeEqual throws on a length mismatch, and the
// length of a token is not a secret.
if (a.length !== b.length || !crypto.timingSafeEqual(a, b)) return null
return { userId, teamId }
}
module.exports = { sign, verify, VERSION }