feat(teams): phase 6 — notifications, and the email sink the web never had #156

Merged
whitlocktech merged 6 commits from feature/teams-phase6-notifications into edge 2026-08-18 23:10:22 +00:00
7 changed files with 533 additions and 5 deletions
Showing only changes of commit 26c23bd603 - Show all commits

View File

@@ -1266,6 +1266,45 @@ CREATE TABLE IF NOT EXISTS team_activity (
INDEX idx_team_activity_feed (team_id, occurred_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Per-Team notification preference (TEAMS.md §6.3/§6.4, phase 6). OPT-OUT, not
-- opt-in: a user in a single Team must never have to configure anything, so the
-- absence of a row is the default and every column here is a deviation from it.
--
-- Team scoping lives HERE and in the recipient computation, never in a stream id.
-- The push catalog is a static registration validated at boot against a namespaced
-- pattern; it cannot express one stream per Team, and stream ids are stored in
-- notification_subscriptions rows that would then need garbage-collecting every
-- time a Team archived. Four fixed streams plus this table is the same feature
-- with nothing to collect.
--
-- `last_digest_at` is the digest's ONLY state. There is no queue of pending items:
-- the worker asks what arrived after this timestamp and re-runs the access
-- resolver, so a deployment that was down for a day sends one correct digest
-- rather than replaying a backlog, and a user who lost forum access between the
-- post and the send is not emailed content they can no longer read.
CREATE TABLE IF NOT EXISTS team_notification_prefs (
user_id INT NOT NULL,
team_id INT NOT NULL,
muted TINYINT(1) NOT NULL DEFAULT 0,
-- 'off', and NOT the design-of-record's 'digest'. Digest-by-default would mean
-- every member of every Team starts receiving daily mail the moment an operator
-- connects Gmail, which is a decision about other people's inboxes made on their
-- behalf. Email is therefore the one sink here that is opt-IN; the mute is still
-- opt-out, because a mute silences something the user already asked for.
--
-- It also keeps this column honest as a deviation-from-default: a row written to
-- set `muted` alone leaves email exactly where it was.
email_mode ENUM('off','digest','immediate') NOT NULL DEFAULT 'off',
last_digest_at DATETIME NULL,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (user_id, team_id),
CONSTRAINT fk_tnp_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
CONSTRAINT fk_tnp_team FOREIGN KEY (team_id) REFERENCES teams(id) ON DELETE CASCADE,
-- The digest worker's driving query is "rows in digest mode, oldest send first",
-- which is a scan of this index rather than of every preference ever written.
INDEX idx_tnp_digest (email_mode, last_digest_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Migrations for databases created before the wiki upgrade. Each statement uses
-- IF NOT EXISTS so re-running on every boot is a harmless no-op. New installs get
-- these columns from the CREATE TABLE above; existing installs get them here.

View File

@@ -2,13 +2,18 @@
//
// What is left of config/notificationStreams.js once the shard-derived catalog
// moved to config/shardStreams.js (MODULE_SYSTEM.md §1.8: push INFRASTRUCTURE is
// core, the CATALOG is content). Exactly one stream is core's: `news.post` is
// produced by the website's own posts path, not by any game feed.
// core, the CATALOG is content). `news.post` is produced by the website's own
// posts path, not by any game feed, and the four `team.*` streams by core's own
// Team sync and forum.
//
// Registered through modules/registries.js like any module's, and read back
// through it — nothing imports this file to get "the catalog", because the
// catalog is core's plus every module's.
//
// Phase 6 added the four Team streams below. They are core's for the same reason
// the Team tables are: a module supplies who is in a Team, but who may be told
// about it is the access resolver's answer, and that is core's (TEAMS.md Part 6).
//
// The payload that ever leaves the server is a CONTENT-FREE tickle
// ({ stream, ref }); the app wakes and PULLS the real, ownership-checked content
// over the authenticated API (docs/android/PLAN.md §11).
@@ -21,6 +26,55 @@ const STREAMS = [
personal: false,
requiresLinkedAccount: false,
},
// ── Teams (TEAMS.md §6.2, phase 6) ───────────────────────────────────────
//
// FOUR streams, and not one per Team. The catalog is a static registration
// validated at boot; it has no way to express an unbounded runtime-created set,
// and a stream id per Team would leave rows in notification_subscriptions to
// collect every time a Team archived. Which Team an event came from lives in
// the RECIPIENT SET (utils/teamNotify.js) and in the `ref`, never in the id.
//
// `requiresLinkedAccount: false` on all four is deliberate and reads oddly.
// These are game-sourced events, so the instinct is to demand a linked game
// account — but a forum-granted user with no game identity at all is exactly
// the population §2.5 path 3 exists for, and they are a legitimate recipient of
// `team.forum.post`. The flag would refuse them a toggle they have every right
// to. What enforces who gets what is the recipient computation, which asks the
// access resolver; the stream flag is not a second, weaker copy of that rule.
//
// `personal: false` for the same reason it is false on news.post: these are not
// owner-keyed events about one account's own property. `publishToUsers` is a
// third fan-out shape alongside "everyone subscribed" and "this one owner", and
// the catalog has no flag for it because the flag would say nothing a caller
// does not already know by choosing the function.
{
id: 'team.member.joined',
label: 'Team — new member',
description: 'Someone joined a Team you belong to.',
personal: false,
requiresLinkedAccount: false,
},
{
id: 'team.leadership.changed',
label: 'Team — leadership change',
description: 'Leadership changed in a Team you belong to.',
personal: false,
requiresLinkedAccount: false,
},
{
id: 'team.forum.post',
label: 'Team — new forum post',
description: 'A new thread or reply in a Team forum you can read.',
personal: false,
requiresLinkedAccount: false,
},
{
id: 'team.announcement',
label: 'Team — announcements',
description: 'A leader posted an announcement in a Team you can read.',
personal: false,
requiresLinkedAccount: false,
},
]
module.exports = { STREAMS }

View File

@@ -48,4 +48,44 @@ const endpointsForUserStream = (userId, streamId) =>
[userId, streamId],
)
module.exports = { upsert, getByUserEndpoint, listByUser, remove, endpointsForStream, endpointsForUserStream }
// `Number.isInteger` alone is not enough: `Number(null)` is 0 and 0 is an
// integer, so a null slipping into a caller's list would become user id 0 and
// ride into an IN clause. No row has id 0, so it is harmless today — which is
// exactly why it would never be noticed.
const isUserId = (n) => Number.isInteger(n) && n > 0
// Endpoints of a COMPUTED SET of users' devices, each still gated on that user's
// own subscription (TEAMS.md §6.2's third fan-out shape).
//
// The set is the whole Team-scoping mechanism: the four `team.*` streams are
// global, and which Team an event belongs to is expressed by who is in `userIds`
// rather than by a stream id per Team. The caller has already resolved access and
// subtracted mutes; this function's only remaining job is to honour each
// recipient's own opt-in, which is why the JOIN is here and not left to the
// caller — a fan-out that skipped it would deliver to a user who had turned the
// stream off.
//
// Returns [] for an empty set rather than building `IN ()`, which is a syntax
// error in MariaDB. That case is common, not exceptional: most Team events have
// no subscribed recipients on a deployment with no app installed at all.
async function endpointsForUsersStream(userIds, streamId) {
const ids = [...new Set((userIds || []).map(Number).filter(isUserId))]
if (ids.length === 0) return []
return query(
`SELECT d.endpoint, d.transport
FROM push_devices d
JOIN notification_subscriptions s ON s.user_id = d.user_id
WHERE s.stream_id = ? AND d.user_id IN (${ids.map(() => '?').join(',')})`,
[streamId, ...ids],
)
}
module.exports = {
upsert,
getByUserEndpoint,
listByUser,
remove,
endpointsForStream,
endpointsForUserStream,
endpointsForUsersStream,
}

View File

@@ -28,5 +28,13 @@ const remove = async (id, userId) => (await db.remove(id, userId)) > 0
// Fan-out helpers: raw { endpoint, transport } rows (not toSafe-shaped).
const endpointsForStream = (streamId) => db.endpointsForStream(streamId)
const endpointsForUserStream = (userId, streamId) => db.endpointsForUserStream(userId, streamId)
const endpointsForUsersStream = (userIds, streamId) => db.endpointsForUsersStream(userIds, streamId)
module.exports = { register, listForUser, remove, endpointsForStream, endpointsForUserStream }
module.exports = {
register,
listForUser,
remove,
endpointsForStream,
endpointsForUserStream,
endpointsForUsersStream,
}

View File

@@ -0,0 +1,223 @@
// SQL for Team notification recipients and per-Team preferences (TEAMS.md Part 6).
//
// **The recipient set is the whole of Team scoping.** The four `team.*` streams
// are global and carry no Team in their id; who an event reaches is decided here.
// That is §6.2's design and it is not an optimisation — the push catalog is a
// static registration validated at boot, so a stream per Team is unexpressible,
// and stream ids live in `notification_subscriptions` rows that a per-Team id
// would leave behind every time a Team archived.
//
// **One recipient query serves all four streams**, because the two populations in
// §6.2's table are the same set written twice: "active members with a user_id,
// plus active forum grants" IS "everyone with resolved forum access", by the
// definition of teamAccess.forumAccess() (membership OR grant). What differs
// between the streams is only who is subtracted — the author of the post that
// caused it — and that is a caller's argument, not a second query.
//
// **Mutes are subtracted in SQL, not in the caller.** A recipient list that came
// back complete and was filtered afterwards would be one refactor away from being
// used unfiltered; there is no function here that returns an unmuted set.
const { query } = require('../../utils/db')
// The union, as a derived table both recipient functions build on. Written once
// so that "who is in a Team for notification purposes" has exactly one definition.
//
// `status = 'active'` on the membership half and `revoked_at IS NULL` on the
// grant half are the same two conditions the access resolver uses; a departed
// member and a revoked guest are both people who could still be read a private
// forum by a query that forgot one.
const RECIPIENT_UNION = `
SELECT user_id FROM team_members
WHERE team_id = ? AND status = 'active' AND user_id IS NOT NULL
UNION
SELECT user_id FROM team_forum_grants
WHERE team_id = ? AND revoked_at IS NULL`
// `Number.isInteger` alone is not enough: `Number(null)` is 0 and 0 is an
// integer, so a null slipping into a caller's list would become user id 0 and
// ride into an IN clause. No row has id 0, so it is harmless today — which is
// exactly why it would never be noticed.
const isUserId = (n) => Number.isInteger(n) && n > 0
/**
* Every user id that may be notified about `teamId`, mutes already removed.
*
* `exclude` is the author of the thing that happened. Passed rather than removed
* afterwards for the reason in the header, and taken as a list because a caller
* with nobody to exclude should not have to invent a sentinel.
*/
async function recipientIds(teamId, { exclude = [] } = {}) {
const skip = [...new Set(exclude.map(Number).filter(isUserId))]
const notMe = skip.length ? `AND r.user_id NOT IN (${skip.map(() => '?').join(',')})` : ''
const rows = await query(
`SELECT DISTINCT r.user_id
FROM (${RECIPIENT_UNION}) r
LEFT JOIN team_notification_prefs p ON p.user_id = r.user_id AND p.team_id = ?
WHERE COALESCE(p.muted, 0) = 0 ${notMe}`,
[teamId, teamId, teamId, ...skip],
)
return rows.map((r) => Number(r.user_id))
}
/**
* The same set, narrowed to those reachable by EMAIL and carrying each one's mode.
*
* A separate query rather than a join onto `recipientIds` because email has two
* conditions push does not: an address to send to, and an account still allowed to
* have one. A banned or disabled account keeps its forum grant in the ledger —
* revoking it is a separate staff decision — but must not keep receiving the
* Team's private discussion in its inbox.
*
* `email_mode` is COALESCEd to the column default rather than read as NULL — and
* that default is `'off'`, so this query returns the whole set with most of it
* marked as not wanting mail. Filtering to a mode is the CALLER's job, because
* `immediate` and `digest` are consumed by two different senders.
*/
async function emailRecipients(teamId, { exclude = [] } = {}) {
const skip = [...new Set(exclude.map(Number).filter(isUserId))]
const notMe = skip.length ? `AND u.id NOT IN (${skip.map(() => '?').join(',')})` : ''
return query(
`SELECT u.id AS user_id, u.username, u.email,
COALESCE(p.email_mode, 'off') AS email_mode,
p.last_digest_at
FROM (${RECIPIENT_UNION}) r
JOIN users u ON u.id = r.user_id
LEFT JOIN team_notification_prefs p ON p.user_id = u.id AND p.team_id = ?
WHERE COALESCE(p.muted, 0) = 0
AND u.email IS NOT NULL AND u.email <> ''
AND u.status = 'active' ${notMe}
GROUP BY u.id, u.username, u.email, p.email_mode, p.last_digest_at`,
[teamId, teamId, teamId, ...skip],
)
}
// ── Preferences ────────────────────────────────────────────────────────────
/**
* One row per Team this user may be notified about, whether or not a preference
* has ever been written for it — the account screen has to offer a Team the user
* has never touched, and a list built from the prefs table alone would be empty
* for exactly the users who have configured nothing.
*
* Archived Teams appear only when a preference row exists for them, so a mute the
* user set does not vanish from the screen the moment a guild disbands, while a
* disbanded guild nobody configured does not linger on it forever.
*/
async function prefsForUser(userId) {
return query(
`SELECT t.id AS team_id, t.slug, t.name, t.display_name_override, t.status AS team_status,
COALESCE(p.muted, 0) AS muted,
COALESCE(p.email_mode, 'off') AS email_mode
FROM teams t
LEFT JOIN team_notification_prefs p ON p.team_id = t.id AND p.user_id = ?
WHERE (
EXISTS (SELECT 1 FROM team_members m
WHERE m.team_id = t.id AND m.user_id = ? AND m.status = 'active')
OR EXISTS (SELECT 1 FROM team_forum_grants g
WHERE g.team_id = t.id AND g.user_id = ? AND g.revoked_at IS NULL)
OR p.user_id IS NOT NULL
)
ORDER BY t.status, t.name`,
[userId, userId, userId],
)
}
/** One Team's preference for one user, or undefined. Read by the mute toggle. */
async function prefFor(userId, teamId) {
const rows = await query(
`SELECT team_id, muted, email_mode, last_digest_at
FROM team_notification_prefs WHERE user_id = ? AND team_id = ?`,
[userId, teamId],
)
return rows[0]
}
/**
* Write one preference.
*
* An upsert that touches ONLY the columns it was given: the one-click unsubscribe
* writes `muted` and must not reset an `email_mode` the user chose, and the
* settings screen writes both. `last_digest_at` is never written here — it is the
* worker's column, and a preference change must not look like a delivery.
*/
async function setPref(userId, teamId, { muted, emailMode }) {
const sets = ['updated_at = CURRENT_TIMESTAMP']
if (muted != null) sets.push('muted = VALUES(muted)')
if (emailMode != null) sets.push('email_mode = VALUES(email_mode)')
await query(
`INSERT INTO team_notification_prefs (user_id, team_id, muted, email_mode)
VALUES (?, ?, ?, ?)
ON DUPLICATE KEY UPDATE ${sets.join(', ')}`,
[userId, teamId, muted ? 1 : 0, emailMode || 'off'],
)
}
/** Stamp a digest as delivered. The worker's column, and its only writer. */
async function stampDigest(userId, teamId, at) {
await query(
`INSERT INTO team_notification_prefs (user_id, team_id, last_digest_at)
VALUES (?, ?, ?)
ON DUPLICATE KEY UPDATE last_digest_at = VALUES(last_digest_at)`,
[userId, teamId, at],
)
}
/**
* Active Teams that have had forum activity since `since` — the digest worker's
* driving query.
*
* Driven from ACTIVITY rather than from the prefs table, which is what makes the
* worker's cost proportional to what was WRITTEN rather than to how many people
* once opened a settings screen. A Team nobody posted in costs one row of this
* query and no recipient computation at all.
*/
async function teamsWithForumActivitySince(since) {
return query(
`SELECT DISTINCT t.id, t.slug, t.name, t.display_name_override
FROM teams t
JOIN team_forum_threads th ON th.team_id = t.id
JOIN team_forum_posts po ON po.thread_id = th.id
WHERE t.status = 'active'
AND po.created_at > ?
AND po.status = 'visible'
AND th.status = 'visible'`,
[since],
)
}
/**
* The posts one digest covers: visible posts in visible threads, newer than the
* recipient's own `since`.
*
* Re-read at send time rather than accumulated at publish time. A queue of pending
* items would have to be garbage-collected, would replay a backlog after an outage,
* and — the reason that actually matters — could email a body a moderator hid in
* between. This query cannot: a hidden post is simply not in it.
*/
async function digestPostsSince(teamId, since, limit = 20) {
return query(
`SELECT po.id, po.thread_id, po.body_html, po.created_at, po.author_username,
th.title, th.type
FROM team_forum_posts po
JOIN team_forum_threads th ON th.id = po.thread_id
WHERE th.team_id = ?
AND po.created_at > ?
AND po.status = 'visible'
AND th.status = 'visible'
ORDER BY po.created_at
LIMIT ?`,
[teamId, since, Number(limit)],
)
}
module.exports = {
recipientIds,
emailRecipients,
prefsForUser,
prefFor,
setPref,
stampDigest,
teamsWithForumActivitySince,
digestPostsSince,
}

View File

@@ -0,0 +1,128 @@
// Per-Team notification preferences, and the recipient sets built from them
// (TEAMS.md §6.2§6.4, phase 6).
//
// **The absence of a row is the default, and the two sinks default OPPOSITE ways.**
// Push is opt-out: a user in one Team must never have to configure anything to be
// tickled about it, and the per-Team mute is how they stop. Email is opt-IN
// (`email_mode` defaults to `'off'`, deviating from §6.4 on the org lead's call):
// turning on Gmail in the admin panel must not start sending daily mail to every
// member of every Team on the deployment.
//
// Both are read the same way — COALESCE to the column default, never treat a
// missing row as "unknown" — so the asymmetry lives in ONE place, the schema, and
// not in a condition anybody has to remember.
//
// **This file never decides who may READ a Team.** It asks the same two tables
// teamAccess.forumAccess() asks, in one query, because a fan-out cannot afford a
// round trip per recipient — but it asks them for the same answer. If the access
// rule ever changes, both must; the SQL in teamNotify.db.js says so at the union
// it builds on, and the test that matters is the one asserting a revoked guest
// receives nothing.
const db = require('./teamNotify.db')
// Stored as an ENUM, restated here because a value arriving from a request body
// must be checked against something in JavaScript before it reaches the column —
// a bad value would otherwise be a 500 from the driver rather than a 400 from us.
const EMAIL_MODES = ['off', 'digest', 'immediate']
const isEmailMode = (v) => EMAIL_MODES.includes(v)
function publicPref(row) {
return {
teamId: Number(row.team_id),
slug: row.slug,
// The same `display_name_override || name` rule every other Team surface
// uses (§2.8.3). A notification screen showing the raw name would show a name
// staff have deliberately replaced everywhere else.
name: row.display_name_override || row.name,
archived: row.team_status === 'archived',
muted: Boolean(Number(row.muted)),
emailMode: row.email_mode,
}
}
/** Every Team this user could be notified about, with its current preference. */
async function listPrefs(userId) {
return (await db.prefsForUser(userId)).map(publicPref)
}
/** One Team's preference for one user, defaults applied. Never null. */
async function prefFor(userId, teamId) {
const row = await db.prefFor(userId, teamId)
return {
teamId: Number(teamId),
muted: Boolean(row && Number(row.muted)),
emailMode: (row && row.email_mode) || 'off',
}
}
/**
* Replace this user's whole set of Team preferences.
*
* PUT-the-whole-set, matching the existing subscription endpoint, and the
* Android gotcha carried forward from `docs/android/PLAN.md` §11 applies to the
* ROUTE rather than to this function: the array is required even when empty.
*
* **A preference may only be written for a Team the caller is actually in.** The
* ids are checked against `listPrefs`, not trusted from the body — otherwise any
* authenticated user could write a row naming any Team, which is a (small) write
* primitive into a table keyed by someone else's private membership. Unknown ids
* are dropped rather than 400'd: a Team the user left between loading the screen
* and saving it is an ordinary race, not a client bug.
*/
async function replacePrefs(userId, entries) {
const allowed = new Map((await listPrefs(userId)).map((p) => [p.teamId, p]))
const written = []
for (const entry of entries) {
const teamId = Number(entry && entry.teamId)
if (!allowed.has(teamId)) continue
const emailMode = isEmailMode(entry.emailMode) ? entry.emailMode : 'off'
// eslint-disable-next-line no-await-in-loop
await db.setPref(userId, teamId, { muted: Boolean(entry.muted), emailMode })
written.push(teamId)
}
return { written, prefs: await listPrefs(userId) }
}
/**
* Mute one Team for one user — the one-click unsubscribe's only effect.
*
* Deliberately narrow. The unsubscribe link is reached without a session, so what
* it can do is what an attacker holding a leaked link can do: silence one Team's
* notifications for one account, visibly and reversibly on the account screen.
* It writes no other column, and there is no "unsubscribe from everything".
*/
async function mute(userId, teamId) {
await db.setPref(userId, teamId, { muted: true })
}
/** Un-mute, for the toggle's other half. */
async function unmute(userId, teamId) {
await db.setPref(userId, teamId, { muted: false })
}
module.exports = {
EMAIL_MODES,
isEmailMode,
listPrefs,
prefFor,
replacePrefs,
mute,
unmute,
// Recipient sets, passed through so callers depend on the model rather than on
// the SQL. The fan-out in utils/teamNotify.js and the digest worker are the only
// callers.
//
// Wrapped rather than re-exported (`recipientIds: db.recipientIds`), which is
// the obvious shorter form and is wrong: that captures the function OBJECT at
// require time, so the layer below can never be substituted afterwards — which
// makes the db layer untestable in isolation and, more to the point, means the
// model is not really the seam it claims to be. These resolve `db.x` at call
// time, so the boundary is real.
recipientIds: (teamId, opts) => db.recipientIds(teamId, opts),
emailRecipients: (teamId, opts) => db.emailRecipients(teamId, opts),
stampDigest: (userId, teamId, at) => db.stampDigest(userId, teamId, at),
teamsWithForumActivitySince: (since) => db.teamsWithForumActivitySince(since),
digestPostsSince: (teamId, since, limit) => db.digestPostsSince(teamId, since, limit),
}

View File

@@ -108,4 +108,40 @@ async function publish(streamId, { ref, ownerUserId } = {}, deps = {}) {
await Promise.all(rows.map((r) => postTickle(r.endpoint, bodyStr, deps)))
}
module.exports = { publish, isAllowedEndpoint }
// `Number.isInteger` alone is not enough: `Number(null)` is 0 and 0 is an
// integer, so a null slipping into a caller's list would become user id 0 and
// ride into an IN clause. No row has id 0, so it is harmless today — which is
// exactly why it would never be noticed.
const isUserId = (n) => Number.isInteger(n) && n > 0
/**
* Publish one content-free tickle to a COMPUTED SET of users (TEAMS.md §6.2).
*
* The third fan-out shape. `publish` answers "everyone subscribed" and "this one
* owner"; Team notifications need "these N users", because the four `team.*`
* streams are global and which Team an event belongs to is expressed by who is in
* the set. Nothing about the tickle changes — same `{ stream, ref }`, same
* untrusted-relay assumption, same SSRF gate on every endpoint.
*
* The set arrives already resolved: the caller has asked the access resolver who
* may read this Team and subtracted the per-Team mutes. What this function still
* enforces is each recipient's own stream subscription, in the query. Never
* throws — a notification failing must not fail the write that produced it.
*/
async function publishToUsers(streamId, { ref, userIds } = {}, deps = {}) {
const devices = deps.pushDevices || pushDevicesModel
const ids = [...new Set((userIds || []).map(Number).filter(isUserId))]
if (ids.length === 0) return
let rows
try {
rows = await devices.endpointsForUsersStream(ids, streamId)
} catch (err) {
log.warn('push endpoint lookup failed', { streamId, message: err.message })
return
}
if (!rows || rows.length === 0) return
const bodyStr = JSON.stringify({ stream: streamId, ref: ref ?? null })
await Promise.all(rows.map((r) => postTickle(r.endpoint, bodyStr, deps)))
}
module.exports = { publish, publishToUsers, isAllowedEndpoint }