feat(engagement): deliverability — suppression, bounces and the verification gate
All checks were successful
PR Checks / client-build (pull_request) Successful in 36s
PR Checks / bot-tests (pull_request) Successful in 36s
PR Checks / server-tests (pull_request) Successful in 5m12s

ENGAGEMENT.md Phase 9, closing gap G16. Two mechanisms decide that somebody in a
rule's audience does not get the mail, and they sit at deliberately different
points in the pipeline.

`engagement_suppressions` is checked at DELIVERY: an outbox row can sit through a
rule's `delay_seconds` grace window and an address can bounce inside it, so the
only correct check is the one taken immediately before the transport call — which
is also what produces the `status='suppressed'` row with no transport call at all.

The Phase 1b verification gate is applied at ENQUEUE, through a new optional
`registerDeliveryChannel({ eligible })` that only `email` declares. Filtering the
shared audience would have silenced the wrong sink: a rule spanning email and
in-app must still put an item in an unverified user's inbox. The excluded counts
reach `summary.ineligible` and the admin reach preview, which until now reported
an audience size that was never the number of people who would be mailed.

`bounceClassify.js` is the only thing that may write a `bounce` row, and it is
deliberately NOT `mailer.PERMANENT_CODES`. That set answers "is retrying
pointless?" and contains EAUTH and 554 — an auth failure and a relay-wide policy
refusal, neither of which is a fact about the recipient. Reusing it would mean one
stale SMTP password suppressing every address the worker touched, silently. The
classifier reads the RFC 3463 enhanced status first, falls back to a phrase match
only past a veto list and only for 550/551/553, and does not suppress anything it
is unsure about.

Scope is engagement rules only: resets, invites, verification and the contact form
still attempt, matching the posture passwordReset.controller.js already stated.

Found on the live rig, against a real MariaDB and a real SMTP conversation: a hard
bounce was being recorded as `failed`, so the Send Log's "Bounced" filter — a
status `engagement_sends` has carried since §4.5 — matched nothing and always
would have. It is now its own outcome; the outbox row stays `failed`, since that
ENUM has no `bounced` and a bounced row is one that finished unsuccessfully.

`address_masked` is this phase's one addition to §4.5's DDL. A hash-only table
cannot be operated — an operator cannot tell three typos from a whole domain
refusing mail — and the domain survives while the local part is destroyed, so the
column can never be read back as an address book.

- schema: `engagement_suppressions` (+ `address_masked`, `created_by`)
- `GET/POST/DELETE /api/v1/admin/engagement/suppressions`, and Admin → Engagement
  → Suppressions, the only way out of the list
- `sendNotification` returns `smtp: { code, responseCode, response }`
- 26 new tests; swagger, routes manifest and guards regenerated

Docs: RunicGateway/docs#191.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-08-31 10:47:34 -05:00
parent 87c4e71025
commit c208543044
22 changed files with 2210 additions and 37 deletions

View File

@@ -132,11 +132,14 @@ const storedModes = async (userIds, streamId, channel) => {
* between the emit and the send is exactly the case this catches. The cost is one
* primary-key lookup on a path that is about to open an SMTP conversation.
*
* **It does not gate on `email_verified`.** Whether an unverified address may
* receive opt-in mail is §7.1 Q1's narrower half, and it is a Phase 9 decision
* with the suppression list in front of it; deciding it here by accident would
* mean every deployment that upgraded before verifying its users stopped mailing
* them.
* **It still does not gate on `email_verified`, and Phase 9 kept it that way.**
* §7.1 Q1's narrower half was settled by the org lead 2026-08-31: the gate
* excludes an unverified user at ENQUEUE, in `emailChannel.eligible`, so no
* outbox row is written and the admin reach preview can say how many were
* dropped. Adding the same condition here as well would look like defence in
* depth and would in fact be a second, invisible answer to the question — this
* function is also what a password reset would reach if it ever routed through
* the channel, and reset mail is deliberately ungated (`passwordReset.controller.js`).
*/
const addressFor = async (userId) => {
const rows = await query(
@@ -147,4 +150,63 @@ const addressFor = async (userId) => {
return rows.length ? { address: rows[0].email } : null
}
module.exports = { active, staff, subscribers, filterActive, storedModes, addressFor, MAX_AUDIENCE }
/**
* Which of these users hold an UNVERIFIED address - the set the Phase 1b gate
* excludes when it is on (Phase 9).
*
* A user with no address at all is in this set, and that is not incidental: the
* gate's question is "may we mail this person", and nowhere to send is a stronger
* no than an unconfirmed somewhere. `addressFor` refuses them at delivery either
* way; including them here is what stops an outbox row being written for a send
* that is already known to be impossible.
*
* One query for a whole audience. The engine calls it once per rule per event
* with up to MAX_AUDIENCE ids, so a per-user lookup would be five thousand round
* trips on the path that is supposed to be the cheap one.
*/
const unverifiedAmong = async (userIds) => {
const wanted = [...new Set(userIds.map(Number).filter((n) => Number.isInteger(n) && n > 0))]
if (!wanted.length) return new Set()
const capped = wanted.slice(0, MAX_AUDIENCE)
const rows = await query(
`SELECT id FROM users
WHERE id IN (${marks(capped)})
AND (email_verified = 0 OR email IS NULL OR email = '')`,
capped,
)
return new Set(ids(rows))
}
/**
* The addresses for a set of active users, as a Map - what the admin reach
* preview hashes to count how many of them are suppressed.
*
* It returns PLAINTEXT, which is the one thing this subsystem otherwise avoids,
* and there is no way around it: a suppression is keyed on the sha256 of an
* address, so answering "how many of these people are suppressed" requires
* hashing each one. The caller (`engagement.controller`) hashes immediately and
* returns only a count - no route ever serializes what this returns.
*/
const addressesFor = async (userIds) => {
const wanted = [...new Set(userIds.map(Number).filter((n) => Number.isInteger(n) && n > 0))]
if (!wanted.length) return new Map()
const capped = wanted.slice(0, MAX_AUDIENCE)
const rows = await query(
`SELECT id, email FROM users
WHERE id IN (${marks(capped)}) AND status = 'active' AND email IS NOT NULL AND email <> ''`,
capped,
)
return new Map(rows.map((r) => [Number(r.id), r.email]))
}
module.exports = {
active,
staff,
subscribers,
filterActive,
storedModes,
addressFor,
unverifiedAmong,
addressesFor,
MAX_AUDIENCE,
}

View File

@@ -0,0 +1,139 @@
const { query } = require('../../utils/db')
/**
* The suppression list (ENGAGEMENT.md G16, Phase 9).
*
* Every function here takes an ALREADY HASHED address. Hashing lives in
* `engagement/suppressions.js` beside the masking, so the two can never disagree
* about what a row for one address looks like; this file only reads and writes.
*/
/** Is this address suppressed on this channel? The row, or null. */
const get = async (addressHash, channel = 'email') => {
const rows = await query(
'SELECT * FROM engagement_suppressions WHERE address_hash = ? AND channel = ?',
[addressHash, channel],
)
return rows.length ? rows[0] : null
}
/**
* Suppress an address, or leave an existing row exactly as it is.
*
* `INSERT IGNORE`, deliberately, rather than an upsert. The FIRST reason an
* address was suppressed is the true one and the one an operator needs: an
* address that hard-bounced in March and was then manually re-added in June
* should still read `bounce`, because that is the fact that explains the mail
* stopping. An upsert would let the most recent write overwrite the diagnosis.
*
* @returns {Promise<boolean>} true when this call created the row
*/
const add = async (entry) => {
const result = await query(
`INSERT IGNORE INTO engagement_suppressions
(address_hash, address_masked, channel, reason, detail, created_by)
VALUES (?, ?, ?, ?, ?, ?)`,
[
entry.address_hash,
entry.address_masked ?? null,
entry.channel || 'email',
entry.reason,
entry.detail ? String(entry.detail).slice(0, 500) : null,
entry.created_by ?? null,
],
)
return Number(result.affectedRows || 0) > 0
}
/**
* Un-suppress an address. The one way out of this table, which is why the admin
* screen exists at all (Phase 9 decision 3).
*
* @returns {Promise<boolean>} true when a row was removed
*/
const remove = async (addressHash, channel = 'email') => {
const result = await query(
'DELETE FROM engagement_suppressions WHERE address_hash = ? AND channel = ?',
[addressHash, channel],
)
return Number(result.affectedRows || 0) > 0
}
/** WHERE-clause builder shared by `list` and `count`, so the two cannot disagree. */
const filters = ({ reason = null, channel = null, search = null } = {}) => {
const where = []
const params = []
if (reason) {
where.push('reason = ?')
params.push(reason)
}
if (channel) {
where.push('channel = ?')
params.push(channel)
}
if (search) {
// Against the MASKED column only. Searching the hash would need the caller to
// hash first, which makes a partial search impossible, and there is no
// plaintext column to search — that is the point of the table. A domain
// ("example.com") is what an operator actually types here, and the mask keeps
// the domain intact precisely so this works.
where.push('address_masked LIKE ?')
params.push(`%${String(search).slice(0, 120)}%`)
}
return { clause: where.length ? `WHERE ${where.join(' AND ')}` : '', params }
}
/** The admin list, newest first. */
const list = (opts = {}) => {
const { clause, params } = filters(opts)
return query(
`SELECT * FROM engagement_suppressions ${clause} ORDER BY created_at DESC, address_hash LIMIT ? OFFSET ?`,
[...params, opts.limit || 50, opts.offset || 0],
)
}
/** How many rows match the same filters — the paged screen's total. */
const count = async (opts = {}) => {
const { clause, params } = filters(opts)
const [row] = await query(`SELECT COUNT(*) AS n FROM engagement_suppressions ${clause}`, params)
return Number(row?.n || 0)
}
/**
* How many suppressions per reason — the summary the screen leads with.
*
* A count by reason is the difference between "eleven addresses are suppressed"
* and "eleven addresses hard-bounced", and only the second tells an operator
* whether to go and look at their relay.
*/
const countsByReason = async () => {
const rows = await query(
'SELECT reason, COUNT(*) AS n FROM engagement_suppressions GROUP BY reason',
)
const out = {}
for (const r of rows) out[r.reason] = Number(r.n || 0)
return out
}
/**
* The subset of these hashes that is suppressed, as a Set.
*
* One query for a whole audience rather than one per recipient. Not used by the
* delivery path — that checks a single address at send time, after the delay
* window, which is the only check that can be correct — but by the admin reach
* preview, which is asked about thousands of users at once and must not become
* thousands of round trips.
*/
const suppressedAmong = async (addressHashes, channel = 'email') => {
const wanted = [...new Set((addressHashes || []).filter(Boolean))]
if (!wanted.length) return new Set()
const marks = wanted.map(() => '?').join(', ')
const rows = await query(
`SELECT address_hash FROM engagement_suppressions
WHERE channel = ? AND address_hash IN (${marks})`,
[channel, ...wanted],
)
return new Set(rows.map((r) => r.address_hash))
}
module.exports = { get, add, remove, list, count, countsByReason, suppressedAmong }