RunicGateway/website
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity

feat(engagement): deliverability — suppression, bounces and the verification gate
All checks were successful
PR Checks / client-build (pull_request) Successful in 36s
Details
PR Checks / bot-tests (pull_request) Successful in 36s
Details
PR Checks / server-tests (pull_request) Successful in 5m12s
Details

Browse Source 
ENGAGEMENT.md Phase 9, closing gap G16. Two mechanisms decide that somebody in a
rule's audience does not get the mail, and they sit at deliberately different
points in the pipeline.

`engagement_suppressions` is checked at DELIVERY: an outbox row can sit through a
rule's `delay_seconds` grace window and an address can bounce inside it, so the
only correct check is the one taken immediately before the transport call — which
is also what produces the `status='suppressed'` row with no transport call at all.

The Phase 1b verification gate is applied at ENQUEUE, through a new optional
`registerDeliveryChannel({ eligible })` that only `email` declares. Filtering the
shared audience would have silenced the wrong sink: a rule spanning email and
in-app must still put an item in an unverified user's inbox. The excluded counts
reach `summary.ineligible` and the admin reach preview, which until now reported
an audience size that was never the number of people who would be mailed.

`bounceClassify.js` is the only thing that may write a `bounce` row, and it is
deliberately NOT `mailer.PERMANENT_CODES`. That set answers "is retrying
pointless?" and contains EAUTH and 554 — an auth failure and a relay-wide policy
refusal, neither of which is a fact about the recipient. Reusing it would mean one
stale SMTP password suppressing every address the worker touched, silently. The
classifier reads the RFC 3463 enhanced status first, falls back to a phrase match
only past a veto list and only for 550/551/553, and does not suppress anything it
is unsure about.

Scope is engagement rules only: resets, invites, verification and the contact form
still attempt, matching the posture passwordReset.controller.js already stated.

Found on the live rig, against a real MariaDB and a real SMTP conversation: a hard
bounce was being recorded as `failed`, so the Send Log's "Bounced" filter — a
status `engagement_sends` has carried since §4.5 — matched nothing and always
would have. It is now its own outcome; the outbox row stays `failed`, since that
ENUM has no `bounced` and a bounced row is one that finished unsuccessfully.

`address_masked` is this phase's one addition to §4.5's DDL. A hash-only table
cannot be operated — an operator cannot tell three typos from a whole domain
refusing mail — and the domain survives while the local part is destroyed, so the
column can never be read back as an address book.

- schema: `engagement_suppressions` (+ `address_masked`, `created_by`)
- `GET/POST/DELETE /api/v1/admin/engagement/suppressions`, and Admin → Engagement
  → Suppressions, the only way out of the list
- `sendNotification` returns `smtp: { code, responseCode, response }`
- 26 new tests; swagger, routes manifest and guards regenerated

Docs: RunicGateway/docs#191.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
claude
2026-08-31 10:47:34 -05:00
parent 87c4e71025
commit c208543044
22 changed files with 2210 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files

216
server/src/engagement/bounceClassify.js Normal file
View File

@@ -0,0 +1,216 @@
// ── Which send failures are facts about the RECIPIENT ───────────────────────
//
// ENGAGEMENT.md Phase 9. The suppression list's whole value is that an address on
// it is genuinely undeliverable; the moment it fills with addresses that were
// fine, an operator learns to ignore it and it may as well not exist. This file
// is the one place that judgement is made.
//
// **It is deliberately NOT `mailer.PERMANENT_CODES`, and reusing that set would
// have been a mass-suppression bug.** That set answers "is retrying pointless?"
// and holds `EAUTH` and `554` alongside `550` — an authentication failure and a
// relay-wide policy refusal. Both are permanent and neither says anything about
// the person: one wrong SMTP password would suppress every address the outbox
// worker touched before anybody noticed the mail had stopped. "Do not retry" and
// "this mailbox does not exist" are different questions, and this file only
// answers the second.
//
// **The primary signal is the enhanced status code (RFC 3463), not the reply
// code.** `550` alone is the catch-all every refusal arrives as; `5.1.1` means
// one specific thing — no such mailbox. Every relay worth configuring emits an
// enhanced code, so it is read first and, when present, decides on its own.
//
// **The fallback is narrow on purpose.** Without an enhanced code a phrase match
// is all that is left, and phrase matching is how a classifier quietly starts
// suppressing everything. So it applies only after the reply code has already
// narrowed the failure to the recipient address — 550, 551 and 553 are RFC 5321's
// recipient-address codes — and only for phrases that cannot mean anything else,
// with a veto list checked first. `552` (storage exceeded) and `554` (transaction
// failed) are excluded from even that: a full mailbox gets emptied, and a generic
// transaction failure is generic.
//
// Anything this file is unsure about is NOT suppressed. The cost of a false
// negative is mailing a dead address again next month; the cost of a false
// positive is a person who silently stops hearing from the deployment and has no
// way to find out.
// RFC 3463 subject.detail pairs that mean "this address will not accept mail,
// today or ever". Kept as strings because `5.1.10` and `5.1.1` are different
// codes and numeric parsing loses that.
const PERMANENT_RECIPIENT = new Set([
'1.1', // bad destination mailbox address — no such user
'1.2', // bad destination system address — the domain does not take mail
'1.3', // bad destination mailbox address syntax
'1.6', // mailbox has moved, no forwarding address
'1.10', // recipient address has a null MX (RFC 7505)
'2.1', // mailbox disabled, not accepting messages
])
// Enhanced subjects that are permanent but are NOT about the recipient. Listed
// rather than merely omitted, because each is a plausible-looking 5.x.y that a
// later edit would otherwise be tempted to add:
// 2.2 — mailbox full. Permanent-coded by some relays, emptied by every user.
// 7.x — policy. Our sending reputation, our SPF, our content; the recipient is
// the one party it is not about.
// 3.x — the destination MAIL SYSTEM is full or refusing. Not the mailbox.
// 5.x — protocol failure. A bug at one end or the other.
const NEVER_RECIPIENT_SUBJECTS = new Set(['3', '5', '7'])
// RFC 5321 reply codes that name the recipient address specifically. 554 is
// absent deliberately: "transaction failed" is what a relay reaches for when it
// does not want to say why, and it is the commonest shape of a content or policy
// rejection.
const RECIPIENT_REPLY_CODES = new Set([550, 551, 553])
// Phrases that only ever mean "no such mailbox", checked only once a reply code
// above has established the failure is about the address. Each is a substring of
// a real refusal from a widely deployed MTA (Postfix, Exim, Exchange, Google,
// Microsoft 365).
const NO_SUCH_MAILBOX = [
'user unknown',
'unknown user',
'no such user',
'no such recipient',
'unknown recipient',
'invalid recipient',
'recipient address rejected',
'recipient not found',
'address does not exist',
'does not exist',
'mailbox unavailable',
'mailbox not found',
'no mailbox',
'user does not exist',
'address rejected',
]
// Phrases that appear alongside the ones above and mean the opposite, checked
// FIRST. "Mailbox unavailable" is a substring of the sentence a relay sends when
// a mailbox is merely full, so a substring match with no veto list would read a
// temporary condition as a dead address.
const NOT_A_DEAD_MAILBOX = [
'full',
'quota',
'storage',
'temporar',
'try again',
'greylist',
'rate limit',
'too many',
'spam',
'blocked',
'blacklist',
'blocklist',
'reputation',
'policy',
'authentication',
'not authorized',
]
/**
* The enhanced status code in an SMTP response, as `{ class, subject, detail }`,
* or null.
*
* Anchored to the start of the line rather than searched for anywhere in it: a
* bounce that quotes another server's answer ("...said: 550 5.1.1...") contains
* two, and the one that matters is the one this relay just gave us. A free search
* finds whichever comes first, which is not the same thing.
*/
function parseEnhanced(response) {
if (!response) return null
const m = /^\s*(\d{3})[\s-]+(\d)\.(\d{1,3})\.(\d{1,3})\b/.exec(String(response))
if (!m) return null
return { class: m[2], subject: m[3], detail: m[4] }
}
/** The three-digit reply code, off the error object or out of the response text. */
function replyCode(err) {
const direct = Number(err && err.responseCode)
if (Number.isInteger(direct) && direct >= 400 && direct <= 599) return direct
const m = /^\s*(\d{3})\b/.exec(String((err && err.response) || ''))
return m ? Number(m[1]) : null
}
const lower = (s) => String(s || '').toLowerCase()
/**
* Should this send failure suppress the address?
*
* @param {object} err the error a transport's send threw, or an object carrying
* the `responseCode` / `response` / `code` lifted off one
* @returns {{ suppress: boolean, reason: string, evidence: string|null }}
*
* `reason` is populated on a refusal too, and that is not decoration: it becomes
* the send log's `detail`, so "not suppressed: 554 does not name the recipient
* address" is the line that stops somebody re-deriving this decision from an
* unexplained non-event six months from now.
*/
function classify(err) {
const e = err || {}
const response = e.response || e.message || ''
const enhanced = parseEnhanced(response)
const code = replyCode(e)
// No reply code at all means the failure happened before or outside the SMTP
// transaction: the connection, the credentials, the socket. Never the
// recipient. `EAUTH` lands here, which is the whole reason this file exists.
if (!code) {
return {
suppress: false,
reason: `no SMTP reply code (${e.code || 'transport failure'}); not a recipient failure`,
evidence: null,
}
}
if (code < 500) {
return { suppress: false, reason: `${code} is a temporary failure`, evidence: null }
}
if (enhanced) {
const pair = `${enhanced.subject}.${enhanced.detail}`
if (enhanced.class !== '5') {
return { suppress: false, reason: `enhanced status ${enhanced.class}.${pair} is not permanent`, evidence: null }
}
if (PERMANENT_RECIPIENT.has(pair)) {
return { suppress: true, reason: 'bounce', evidence: `5.${pair}` }
}
if (NEVER_RECIPIENT_SUBJECTS.has(enhanced.subject)) {
return {
suppress: false,
reason: `5.${pair} is about the server or our standing with it, not the address`,
evidence: `5.${pair}`,
}
}
// A permanent 5.x.y this file has no opinion on. Unknown means no.
return {
suppress: false,
reason: `5.${pair} is not a known recipient failure`,
evidence: `5.${pair}`,
}
}
// No enhanced code: the narrow fallback.
if (!RECIPIENT_REPLY_CODES.has(code)) {
return { suppress: false, reason: `${code} does not name the recipient address`, evidence: null }
}
const text = lower(response)
const veto = NOT_A_DEAD_MAILBOX.find((p) => text.includes(p))
if (veto) {
return { suppress: false, reason: `${code}, but the response says "${veto}"`, evidence: null }
}
const hit = NO_SUCH_MAILBOX.find((p) => text.includes(p))
if (hit) {
return { suppress: true, reason: 'bounce', evidence: `${code} "${hit}"` }
}
return { suppress: false, reason: `${code} with no enhanced status and no recognised reason`, evidence: null }
}
module.exports = {
classify,
parseEnhanced,
replyCode,
PERMANENT_RECIPIENT,
NEVER_RECIPIENT_SUBJECTS,
RECIPIENT_REPLY_CODES,
NO_SUCH_MAILBOX,
NOT_A_DEAD_MAILBOX,
}

52
server/src/engagement/channels.js
View File

@@ -53,6 +53,24 @@ const isMode = (value) => MODES.includes(value)
* without one is declared but not yet deliverable, which is exactly what
* `inapp` is until Phase 7; the worker finishes such a row `failed` and
* says so in the send log rather than pretending it was sent.
* @param {(userIds: number[]) => Promise<{userIds: number[], excluded: object}>} [def.eligible]
* Phase 9. Narrow an already-resolved audience to the users this channel
* may write an outbox row for, and say how many it dropped and why.
*
* **It exists so the engine can stay channel-agnostic.** The verification
* gate is an email fact — an unverified address is a reason not to mail
* somebody and no reason at all not to put an item in their inbox — and a
* rule may name both channels. Filtering the shared audience before the
* per-channel loop would have silenced the wrong sink; an `if (channel ===
* 'email')` in `engine.js` would have put a channel's rule inside the
* generic engine. This is the seam that is neither.
*
* Distinct from `deliver`'s refusals on purpose: this runs at ENQUEUE and
* is for standing properties of a user (is this address verified), which
* are stable across a delay window and are worth not writing a row for.
* A suppression is not one of those — it can appear between the enqueue
* and the send — so it is checked in `deliver`, where it produces a
* `suppressed` row in the send log the acceptance criterion asks for.
*/
function registerDeliveryChannel(def) {
if (!def || typeof def !== 'object') throw new Error('registerDeliveryChannel: definition required')
@@ -81,7 +99,7 @@ function registerDeliveryChannel(def) {
// silently never delivers — the failure Phase 3 deferred the whole behavioural
// half to avoid freezing, and the one the worker's "no delivery implementation
// yet" branch would report as if it were by design.
for (const fn of ['addressFor', 'deliver']) {
for (const fn of ['addressFor', 'deliver', 'eligible']) {
if (def[fn] !== undefined && typeof def[fn] !== 'function') {
throw new Error(`registerDeliveryChannel(${id}): ${fn} must be a function`)
}
@@ -96,6 +114,7 @@ function registerDeliveryChannel(def) {
supportsDigest,
addressFor: def.addressFor,
deliver: def.deliver,
eligible: def.eligible,
})
return id
}
@@ -103,13 +122,14 @@ function registerDeliveryChannel(def) {
/**
* Every channel, in registration order. The preferences screen's column set.
*
* **Declarative fields only** — `addressFor` and `deliver` are stripped. This is
* what a route serializes, and a function on an object bound for `res.json` is a
* key that silently disappears rather than an error; keeping the boundary here
* means the API shape is decided in one place instead of by JSON.stringify.
* **Declarative fields only** — `addressFor`, `deliver` and `eligible` are
* stripped. This is what a route serializes, and a function on an object bound
* for `res.json` is a key that silently disappears rather than an error; keeping
* the boundary here means the API shape is decided in one place instead of by
* JSON.stringify.
*/
const all = () =>
[...channels.values()].map(({ addressFor, deliver, ...declared }) => ({ ...declared }))
[...channels.values()].map(({ addressFor, deliver, eligible, ...declared }) => ({ ...declared }))
/** Just the ids. */
const ids = () => [...channels.keys()]
@@ -137,6 +157,25 @@ const modesFor = (id) => {
/** Is `mode` a mode this channel accepts? The gate on every preference write. */
const acceptsMode = (id, mode) => modesFor(id).includes(mode)
/**
* Narrow an audience to the users this channel may enqueue for (Phase 9).
*
* The default for a channel that declares no `eligible` is "everyone the
* audience resolved to", which is what every channel but email does. It lives
* here rather than at each call site so the two consumers — the engine and the
* admin reach preview — cannot answer the question differently, which is exactly
* how a preview comes to promise a number the engine will not deliver.
*/
async function eligibleFor(id, userIds) {
const c = channels.get(id)
if (!c || typeof c.eligible !== 'function') return { userIds: userIds.slice(), excluded: {} }
const result = await c.eligible(userIds)
return {
userIds: (result && result.userIds) || [],
excluded: (result && result.excluded) || {},
}
}
// Test-only: the registry is module-level state.
function _reset() {
channels.clear()
@@ -153,5 +192,6 @@ module.exports = {
defaultMode,
modesFor,
acceptsMode,
eligibleFor,
_reset,
}

4
server/src/engagement/coreChannels.js
View File

@@ -66,6 +66,10 @@ const CHANNELS = [
// time (§4.2b), which is a different delivery path rather than a batched one.
addressFor: emailChannel.addressFor,
deliver: emailChannel.deliver,
// Phase 9: the only channel that declares one. The Phase 1b verification
// gate is an email fact, and this is the seam that keeps it out of the
// generic engine - see channels.js's `eligible` docs.
eligible: emailChannel.eligible,
},
{
id: 'inapp',

116
server/src/engagement/emailChannel.js
View File

@@ -29,11 +29,12 @@
// `teamName` (a display string the cooldown keys on) and the scope is `team:12`.
// A Team renamed between the mail and the click must not orphan the link in it.
const crypto = require('crypto')
const rulesDb = require('../model/engagement/engagementRules.db')
const recipients = require('../model/engagement/engagementRecipients.db')
const templates = require('./templates')
const projection = require('./projection')
const suppressions = require('./suppressions')
const settings = require('../model/settings/settings.model')
const unsubscribeToken = require('../utils/unsubscribeToken')
const log = require('../utils/logger')('engagement')
@@ -54,12 +55,10 @@ const DEFAULT_TEMPLATE = 'notify.event'
const baseUrl = () => templates.baseUrl()
// Lower-cased first: a bounce reported for "Darrow@example.com" has to match the
// row written for "darrow@example.com", and a hash of two spellings is two
// different rows. The local part is technically case-sensitive per RFC 5321 and
// no relay anybody deploys treats it that way.
const hashAddress = (address) =>
crypto.createHash('sha256').update(String(address).trim().toLowerCase()).digest('hex')
// Re-exported rather than defined here since Phase 9: the send log's hash and
// the suppression list's key have to be the same function or a bounce never finds
// the row it belongs to. `suppressions.js` owns it, next to the masking.
const hashAddress = suppressions.hashAddress
/**
* The two unsubscribe URLs for one recipient of one scope, or nulls.
@@ -94,6 +93,52 @@ function unsubscribeUrls(userId, scopeKey) {
/** Where this channel would send to, or null. */
const addressFor = (userId) => recipients.addressFor(userId)
/**
* Narrow an audience to the users this channel may write an outbox row for
* (Phase 9, decision 4).
*
* **One gate, and it is the Phase 1b verification setting.** With
* `email_verification_required` on, a user whose address is unverified is
* excluded here rather than refused at delivery, and the org lead settled it that
* way for two reasons. It is a STANDING property — unlike a suppression, which
* can appear inside a `delay_seconds` window and therefore has to be re-checked
* at send time — so the outbox row would be written only to be thrown away. And a
* deployment that upgraded before verifying anybody has an audience that is
* almost entirely unverified: excluding at delivery would write a `suppressed`
* row per person per rule firing, which is a send log nobody can read.
*
* The count comes back so the admin reach preview can say "1,204 excluded:
* unverified" instead of quietly promising a number the engine will not deliver.
*
* **It fails OPEN, and the try/catch is load-bearing rather than defensive
* habit.** `settings.isEmailVerificationRequired` swallows its own errors and
* answers `off`, but `unverifiedAmong` does not, and an uncaught throw here does
* not fail one recipient — `applyRule` awaits this before the per-user loop, so
* it would abandon the whole rule for every channel it names. A database having
* a bad minute would become a rule that silently sent nothing, with a clean send
* log and nothing in the outbox to retry. Same direction as the suppression
* check, for the same reason: the recoverable mistake is mail going out, not mail
* silently stopping.
*/
async function eligible(userIds) {
const list = userIds || []
if (!list.length) return { userIds: [], excluded: {} }
try {
if (!(await settings.isEmailVerificationRequired())) {
return { userIds: list.slice(), excluded: {} }
}
const unverified = await recipients.unverifiedAmong(list)
if (!unverified.size) return { userIds: list.slice(), excluded: {} }
return {
userIds: list.filter((id) => !unverified.has(Number(id))),
excluded: { unverified: unverified.size },
}
} catch (err) {
log.error('verification gate could not be evaluated; not excluding anyone', { message: err.message })
return { userIds: list.slice(), excluded: {} }
}
}
/**
* Deliver one claimed outbox row.
*
@@ -130,11 +175,60 @@ async function deliver(row) {
log.debug('template variables had no value', { key, missing: rendered.missing })
}
// **The suppression check is HERE and not at enqueue** (Phase 9). An outbox
// row can sit through a `delay_seconds` grace window, and an address can hard
// bounce inside it — so the only check that can be correct is the one taken
// immediately before the transport call. It is also the check the acceptance
// criterion describes: a `suppressed` row in the send log, and no transport
// call at all.
const blocked = await suppressions.isSuppressed(to.address)
if (blocked) {
return {
ok: false,
suppressed: true,
detail: `address is suppressed (${blocked.reason})`,
addressHash: hashAddress(to.address),
}
}
const result = await mailer().sendNotification({ to: to.address, rendered, ...unsub })
// A failed send is where a hard bounce enters the system on SMTP, and the
// reason it is worth catching rather than waiting for an API provider: a
// single-recipient send refused at RCPT TO is a synchronous 5.1.1, which is
// the most valuable deliverability signal there is and it was already being
// thrown away. `considerFailure` is narrow — see bounceClassify.js — and its
// note goes into the log on BOTH outcomes, so "this failed and was not
// suppressed" says why.
if (result && !result.ok && result.smtp) {
const verdict = await suppressions.considerFailure({ address: to.address, error: result.smtp })
// A bounce is terminal by definition. Overriding `retry` matters because
// `PERMANENT_CODES` does not contain every code that can carry a 5.1.x, so
// without this a genuine dead mailbox could still be retried four more
// times — each one another refusal on our record with the relay.
if (verdict.suppressed) {
return {
ok: false,
retry: false,
// `engagement_sends.status` has carried 'bounced' since §4.5 and
// nothing wrote it until here, so the Send Log's "Bounced" filter
// matched nothing — the live rig is what showed that. It is a distinct
// status rather than a flavour of 'failed' because the two need
// different actions: a failure means look at the relay, and a bounce
// means that person's address is gone.
bounced: true,
transport: result.transport,
detail: `${result.detail} — ${verdict.note}`,
addressHash: hashAddress(to.address),
}
}
return { ...result, detail: `${result.detail} — ${verdict.note}`, addressHash: hashAddress(to.address) }
}
// The send log stores a sha256 of the address and never the address itself
// (schema.sql): enough to correlate a bounce in Phase 9, useless as a mailing
// list. Attached on every outcome, because a failure is exactly the row a
// bounce would need to be matched against.
// (schema.sql): enough to correlate a bounce, useless as a mailing list.
// Attached on every outcome, because a failure is exactly the row a bounce
// would need to be matched against.
return { ...result, addressHash: hashAddress(to.address) }
} catch (err) {
// See the header: a throw here would be retried as if it were the relay's
@@ -144,4 +238,4 @@ async function deliver(row) {
}
}
module.exports = { addressFor, deliver, unsubscribeUrls, hashAddress, DEFAULT_TEMPLATE }
module.exports = { addressFor, eligible, deliver, unsubscribeUrls, hashAddress, DEFAULT_TEMPLATE }

25
server/src/engagement/engine.js
View File

@@ -120,7 +120,18 @@ async function subscribedTo(userIds, streamId, channel, scopeKey = null) {
* for tests; it is not read by the caller for control flow.
*/
async function applyRule(rule, event, now) {
const summary = { ruleId: rule.id, enqueued: 0, deduped: 0, cooled: 0, capped: 0, skipped: null }
const summary = {
ruleId: rule.id,
enqueued: 0,
deduped: 0,
cooled: 0,
capped: 0,
// Phase 9: people a CHANNEL refused to enqueue for, by reason. Counted
// separately from `cooled` and `capped` because those are the engine holding
// a message back and this is a channel saying it cannot carry one at all.
ineligible: {},
skipped: null,
}
if (!conditions.evaluate(rule.conditions, event.data)) {
summary.skipped = 'conditions'
@@ -176,7 +187,17 @@ async function applyRule(rule, event, now) {
const dueAt = new Date(now.getTime() + Math.max(0, rule.delay_seconds) * 1000)
for (const channel of live) {
const eligible = await subscribedTo(resolved.userIds, event.triggerId, channel, event.scopeKey)
// Phase 9, and it runs BEFORE the preference filter rather than after. Both
// orders reach the same recipients; this one costs one query against the
// narrower set only when the channel declares an `eligible` at all, and it
// means `summary.ineligible` counts people the CHANNEL cannot reach rather
// than people who happened to also be opted in. Channels that declare none —
// push and in-app — pass straight through.
const gated = await channels.eligibleFor(channel, resolved.userIds)
for (const [why, n] of Object.entries(gated.excluded)) {
summary.ineligible[why] = (summary.ineligible[why] || 0) + n
}
const eligible = await subscribedTo(gated.userIds, event.triggerId, channel, event.scopeKey)
for (const userId of eligible) {
if (budget <= 0) {
summary.capped += 1

160
server/src/engagement/suppressions.js Normal file
View File

@@ -0,0 +1,160 @@
// ── The suppression list ───────────────────────────────────────────────────
//
// ENGAGEMENT.md G16, Phase 9. Addresses this deployment has stopped mailing,
// and the two questions asked of them: "may I send to this one?" at delivery
// time, and "why did this one stop?" on the admin screen.
//
// **Scope: the engagement email channel only** (Phase 9 decision 2). A password
// reset, an invite, a verification mail and the contact form are all
// user-INITIATED and still attempt, exactly as they still attempt to an
// unverified address (`passwordReset.controller.js`). The posture is the same one
// that file already states: a background system's opinion about an address must
// not be able to lock somebody out of their own account. One reset to a dead
// mailbox is not a reputation problem; a rule mailing three thousand people every
// week is, and that is what this list guards.
//
// **The table holds a hash and a mask, never an address.** The hash is what
// correlates a bounce back to an `engagement_sends` row (Phase 6 was already
// writing `address_hash` on every outcome for this). The mask —
// `d***@example.com` — is Phase 9's one addition to §4.5's DDL and exists because
// a screen of sha256 digests cannot be operated: an operator has to be able to
// see that a whole domain is refusing mail, and to find the person who fixed
// their mailbox and let them back in. The local part is DESTROYED rather than
// shortened, so the column cannot be turned back into an address book.
//
// **Nothing here writes a suppression from "the send failed".** What may write
// one is `bounceClassify.classify`, which is a much narrower question — see that
// file's header for why reusing `mailer.PERMANENT_CODES` would have suppressed
// every address the moment an SMTP password went stale.
const crypto = require('crypto')
const db = require('../model/engagement/engagementSuppressions.db')
const bounceClassify = require('./bounceClassify')
const log = require('../utils/logger')('engagement')
const REASONS = ['bounce', 'complaint', 'manual', 'unverified']
/**
* The key an address is stored under.
*
* Lower-cased first, and that matters more here than anywhere else in the
* subsystem: a bounce reported for `Darrow@example.com` has to find the row
* written for `darrow@example.com`, and a hash of two spellings is two rows that
* never meet. RFC 5321 says the local part is technically case-sensitive; no
* relay anybody deploys treats it that way.
*/
const hashAddress = (address) =>
crypto.createHash('sha256').update(String(address).trim().toLowerCase()).digest('hex')
/**
* `darrow@example.com` → `d***@example.com`. Null for anything that is not an
* address.
*
* The domain survives intact because domain-level patterns are the signal an
* operator is actually looking for — "everything to this company is bouncing" is
* a different problem from three people mistyping their own address, and only the
* domain distinguishes them.
*
* The first character of the local part survives only when there are at least
* three, which is not fussiness: for a two-letter local part, one revealed
* character plus the domain is most of the address.
*/
function maskAddress(address) {
const s = String(address || '').trim()
const at = s.lastIndexOf('@')
if (at <= 0 || at === s.length - 1) return null
const local = s.slice(0, at)
const domain = s.slice(at + 1).toLowerCase()
const head = local.length >= 3 ? local[0].toLowerCase() : ''
return `${head}***@${domain}`.slice(0, 190)
}
/** Is this address suppressed on this channel? */
async function isSuppressed(address, channel = 'email') {
if (!address) return null
try {
return await db.get(hashAddress(address), channel)
} catch (err) {
// Fail OPEN, and the direction is deliberate. A database that cannot answer
// "is this suppressed" must not stop the deployment's mail; the failure mode
// it would otherwise produce is total silence with a clean send log, which is
// exactly G22's shape. Mailing one dead address during an outage is the
// cheaper mistake.
log.error('suppression check failed; sending anyway', { message: err.message })
return null
}
}
/**
* Suppress an address. Returns true when this call created the row.
*
* `reason` is validated rather than trusted: it is an ENUM in the schema, so an
* unknown value is a 500 from the driver at the worst possible moment (inside a
* failure handler), and the callers include an admin route.
*/
async function suppress({ address, reason, detail = null, channel = 'email', createdBy = null }) {
if (!address) return false
if (!REASONS.includes(reason)) throw new Error(`suppress: unknown reason "${reason}"`)
const created = await db.add({
address_hash: hashAddress(address),
address_masked: maskAddress(address),
channel,
reason,
detail,
created_by: createdBy,
})
if (created) {
// Masked, never the address — the same rule every other log line in this
// subsystem follows. It is logged at all because an address dropping off the
// mailing list is the kind of change an operator finds out about weeks later
// otherwise.
log.info('address suppressed', { address: maskAddress(address), reason, channel })
}
return created
}
/** Un-suppress. Returns true when a row was removed. */
async function unsuppress(address, channel = 'email') {
if (!address) return false
const removed = await db.remove(hashAddress(address), channel)
if (removed) log.info('suppression lifted', { address: maskAddress(address), channel })
return removed
}
/**
* Consider a failed send for suppression, and say what was decided.
*
* The seam between a delivery failure and this list, and the only one — nothing
* else in the codebase writes a `bounce` row. Called from `emailChannel.deliver`
* with the error the transport threw.
*
* @returns {Promise<{suppressed: boolean, note: string}>} `note` goes into the
* send log's detail, on both outcomes.
*/
async function considerFailure({ address, error, channel = 'email' }) {
const verdict = bounceClassify.classify(error)
if (!verdict.suppress) {
return { suppressed: false, note: `not suppressed (${verdict.reason})` }
}
try {
const detail = verdict.evidence ? `hard bounce: ${verdict.evidence}` : 'hard bounce'
await suppress({ address, reason: 'bounce', detail, channel })
return { suppressed: true, note: detail }
} catch (err) {
// A failure to record the suppression must not change how the send itself is
// reported. The mail failed either way, and that is the row the log owes.
log.error('could not record a suppression', { message: err.message })
return { suppressed: false, note: `hard bounce, not recorded: ${err.message}` }
}
}
module.exports = {
hashAddress,
maskAddress,
isSuppressed,
suppress,
unsuppress,
considerFailure,
REASONS,
}