feat(engagement): deliverability — suppression, bounces and the verification gate
All checks were successful
PR Checks / client-build (pull_request) Successful in 36s
PR Checks / bot-tests (pull_request) Successful in 36s
PR Checks / server-tests (pull_request) Successful in 5m12s

ENGAGEMENT.md Phase 9, closing gap G16. Two mechanisms decide that somebody in a
rule's audience does not get the mail, and they sit at deliberately different
points in the pipeline.

`engagement_suppressions` is checked at DELIVERY: an outbox row can sit through a
rule's `delay_seconds` grace window and an address can bounce inside it, so the
only correct check is the one taken immediately before the transport call — which
is also what produces the `status='suppressed'` row with no transport call at all.

The Phase 1b verification gate is applied at ENQUEUE, through a new optional
`registerDeliveryChannel({ eligible })` that only `email` declares. Filtering the
shared audience would have silenced the wrong sink: a rule spanning email and
in-app must still put an item in an unverified user's inbox. The excluded counts
reach `summary.ineligible` and the admin reach preview, which until now reported
an audience size that was never the number of people who would be mailed.

`bounceClassify.js` is the only thing that may write a `bounce` row, and it is
deliberately NOT `mailer.PERMANENT_CODES`. That set answers "is retrying
pointless?" and contains EAUTH and 554 — an auth failure and a relay-wide policy
refusal, neither of which is a fact about the recipient. Reusing it would mean one
stale SMTP password suppressing every address the worker touched, silently. The
classifier reads the RFC 3463 enhanced status first, falls back to a phrase match
only past a veto list and only for 550/551/553, and does not suppress anything it
is unsure about.

Scope is engagement rules only: resets, invites, verification and the contact form
still attempt, matching the posture passwordReset.controller.js already stated.

Found on the live rig, against a real MariaDB and a real SMTP conversation: a hard
bounce was being recorded as `failed`, so the Send Log's "Bounced" filter — a
status `engagement_sends` has carried since §4.5 — matched nothing and always
would have. It is now its own outcome; the outbox row stays `failed`, since that
ENUM has no `bounced` and a bounced row is one that finished unsuccessfully.

`address_masked` is this phase's one addition to §4.5's DDL. A hash-only table
cannot be operated — an operator cannot tell three typos from a whole domain
refusing mail — and the domain survives while the local part is destroyed, so the
column can never be read back as an address book.

- schema: `engagement_suppressions` (+ `address_masked`, `created_by`)
- `GET/POST/DELETE /api/v1/admin/engagement/suppressions`, and Admin → Engagement
  → Suppressions, the only way out of the list
- `sendNotification` returns `smtp: { code, responseCode, response }`
- 26 new tests; swagger, routes manifest and guards regenerated

Docs: RunicGateway/docs#191.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-08-31 10:47:34 -05:00
parent 87c4e71025
commit c208543044
22 changed files with 2210 additions and 37 deletions

View File

@@ -0,0 +1,216 @@
// ── Which send failures are facts about the RECIPIENT ───────────────────────
//
// ENGAGEMENT.md Phase 9. The suppression list's whole value is that an address on
// it is genuinely undeliverable; the moment it fills with addresses that were
// fine, an operator learns to ignore it and it may as well not exist. This file
// is the one place that judgement is made.
//
// **It is deliberately NOT `mailer.PERMANENT_CODES`, and reusing that set would
// have been a mass-suppression bug.** That set answers "is retrying pointless?"
// and holds `EAUTH` and `554` alongside `550` — an authentication failure and a
// relay-wide policy refusal. Both are permanent and neither says anything about
// the person: one wrong SMTP password would suppress every address the outbox
// worker touched before anybody noticed the mail had stopped. "Do not retry" and
// "this mailbox does not exist" are different questions, and this file only
// answers the second.
//
// **The primary signal is the enhanced status code (RFC 3463), not the reply
// code.** `550` alone is the catch-all every refusal arrives as; `5.1.1` means
// one specific thing — no such mailbox. Every relay worth configuring emits an
// enhanced code, so it is read first and, when present, decides on its own.
//
// **The fallback is narrow on purpose.** Without an enhanced code a phrase match
// is all that is left, and phrase matching is how a classifier quietly starts
// suppressing everything. So it applies only after the reply code has already
// narrowed the failure to the recipient address — 550, 551 and 553 are RFC 5321's
// recipient-address codes — and only for phrases that cannot mean anything else,
// with a veto list checked first. `552` (storage exceeded) and `554` (transaction
// failed) are excluded from even that: a full mailbox gets emptied, and a generic
// transaction failure is generic.
//
// Anything this file is unsure about is NOT suppressed. The cost of a false
// negative is mailing a dead address again next month; the cost of a false
// positive is a person who silently stops hearing from the deployment and has no
// way to find out.
// RFC 3463 subject.detail pairs that mean "this address will not accept mail,
// today or ever". Kept as strings because `5.1.10` and `5.1.1` are different
// codes and numeric parsing loses that.
const PERMANENT_RECIPIENT = new Set([
'1.1', // bad destination mailbox address — no such user
'1.2', // bad destination system address — the domain does not take mail
'1.3', // bad destination mailbox address syntax
'1.6', // mailbox has moved, no forwarding address
'1.10', // recipient address has a null MX (RFC 7505)
'2.1', // mailbox disabled, not accepting messages
])
// Enhanced subjects that are permanent but are NOT about the recipient. Listed
// rather than merely omitted, because each is a plausible-looking 5.x.y that a
// later edit would otherwise be tempted to add:
// 2.2 — mailbox full. Permanent-coded by some relays, emptied by every user.
// 7.x — policy. Our sending reputation, our SPF, our content; the recipient is
// the one party it is not about.
// 3.x — the destination MAIL SYSTEM is full or refusing. Not the mailbox.
// 5.x — protocol failure. A bug at one end or the other.
const NEVER_RECIPIENT_SUBJECTS = new Set(['3', '5', '7'])
// RFC 5321 reply codes that name the recipient address specifically. 554 is
// absent deliberately: "transaction failed" is what a relay reaches for when it
// does not want to say why, and it is the commonest shape of a content or policy
// rejection.
const RECIPIENT_REPLY_CODES = new Set([550, 551, 553])
// Phrases that only ever mean "no such mailbox", checked only once a reply code
// above has established the failure is about the address. Each is a substring of
// a real refusal from a widely deployed MTA (Postfix, Exim, Exchange, Google,
// Microsoft 365).
const NO_SUCH_MAILBOX = [
'user unknown',
'unknown user',
'no such user',
'no such recipient',
'unknown recipient',
'invalid recipient',
'recipient address rejected',
'recipient not found',
'address does not exist',
'does not exist',
'mailbox unavailable',
'mailbox not found',
'no mailbox',
'user does not exist',
'address rejected',
]
// Phrases that appear alongside the ones above and mean the opposite, checked
// FIRST. "Mailbox unavailable" is a substring of the sentence a relay sends when
// a mailbox is merely full, so a substring match with no veto list would read a
// temporary condition as a dead address.
const NOT_A_DEAD_MAILBOX = [
'full',
'quota',
'storage',
'temporar',
'try again',
'greylist',
'rate limit',
'too many',
'spam',
'blocked',
'blacklist',
'blocklist',
'reputation',
'policy',
'authentication',
'not authorized',
]
/**
* The enhanced status code in an SMTP response, as `{ class, subject, detail }`,
* or null.
*
* Anchored to the start of the line rather than searched for anywhere in it: a
* bounce that quotes another server's answer ("...said: 550 5.1.1...") contains
* two, and the one that matters is the one this relay just gave us. A free search
* finds whichever comes first, which is not the same thing.
*/
function parseEnhanced(response) {
if (!response) return null
const m = /^\s*(\d{3})[\s-]+(\d)\.(\d{1,3})\.(\d{1,3})\b/.exec(String(response))
if (!m) return null
return { class: m[2], subject: m[3], detail: m[4] }
}
/** The three-digit reply code, off the error object or out of the response text. */
function replyCode(err) {
const direct = Number(err && err.responseCode)
if (Number.isInteger(direct) && direct >= 400 && direct <= 599) return direct
const m = /^\s*(\d{3})\b/.exec(String((err && err.response) || ''))
return m ? Number(m[1]) : null
}
const lower = (s) => String(s || '').toLowerCase()
/**
* Should this send failure suppress the address?
*
* @param {object} err the error a transport's send threw, or an object carrying
* the `responseCode` / `response` / `code` lifted off one
* @returns {{ suppress: boolean, reason: string, evidence: string|null }}
*
* `reason` is populated on a refusal too, and that is not decoration: it becomes
* the send log's `detail`, so "not suppressed: 554 does not name the recipient
* address" is the line that stops somebody re-deriving this decision from an
* unexplained non-event six months from now.
*/
function classify(err) {
const e = err || {}
const response = e.response || e.message || ''
const enhanced = parseEnhanced(response)
const code = replyCode(e)
// No reply code at all means the failure happened before or outside the SMTP
// transaction: the connection, the credentials, the socket. Never the
// recipient. `EAUTH` lands here, which is the whole reason this file exists.
if (!code) {
return {
suppress: false,
reason: `no SMTP reply code (${e.code || 'transport failure'}); not a recipient failure`,
evidence: null,
}
}
if (code < 500) {
return { suppress: false, reason: `${code} is a temporary failure`, evidence: null }
}
if (enhanced) {
const pair = `${enhanced.subject}.${enhanced.detail}`
if (enhanced.class !== '5') {
return { suppress: false, reason: `enhanced status ${enhanced.class}.${pair} is not permanent`, evidence: null }
}
if (PERMANENT_RECIPIENT.has(pair)) {
return { suppress: true, reason: 'bounce', evidence: `5.${pair}` }
}
if (NEVER_RECIPIENT_SUBJECTS.has(enhanced.subject)) {
return {
suppress: false,
reason: `5.${pair} is about the server or our standing with it, not the address`,
evidence: `5.${pair}`,
}
}
// A permanent 5.x.y this file has no opinion on. Unknown means no.
return {
suppress: false,
reason: `5.${pair} is not a known recipient failure`,
evidence: `5.${pair}`,
}
}
// No enhanced code: the narrow fallback.
if (!RECIPIENT_REPLY_CODES.has(code)) {
return { suppress: false, reason: `${code} does not name the recipient address`, evidence: null }
}
const text = lower(response)
const veto = NOT_A_DEAD_MAILBOX.find((p) => text.includes(p))
if (veto) {
return { suppress: false, reason: `${code}, but the response says "${veto}"`, evidence: null }
}
const hit = NO_SUCH_MAILBOX.find((p) => text.includes(p))
if (hit) {
return { suppress: true, reason: 'bounce', evidence: `${code} "${hit}"` }
}
return { suppress: false, reason: `${code} with no enhanced status and no recognised reason`, evidence: null }
}
module.exports = {
classify,
parseEnhanced,
replyCode,
PERMANENT_RECIPIENT,
NEVER_RECIPIENT_SUBJECTS,
RECIPIENT_REPLY_CODES,
NO_SUCH_MAILBOX,
NOT_A_DEAD_MAILBOX,
}