ENGAGEMENT.md Phase 9, closing gap G16. Two mechanisms decide that somebody in a
rule's audience does not get the mail, and they sit at deliberately different
points in the pipeline.
`engagement_suppressions` is checked at DELIVERY: an outbox row can sit through a
rule's `delay_seconds` grace window and an address can bounce inside it, so the
only correct check is the one taken immediately before the transport call — which
is also what produces the `status='suppressed'` row with no transport call at all.
The Phase 1b verification gate is applied at ENQUEUE, through a new optional
`registerDeliveryChannel({ eligible })` that only `email` declares. Filtering the
shared audience would have silenced the wrong sink: a rule spanning email and
in-app must still put an item in an unverified user's inbox. The excluded counts
reach `summary.ineligible` and the admin reach preview, which until now reported
an audience size that was never the number of people who would be mailed.
`bounceClassify.js` is the only thing that may write a `bounce` row, and it is
deliberately NOT `mailer.PERMANENT_CODES`. That set answers "is retrying
pointless?" and contains EAUTH and 554 — an auth failure and a relay-wide policy
refusal, neither of which is a fact about the recipient. Reusing it would mean one
stale SMTP password suppressing every address the worker touched, silently. The
classifier reads the RFC 3463 enhanced status first, falls back to a phrase match
only past a veto list and only for 550/551/553, and does not suppress anything it
is unsure about.
Scope is engagement rules only: resets, invites, verification and the contact form
still attempt, matching the posture passwordReset.controller.js already stated.
Found on the live rig, against a real MariaDB and a real SMTP conversation: a hard
bounce was being recorded as `failed`, so the Send Log's "Bounced" filter — a
status `engagement_sends` has carried since §4.5 — matched nothing and always
would have. It is now its own outcome; the outbox row stays `failed`, since that
ENUM has no `bounced` and a bounced row is one that finished unsuccessfully.
`address_masked` is this phase's one addition to §4.5's DDL. A hash-only table
cannot be operated — an operator cannot tell three typos from a whole domain
refusing mail — and the domain survives while the local part is destroyed, so the
column can never be read back as an address book.
- schema: `engagement_suppressions` (+ `address_masked`, `created_by`)
- `GET/POST/DELETE /api/v1/admin/engagement/suppressions`, and Admin → Engagement
→ Suppressions, the only way out of the list
- `sendNotification` returns `smtp: { code, responseCode, response }`
- 26 new tests; swagger, routes manifest and guards regenerated
Docs: RunicGateway/docs#191.
Co-Authored-By: Claude <noreply@anthropic.com>
161 lines
6.9 KiB
JavaScript
161 lines
6.9 KiB
JavaScript
// ── The suppression list ───────────────────────────────────────────────────
|
|
//
|
|
// ENGAGEMENT.md G16, Phase 9. Addresses this deployment has stopped mailing,
|
|
// and the two questions asked of them: "may I send to this one?" at delivery
|
|
// time, and "why did this one stop?" on the admin screen.
|
|
//
|
|
// **Scope: the engagement email channel only** (Phase 9 decision 2). A password
|
|
// reset, an invite, a verification mail and the contact form are all
|
|
// user-INITIATED and still attempt, exactly as they still attempt to an
|
|
// unverified address (`passwordReset.controller.js`). The posture is the same one
|
|
// that file already states: a background system's opinion about an address must
|
|
// not be able to lock somebody out of their own account. One reset to a dead
|
|
// mailbox is not a reputation problem; a rule mailing three thousand people every
|
|
// week is, and that is what this list guards.
|
|
//
|
|
// **The table holds a hash and a mask, never an address.** The hash is what
|
|
// correlates a bounce back to an `engagement_sends` row (Phase 6 was already
|
|
// writing `address_hash` on every outcome for this). The mask —
|
|
// `d***@example.com` — is Phase 9's one addition to §4.5's DDL and exists because
|
|
// a screen of sha256 digests cannot be operated: an operator has to be able to
|
|
// see that a whole domain is refusing mail, and to find the person who fixed
|
|
// their mailbox and let them back in. The local part is DESTROYED rather than
|
|
// shortened, so the column cannot be turned back into an address book.
|
|
//
|
|
// **Nothing here writes a suppression from "the send failed".** What may write
|
|
// one is `bounceClassify.classify`, which is a much narrower question — see that
|
|
// file's header for why reusing `mailer.PERMANENT_CODES` would have suppressed
|
|
// every address the moment an SMTP password went stale.
|
|
|
|
const crypto = require('crypto')
|
|
|
|
const db = require('../model/engagement/engagementSuppressions.db')
|
|
const bounceClassify = require('./bounceClassify')
|
|
const log = require('../utils/logger')('engagement')
|
|
|
|
const REASONS = ['bounce', 'complaint', 'manual', 'unverified']
|
|
|
|
/**
|
|
* The key an address is stored under.
|
|
*
|
|
* Lower-cased first, and that matters more here than anywhere else in the
|
|
* subsystem: a bounce reported for `Darrow@example.com` has to find the row
|
|
* written for `darrow@example.com`, and a hash of two spellings is two rows that
|
|
* never meet. RFC 5321 says the local part is technically case-sensitive; no
|
|
* relay anybody deploys treats it that way.
|
|
*/
|
|
const hashAddress = (address) =>
|
|
crypto.createHash('sha256').update(String(address).trim().toLowerCase()).digest('hex')
|
|
|
|
/**
|
|
* `darrow@example.com` → `d***@example.com`. Null for anything that is not an
|
|
* address.
|
|
*
|
|
* The domain survives intact because domain-level patterns are the signal an
|
|
* operator is actually looking for — "everything to this company is bouncing" is
|
|
* a different problem from three people mistyping their own address, and only the
|
|
* domain distinguishes them.
|
|
*
|
|
* The first character of the local part survives only when there are at least
|
|
* three, which is not fussiness: for a two-letter local part, one revealed
|
|
* character plus the domain is most of the address.
|
|
*/
|
|
function maskAddress(address) {
|
|
const s = String(address || '').trim()
|
|
const at = s.lastIndexOf('@')
|
|
if (at <= 0 || at === s.length - 1) return null
|
|
const local = s.slice(0, at)
|
|
const domain = s.slice(at + 1).toLowerCase()
|
|
const head = local.length >= 3 ? local[0].toLowerCase() : ''
|
|
return `${head}***@${domain}`.slice(0, 190)
|
|
}
|
|
|
|
/** Is this address suppressed on this channel? */
|
|
async function isSuppressed(address, channel = 'email') {
|
|
if (!address) return null
|
|
try {
|
|
return await db.get(hashAddress(address), channel)
|
|
} catch (err) {
|
|
// Fail OPEN, and the direction is deliberate. A database that cannot answer
|
|
// "is this suppressed" must not stop the deployment's mail; the failure mode
|
|
// it would otherwise produce is total silence with a clean send log, which is
|
|
// exactly G22's shape. Mailing one dead address during an outage is the
|
|
// cheaper mistake.
|
|
log.error('suppression check failed; sending anyway', { message: err.message })
|
|
return null
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Suppress an address. Returns true when this call created the row.
|
|
*
|
|
* `reason` is validated rather than trusted: it is an ENUM in the schema, so an
|
|
* unknown value is a 500 from the driver at the worst possible moment (inside a
|
|
* failure handler), and the callers include an admin route.
|
|
*/
|
|
async function suppress({ address, reason, detail = null, channel = 'email', createdBy = null }) {
|
|
if (!address) return false
|
|
if (!REASONS.includes(reason)) throw new Error(`suppress: unknown reason "${reason}"`)
|
|
const created = await db.add({
|
|
address_hash: hashAddress(address),
|
|
address_masked: maskAddress(address),
|
|
channel,
|
|
reason,
|
|
detail,
|
|
created_by: createdBy,
|
|
})
|
|
if (created) {
|
|
// Masked, never the address — the same rule every other log line in this
|
|
// subsystem follows. It is logged at all because an address dropping off the
|
|
// mailing list is the kind of change an operator finds out about weeks later
|
|
// otherwise.
|
|
log.info('address suppressed', { address: maskAddress(address), reason, channel })
|
|
}
|
|
return created
|
|
}
|
|
|
|
/** Un-suppress. Returns true when a row was removed. */
|
|
async function unsuppress(address, channel = 'email') {
|
|
if (!address) return false
|
|
const removed = await db.remove(hashAddress(address), channel)
|
|
if (removed) log.info('suppression lifted', { address: maskAddress(address), channel })
|
|
return removed
|
|
}
|
|
|
|
/**
|
|
* Consider a failed send for suppression, and say what was decided.
|
|
*
|
|
* The seam between a delivery failure and this list, and the only one — nothing
|
|
* else in the codebase writes a `bounce` row. Called from `emailChannel.deliver`
|
|
* with the error the transport threw.
|
|
*
|
|
* @returns {Promise<{suppressed: boolean, note: string}>} `note` goes into the
|
|
* send log's detail, on both outcomes.
|
|
*/
|
|
async function considerFailure({ address, error, channel = 'email' }) {
|
|
const verdict = bounceClassify.classify(error)
|
|
if (!verdict.suppress) {
|
|
return { suppressed: false, note: `not suppressed (${verdict.reason})` }
|
|
}
|
|
try {
|
|
const detail = verdict.evidence ? `hard bounce: ${verdict.evidence}` : 'hard bounce'
|
|
await suppress({ address, reason: 'bounce', detail, channel })
|
|
return { suppressed: true, note: detail }
|
|
} catch (err) {
|
|
// A failure to record the suppression must not change how the send itself is
|
|
// reported. The mail failed either way, and that is the row the log owes.
|
|
log.error('could not record a suppression', { message: err.message })
|
|
return { suppressed: false, note: `hard bounce, not recorded: ${err.message}` }
|
|
}
|
|
}
|
|
|
|
module.exports = {
|
|
hashAddress,
|
|
maskAddress,
|
|
isSuppressed,
|
|
suppress,
|
|
unsuppress,
|
|
considerFailure,
|
|
REASONS,
|
|
}
|