feat(engagement): the rules engine, cooldowns and outbox (engagement Phase 4a)
Phase 4 of docs/website/ENGAGEMENT.md, split 4a/4b at the org lead's direction. This is 4a: the engine, server only, with no HTTP surface at all. A fired trigger now produces outbox rows and send-log entries; Admin - Engagement - Rules and the segment composition UI are 4b. Five tables (rules, audience segments, cooldowns, outbox, sends), the sweep worker, audience resolution, condition evaluation, the grace window and its cancellation, and the save-path validation 4b's form will call. engagementEmit's Phase 2 log line becomes the engine call. Two settled questions this phase was blocked on: Q2 (multi-instance) - neither SKIP LOCKED nor documented single-instance: the outbox claims each row with a compare-and-set into the 'sending' state the ENUM already carried. It makes the outbox safe for two instances, not the deployment. Q4 (admin surface) - its own top-level nav group, built in 4b. Two defects in the plan's own section 4, both found by building it: The global UNIQUE(dedupe_key) was data loss. A dedupe key names the EVENT, and one event is one row per (rule, user, channel) - so a fifty-person audience would have had one row admitted and forty-nine silently ignored. Scoped. Section 4.1's single INSERT ... ON DUPLICATE KEY UPDATE cooldown claim always passes against this codebase's pool: the mariadb connector defaults foundRows:true, so a no-op update reports affectedRows 1 rather than 0. It is two statements now, with the interval guard in a WHERE clause. The second defect is why there is a second test file. The stubbed suite was green against the broken claim, because a stub can only agree with whoever wrote it; engagementEngineSql.test.js runs the raw statements against a real MariaDB and skips when there is none. Verification: 43 new tests green in engagementEngine.test.js, 12 more against MariaDB 11.8, and the whole path exercised end to end against a live database - per-subject cooldowns, conditions, the CAS claim, the send log's honest failure detail, and dormancy on uninstall. The three pre-existing Windows-only CRLF failures in the generated-artifact tests are unchanged from clean edge. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
78
server/src/model/engagement/engagementCooldowns.db.js
Normal file
78
server/src/model/engagement/engagementCooldowns.db.js
Normal file
@@ -0,0 +1,78 @@
|
||||
const { query } = require('../../utils/db')
|
||||
|
||||
/**
|
||||
* Claim a fire for (rule, user, subject), or refuse it because the pair is still
|
||||
* cooling. ENGAGEMENT.md §4.1.
|
||||
*
|
||||
* **Two statements, each of which is its own atomic decision** - and it is worth
|
||||
* saying why it is not the single `INSERT ... ON DUPLICATE KEY UPDATE` §4.1
|
||||
* describes, because that version was written, tested green against an in-memory
|
||||
* stub, and disproved by the first run against a real MariaDB.
|
||||
*
|
||||
* The one-statement form reads its answer out of `affectedRows`, on the usual
|
||||
* contract: 1 for an insert, 2 for an update that changed something, and 0 for a
|
||||
* duplicate key whose update changed nothing - that 0 being "the guard failed, so
|
||||
* this pair is still cooling". **The mariadb Node connector sets `foundRows: true`
|
||||
* by default**, which makes `affectedRows` report rows MATCHED rather than rows
|
||||
* CHANGED, and `utils/db.js` does not override it. Under that pool the no-op case
|
||||
* returns 1, indistinguishable from a fresh insert: every cooldown would have
|
||||
* passed, always, and nothing in a stubbed test could have noticed.
|
||||
*
|
||||
* So the guard moves into a WHERE clause, where a row either matches or does not
|
||||
* and `foundRows` has nothing to fold together:
|
||||
*
|
||||
* 1. UPDATE the row, guarded on the interval. `affectedRows = 1` means this
|
||||
* caller moved it and owns the fire.
|
||||
* 2. If that matched nothing, the row either does not exist yet or is still
|
||||
* cooling. `INSERT IGNORE` separates the two: 1 means we inserted the first
|
||||
* fire, 0 means the row was there and step 1 already said it is cooling.
|
||||
*
|
||||
* It is still race-free, and each race resolves the right way:
|
||||
* - two concurrent first fires: neither UPDATEs, both INSERT IGNORE, exactly
|
||||
* one gets 1 (the primary key decides). The loser is treated as cooling.
|
||||
* - two concurrent fires after expiry: the row is locked by the first UPDATE,
|
||||
* and the second re-evaluates its guard against the committed row - which now
|
||||
* holds `now`, so it fails and is refused.
|
||||
*
|
||||
* `cooldown_seconds = 0` always passes, which is the documented meaning of a rule
|
||||
* with no cooldown: the guard becomes `last_fired_at <= now`, and it is.
|
||||
*/
|
||||
async function claim(ruleId, userId, subjectKey, cooldownSeconds, now = new Date()) {
|
||||
const moved = await query(
|
||||
`UPDATE engagement_cooldowns
|
||||
SET last_fired_at = ?, fire_count = fire_count + 1
|
||||
WHERE rule_id = ? AND user_id = ? AND subject_key = ?
|
||||
AND last_fired_at <= ? - INTERVAL ? SECOND`,
|
||||
[now, ruleId, userId, subjectKey, now, cooldownSeconds],
|
||||
)
|
||||
if (Number(moved?.affectedRows || 0) === 1) return true
|
||||
|
||||
const inserted = await query(
|
||||
`INSERT IGNORE INTO engagement_cooldowns (rule_id, user_id, subject_key, last_fired_at, fire_count)
|
||||
VALUES (?, ?, ?, ?, 1)`,
|
||||
[ruleId, userId, subjectKey, now],
|
||||
)
|
||||
return Number(inserted?.affectedRows || 0) === 1
|
||||
}
|
||||
|
||||
const get = async (ruleId, userId, subjectKey) => {
|
||||
const [row] = await query(
|
||||
'SELECT * FROM engagement_cooldowns WHERE rule_id = ? AND user_id = ? AND subject_key = ?',
|
||||
[ruleId, userId, subjectKey],
|
||||
)
|
||||
return row || null
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop cooldown rows older than `olderThan`.
|
||||
*
|
||||
* `idx_engc_sweep (last_fired_at)` exists for this: the table is written on every
|
||||
* fire and read once per fire, so without a prune it is the unbounded-growth
|
||||
* failure `teamActivityPrune` was written for. A dropped row means the next fire
|
||||
* is treated as a first fire, which is correct as long as the retention window is
|
||||
* longer than the longest configured cooldown - the caller's job, not this one's.
|
||||
*/
|
||||
const prune = (olderThan) =>
|
||||
query('DELETE FROM engagement_cooldowns WHERE last_fired_at < ?', [olderThan])
|
||||
|
||||
module.exports = { claim, get, prune }
|
||||
158
server/src/model/engagement/engagementOutbox.db.js
Normal file
158
server/src/model/engagement/engagementOutbox.db.js
Normal file
@@ -0,0 +1,158 @@
|
||||
const { query } = require('../../utils/db')
|
||||
const { parseJson } = require('./engagementRules.db')
|
||||
|
||||
const hydrate = (row) => row && { ...row, payload: parseJson(row.payload, {}) }
|
||||
|
||||
/**
|
||||
* Enqueue one (rule, user, channel) row, idempotently.
|
||||
*
|
||||
* `INSERT IGNORE` rather than a plain INSERT, because `uq_engo_dedupe` is the
|
||||
* replay guard (§4.2a): the sidecar feed is at-least-once and a reconnect
|
||||
* backfills, so the same event arriving twice must produce one row and not two
|
||||
* mails. IGNORE turns that into a silent no-op, which is what a replay should be.
|
||||
*
|
||||
* Returns the new id, or null when the row already existed. A null is a
|
||||
* SUCCESSFUL duplicate, not a failure - the caller counts it as such.
|
||||
*
|
||||
* A NULL dedupe_key never collides (multiple NULLs are legal under a UNIQUE
|
||||
* index), so an emit that carries no key always enqueues. That is the right
|
||||
* default: dedupe is something the emitter opts into by naming a key, and core
|
||||
* cannot invent one that means anything.
|
||||
*/
|
||||
async function enqueue(row) {
|
||||
const result = await query(
|
||||
`INSERT IGNORE INTO engagement_outbox
|
||||
(rule_id, trigger_id, user_id, channel, subject_key, payload, dedupe_key, due_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
[
|
||||
row.rule_id,
|
||||
row.trigger_id,
|
||||
row.user_id,
|
||||
row.channel,
|
||||
row.subject_key || '',
|
||||
JSON.stringify(row.payload || {}),
|
||||
row.dedupe_key ?? null,
|
||||
row.due_at,
|
||||
],
|
||||
)
|
||||
return Number(result?.affectedRows || 0) === 1 ? result.insertId : null
|
||||
}
|
||||
|
||||
/**
|
||||
* Rows that are due. `idx_engo_due (status, due_at)` is this query.
|
||||
*
|
||||
* It selects rather than claims - claiming is `claim()` below, one row at a
|
||||
* time - so two instances sweeping at once both see the same candidates and then
|
||||
* disagree, harmlessly, about which of them owns each.
|
||||
*/
|
||||
const findDue = async (now, limit = 100) =>
|
||||
(
|
||||
await query(
|
||||
"SELECT * FROM engagement_outbox WHERE status = 'scheduled' AND due_at <= ? ORDER BY due_at, id LIMIT ?",
|
||||
[now, limit],
|
||||
)
|
||||
).map(hydrate)
|
||||
|
||||
/**
|
||||
* Take ownership of one due row: a compare-and-set from 'scheduled' to 'sending'.
|
||||
*
|
||||
* **This is §7.1 Q2's answer** (settled by the org lead 2026-08-29, over
|
||||
* `SELECT ... FOR UPDATE SKIP LOCKED`). The winner is whoever the server reports
|
||||
* `affectedRows = 1` to; every other sweeper gets 0 and moves on. No explicit
|
||||
* transaction, no MariaDB version floor, and it uses a status the ENUM already
|
||||
* carried for exactly this.
|
||||
*
|
||||
* What it makes safe is the OUTBOX and only the outbox. `announceWorker`,
|
||||
* `teamDigestWorker`, `teamForumUploadSweep` and `teamActivityPrune` are all
|
||||
* still written for a single instance, so this does not make the deployment
|
||||
* multi-instance - it makes the one table that will carry mail ready for the day
|
||||
* it is, which is cheap now and expensive after mail has doubled once.
|
||||
*/
|
||||
async function claim(id) {
|
||||
const result = await query(
|
||||
`UPDATE engagement_outbox
|
||||
SET status = 'sending', attempts = attempts + 1
|
||||
WHERE id = ? AND status = 'scheduled'`,
|
||||
[id],
|
||||
)
|
||||
return Number(result?.affectedRows || 0) === 1
|
||||
}
|
||||
|
||||
/**
|
||||
* Release a claimed row back to 'scheduled' with a later `due_at` - a transient
|
||||
* failure that should be retried. The mirror of announceJobs' backoff.
|
||||
*/
|
||||
const reschedule = (id, dueAt, error) =>
|
||||
query(
|
||||
"UPDATE engagement_outbox SET status = 'scheduled', due_at = ?, last_error = ? WHERE id = ? AND status = 'sending'",
|
||||
[dueAt, error ? String(error).slice(0, 2000) : null, id],
|
||||
)
|
||||
|
||||
/** A terminal outcome: 'sent', 'failed' or 'suppressed'. */
|
||||
const finish = (id, status, error) =>
|
||||
query(
|
||||
`UPDATE engagement_outbox
|
||||
SET status = ?, last_error = ?, sent_at = IF(? = 'sent', NOW(), sent_at)
|
||||
WHERE id = ?`,
|
||||
[status, error ? String(error).slice(0, 2000) : null, status, id],
|
||||
)
|
||||
|
||||
/**
|
||||
* Cancel every still-scheduled row for a (rule, subject) - the point of the
|
||||
* grace window (§4.2a). `userId` narrows it to one recipient when the resolving
|
||||
* event names one; a resolving event with no owner cancels for everyone the
|
||||
* original event was queued for, which is the house-repaired case.
|
||||
*
|
||||
* Only 'scheduled' rows are touched: a row already claimed into 'sending' is
|
||||
* somebody's in-flight send and cancelling it would leave two workers writing
|
||||
* one row's outcome.
|
||||
*/
|
||||
async function cancel(ruleId, subjectKey, userId = null) {
|
||||
const params = [ruleId, subjectKey]
|
||||
let sql = "UPDATE engagement_outbox SET status = 'cancelled' WHERE rule_id = ? AND subject_key = ? AND status = 'scheduled'"
|
||||
if (userId !== null && userId !== undefined) {
|
||||
sql += ' AND user_id = ?'
|
||||
params.push(userId)
|
||||
}
|
||||
const result = await query(sql, params)
|
||||
return Number(result?.affectedRows || 0)
|
||||
}
|
||||
|
||||
/**
|
||||
* Recover rows stranded in 'sending' by a crash between the claim and the
|
||||
* outcome.
|
||||
*
|
||||
* Without this the CAS claim leaks: the claiming process died, no other sweeper
|
||||
* will ever match `status = 'scheduled'`, and the row sits in 'sending' forever.
|
||||
* `updated_at` is the clock (it is ON UPDATE CURRENT_TIMESTAMP, so the claim
|
||||
* stamped it), and the window has to be comfortably longer than the slowest
|
||||
* legitimate send or this reclaims rows that are merely slow.
|
||||
*/
|
||||
const reclaimStale = (before) =>
|
||||
query(
|
||||
"UPDATE engagement_outbox SET status = 'scheduled' WHERE status = 'sending' AND updated_at < ?",
|
||||
[before],
|
||||
)
|
||||
|
||||
const getById = async (id) => {
|
||||
const [row] = await query('SELECT * FROM engagement_outbox WHERE id = ?', [id])
|
||||
return hydrate(row)
|
||||
}
|
||||
|
||||
/** Admin/read surfaces (Phase 4b) and tests. */
|
||||
const listForRule = async (ruleId, limit = 100) =>
|
||||
(
|
||||
await query('SELECT * FROM engagement_outbox WHERE rule_id = ? ORDER BY id DESC LIMIT ?', [ruleId, limit])
|
||||
).map(hydrate)
|
||||
|
||||
module.exports = {
|
||||
enqueue,
|
||||
findDue,
|
||||
claim,
|
||||
reschedule,
|
||||
finish,
|
||||
cancel,
|
||||
reclaimStale,
|
||||
getById,
|
||||
listForRule,
|
||||
}
|
||||
125
server/src/model/engagement/engagementRecipients.db.js
Normal file
125
server/src/model/engagement/engagementRecipients.db.js
Normal file
@@ -0,0 +1,125 @@
|
||||
const { query } = require('../../utils/db')
|
||||
|
||||
// A bound on every "resolve an audience" query. `authenticated` on a large
|
||||
// deployment is the whole user table, and the engine turns each id into an
|
||||
// outbox row - so the read that feeds it has to have a ceiling of its own. The
|
||||
// per-rule hourly cap (§7.1 Q3) is the operator-facing limit; this is the one
|
||||
// that keeps a single emit from loading a hundred thousand rows into memory.
|
||||
const MAX_AUDIENCE = 5000
|
||||
|
||||
const ids = (rows) => rows.map((r) => Number(r.id)).filter((n) => Number.isInteger(n) && n > 0)
|
||||
|
||||
const marks = (list) => list.map(() => '?').join(', ')
|
||||
|
||||
/**
|
||||
* Every active user. The `authenticated` audience - and `everyone`, which has no
|
||||
* distinct meaning here: a signed-out visitor has no address, no device and no
|
||||
* inbox, so the widest set the engine can actually deliver to is this one. The
|
||||
* ceiling lattice still distinguishes them (a trigger ceilinged `everyone`
|
||||
* permits an `authenticated` rule and not the reverse); only the resolution
|
||||
* coincides.
|
||||
*
|
||||
* `status = 'active'` on every query in this file: a banned or disabled account
|
||||
* is refused at login, and mailing it engagement content would be the one
|
||||
* surface that did not get the message.
|
||||
*/
|
||||
const active = async (limit = MAX_AUDIENCE) =>
|
||||
ids(await query("SELECT id FROM users WHERE status = 'active' ORDER BY id LIMIT ?", [limit]))
|
||||
|
||||
/** The `staff` audience. Roles come from `ceilings.STAFF_CEILING_ROLES`. */
|
||||
const staff = async (roles, limit = MAX_AUDIENCE) => {
|
||||
if (!roles.length) return []
|
||||
return ids(
|
||||
await query(
|
||||
`SELECT id FROM users WHERE status = 'active' AND role IN (${marks(roles)}) ORDER BY id LIMIT ?`,
|
||||
[...roles, limit],
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* The `subscribers` audience: active users who have opted into this id on at
|
||||
* least one channel.
|
||||
*
|
||||
* "Opted in" is the EFFECTIVE mode, not the stored one, and that is why this is
|
||||
* not simply `WHERE mode <> 'off'`. A row exists only where a user said
|
||||
* something; absence means the channel's `defaultMode` (§3.1). All three of
|
||||
* core's channels default 'off' today, so the second half of the WHERE matches
|
||||
* nobody - but writing it means the day a channel ships with a non-off default,
|
||||
* this audience is already right rather than silently excluding everyone who
|
||||
* never opened the preferences screen.
|
||||
*
|
||||
* `defaultOnChannels` is the caller's list of channels whose defaultMode is not
|
||||
* 'off'; it comes from the channel registry, so the default lives in exactly one
|
||||
* place here too.
|
||||
*/
|
||||
const subscribers = async (streamId, defaultOnChannels = [], limit = MAX_AUDIENCE) => {
|
||||
const optedIn = `EXISTS (
|
||||
SELECT 1 FROM notification_channel_prefs p
|
||||
WHERE p.user_id = u.id AND p.stream_id = ? AND p.mode <> 'off')`
|
||||
|
||||
if (!defaultOnChannels.length) {
|
||||
return ids(
|
||||
await query(
|
||||
`SELECT u.id FROM users u WHERE u.status = 'active' AND ${optedIn} ORDER BY u.id LIMIT ?`,
|
||||
[streamId, limit],
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
// "At least one default-on channel has no row for this user" - counted rather
|
||||
// than NOT EXISTS, because NOT EXISTS would mean "none of them has a row".
|
||||
const defaulted = `(
|
||||
SELECT COUNT(*) FROM notification_channel_prefs p2
|
||||
WHERE p2.user_id = u.id AND p2.stream_id = ? AND p2.channel IN (${marks(defaultOnChannels)})
|
||||
) < ?`
|
||||
|
||||
return ids(
|
||||
await query(
|
||||
`SELECT u.id FROM users u
|
||||
WHERE u.status = 'active' AND (${optedIn} OR ${defaulted})
|
||||
ORDER BY u.id LIMIT ?`,
|
||||
[streamId, streamId, ...defaultOnChannels, defaultOnChannels.length, limit],
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Narrow a set of user ids to the active ones.
|
||||
*
|
||||
* Every audience that does NOT come from a query in this file goes through here:
|
||||
* `owner` is a single id off the event envelope, and a module-declared audience
|
||||
* (§5.1a) is a list of ids a module's own resolver produced. Neither has any
|
||||
* notion of account status, and a module must not be able to mail a banned
|
||||
* account by returning its id.
|
||||
*/
|
||||
const filterActive = async (userIds) => {
|
||||
const wanted = [...new Set(userIds.map(Number).filter((n) => Number.isInteger(n) && n > 0))]
|
||||
if (!wanted.length) return []
|
||||
const capped = wanted.slice(0, MAX_AUDIENCE)
|
||||
return ids(
|
||||
await query(
|
||||
`SELECT id FROM users WHERE status = 'active' AND id IN (${marks(capped)}) ORDER BY id`,
|
||||
capped,
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* The stored mode for one (id, channel) across a set of users, as a Map.
|
||||
*
|
||||
* The caller applies the channel's `defaultMode` to anyone missing from the map,
|
||||
* which keeps the defaulting in the one place §3.1 put it. Returning stored rows
|
||||
* rather than a decision is what makes that possible.
|
||||
*/
|
||||
const storedModes = async (userIds, streamId, channel) => {
|
||||
if (!userIds.length) return new Map()
|
||||
const rows = await query(
|
||||
`SELECT user_id, mode FROM notification_channel_prefs
|
||||
WHERE stream_id = ? AND channel = ? AND user_id IN (${marks(userIds)})`,
|
||||
[streamId, channel, ...userIds],
|
||||
)
|
||||
return new Map(rows.map((r) => [Number(r.user_id), r.mode]))
|
||||
}
|
||||
|
||||
module.exports = { active, staff, subscribers, filterActive, storedModes, MAX_AUDIENCE }
|
||||
127
server/src/model/engagement/engagementRules.db.js
Normal file
127
server/src/model/engagement/engagementRules.db.js
Normal file
@@ -0,0 +1,127 @@
|
||||
const { query } = require('../../utils/db')
|
||||
|
||||
// JSON columns come back from the driver already parsed on some MariaDB/driver
|
||||
// combinations and as a string on others (it depends on whether the column is a
|
||||
// real JSON type or the LONGTEXT + CHECK alias MariaDB implements it as). Every
|
||||
// read below goes through this, so no caller has to know which it got.
|
||||
function parseJson(value, fallback) {
|
||||
if (value === null || value === undefined) return fallback
|
||||
if (typeof value !== 'string') return value
|
||||
try {
|
||||
return JSON.parse(value)
|
||||
} catch {
|
||||
return fallback
|
||||
}
|
||||
}
|
||||
|
||||
const hydrate = (row) =>
|
||||
row && {
|
||||
...row,
|
||||
enabled: Boolean(row.enabled),
|
||||
channels: parseJson(row.channels, []),
|
||||
template_keys: parseJson(row.template_keys, {}),
|
||||
conditions: parseJson(row.conditions, null),
|
||||
cancel_on: parseJson(row.cancel_on, []),
|
||||
}
|
||||
|
||||
const list = async () =>
|
||||
(await query('SELECT * FROM engagement_rules ORDER BY trigger_id, name, id')).map(hydrate)
|
||||
|
||||
const getById = async (id) => {
|
||||
const [row] = await query('SELECT * FROM engagement_rules WHERE id = ?', [id])
|
||||
return hydrate(row)
|
||||
}
|
||||
|
||||
/**
|
||||
* Every ENABLED rule for one trigger. The engine's hot path: one indexed read
|
||||
* per emit, and `idx_engr_trigger (trigger_id, enabled)` is exactly this query.
|
||||
*/
|
||||
const enabledForTrigger = async (triggerId) =>
|
||||
(await query('SELECT * FROM engagement_rules WHERE trigger_id = ? AND enabled = 1', [triggerId])).map(hydrate)
|
||||
|
||||
/**
|
||||
* Every enabled rule that names `triggerId` in its `cancel_on`.
|
||||
*
|
||||
* A JSON_CONTAINS rather than a scan: `cancel_on` is a small array on a small
|
||||
* table, but this runs on EVERY emit — including the overwhelming majority that
|
||||
* cancel nothing — so it must not be a full table read of the rule set.
|
||||
*/
|
||||
const enabledCancelledBy = async (triggerId) =>
|
||||
(
|
||||
await query(
|
||||
"SELECT * FROM engagement_rules WHERE enabled = 1 AND cancel_on IS NOT NULL AND JSON_CONTAINS(cancel_on, JSON_QUOTE(?))",
|
||||
[triggerId],
|
||||
)
|
||||
).map(hydrate)
|
||||
|
||||
const insert = async (rule) => {
|
||||
const result = await query(
|
||||
`INSERT INTO engagement_rules
|
||||
(trigger_id, name, enabled, audience, audience_segment_id, max_sends_per_hour,
|
||||
channels, template_keys, conditions, cooldown_seconds, delay_seconds, cancel_on, updated_by)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
[
|
||||
rule.trigger_id,
|
||||
rule.name,
|
||||
rule.enabled ? 1 : 0,
|
||||
rule.audience,
|
||||
rule.audience_segment_id,
|
||||
rule.max_sends_per_hour,
|
||||
JSON.stringify(rule.channels),
|
||||
JSON.stringify(rule.template_keys),
|
||||
rule.conditions === null ? null : JSON.stringify(rule.conditions),
|
||||
rule.cooldown_seconds,
|
||||
rule.delay_seconds,
|
||||
JSON.stringify(rule.cancel_on || []),
|
||||
rule.updated_by,
|
||||
],
|
||||
)
|
||||
return result.insertId
|
||||
}
|
||||
|
||||
const update = (id, rule) =>
|
||||
query(
|
||||
`UPDATE engagement_rules
|
||||
SET name = ?, enabled = ?, audience = ?, audience_segment_id = ?, max_sends_per_hour = ?,
|
||||
channels = ?, template_keys = ?, conditions = ?, cooldown_seconds = ?,
|
||||
delay_seconds = ?, cancel_on = ?, updated_by = ?
|
||||
WHERE id = ?`,
|
||||
[
|
||||
rule.name,
|
||||
rule.enabled ? 1 : 0,
|
||||
rule.audience,
|
||||
rule.audience_segment_id,
|
||||
rule.max_sends_per_hour,
|
||||
JSON.stringify(rule.channels),
|
||||
JSON.stringify(rule.template_keys),
|
||||
rule.conditions === null ? null : JSON.stringify(rule.conditions),
|
||||
rule.cooldown_seconds,
|
||||
rule.delay_seconds,
|
||||
JSON.stringify(rule.cancel_on || []),
|
||||
rule.updated_by,
|
||||
id,
|
||||
],
|
||||
)
|
||||
|
||||
const remove = (id) => query('DELETE FROM engagement_rules WHERE id = ?', [id])
|
||||
|
||||
/** Does any rule still point at this segment? The check before a segment delete. */
|
||||
const countUsingSegment = async (segmentId) => {
|
||||
const [row] = await query(
|
||||
'SELECT COUNT(*) AS n FROM engagement_rules WHERE audience_segment_id = ?',
|
||||
[segmentId],
|
||||
)
|
||||
return Number(row?.n || 0)
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
list,
|
||||
getById,
|
||||
enabledForTrigger,
|
||||
enabledCancelledBy,
|
||||
insert,
|
||||
update,
|
||||
remove,
|
||||
countUsingSegment,
|
||||
parseJson,
|
||||
}
|
||||
225
server/src/model/engagement/engagementRules.model.js
Normal file
225
server/src/model/engagement/engagementRules.model.js
Normal file
@@ -0,0 +1,225 @@
|
||||
// ── Engagement rules — the save path ───────────────────────────────────────
|
||||
//
|
||||
// ENGAGEMENT.md §4.5 / §7.1 Q3, Phase 4a. A rule is **operator-editable data**,
|
||||
// not code, and that was a deliberate choice with a condition attached: it is
|
||||
// safe to choose only because `enabled` defaults to 0 and every rule carries a
|
||||
// hard per-hour send ceiling. Both of those live in this file's validation, not
|
||||
// in the screen that calls it - Phase 4b builds a form over this, and a rule that
|
||||
// arrives by any other route (a restore, a fixture, a future import) gets the
|
||||
// same answer.
|
||||
//
|
||||
// **Every check here is a boundary, not a convenience.** The rule editor will
|
||||
// re-implement some of them for the sake of a good error message, and that
|
||||
// second copy is expected to drift - so this one is the one that decides.
|
||||
//
|
||||
// The check with teeth is the ceiling (G24): an operator may narrow a rule's
|
||||
// audience as much as they like and may never widen it past what the trigger
|
||||
// declared. `ceilings.permits` is that arithmetic, `segments.validate` derives
|
||||
// it for a composed audience, and the engine re-runs the same check at SEND
|
||||
// time in case a module upgrade narrowed the declaration underneath a saved rule.
|
||||
|
||||
const db = require('./engagementRules.db')
|
||||
const segmentsDb = require('./engagementSegments.db')
|
||||
const registries = require('../../modules/registries')
|
||||
const ceilings = require('../../modules/ceilings')
|
||||
const channels = require('../../engagement/channels')
|
||||
const conditions = require('../../engagement/conditions')
|
||||
|
||||
// A day. Longer than this and "cooldown" is really "send once", which a rule
|
||||
// expresses by being disabled rather than by a decade-long interval.
|
||||
const MAX_COOLDOWN_SECONDS = 86_400
|
||||
// The grace window (§4.2a). A delay longer than a day outlives the thing it is
|
||||
// about - and, more practically, a queue row that sits for a week is a row whose
|
||||
// payload no longer describes the world.
|
||||
const MAX_DELAY_SECONDS = 86_400
|
||||
// The upper bound on the operator-set hourly ceiling. It is not "unlimited by
|
||||
// another name": the number exists so that a misconfiguration is a bad hour
|
||||
// rather than an unbounded one, and a ceiling nobody can raise past a bound is
|
||||
// what makes rules-as-data safe (§7.1 Q3).
|
||||
const MAX_SENDS_PER_HOUR = 10_000
|
||||
|
||||
const isPlainObject = (v) => v !== null && typeof v === 'object' && !Array.isArray(v)
|
||||
|
||||
/**
|
||||
* Validate a rule against the registries and the lattice.
|
||||
*
|
||||
* Returns `{ ok: true, rule }` with a normalised row ready for insert/update, or
|
||||
* `{ ok: false, errors }` listing every problem.
|
||||
*
|
||||
* `triggerId` may name a trigger nobody currently registers ONLY on an update of
|
||||
* an existing rule - a dormant rule must stay editable (its module can come
|
||||
* back), and refusing to save it would make an uninstall destructive after the
|
||||
* fact. A NEW rule must name a live trigger, because there is nothing to
|
||||
* preserve and a typo should be caught now.
|
||||
*/
|
||||
async function validate(input, { existing = null } = {}) {
|
||||
const errors = []
|
||||
const raw = isPlainObject(input) ? input : {}
|
||||
|
||||
const triggerId = typeof raw.triggerId === 'string' ? raw.triggerId : existing?.trigger_id
|
||||
const declaration = triggerId ? registries.eventTrigger(triggerId) : null
|
||||
if (!triggerId) errors.push('triggerId is required')
|
||||
else if (!declaration && !existing) errors.push(`no trigger "${triggerId}" is registered`)
|
||||
|
||||
const name = typeof raw.name === 'string' ? raw.name.trim() : ''
|
||||
if (!name) errors.push('name is required')
|
||||
else if (name.length > 160) errors.push('name is longer than 160 characters')
|
||||
|
||||
// Channels are stored as data and checked against the registry, so a rule
|
||||
// cannot name a sink that does not exist. Phase 4b's form offers the registered
|
||||
// set; this is what makes that an affordance rather than the rule.
|
||||
const wanted = Array.isArray(raw.channels) ? [...new Set(raw.channels)] : []
|
||||
if (!wanted.length) errors.push('at least one channel is required')
|
||||
for (const c of wanted) if (!channels.has(c)) errors.push(`no channel "${c}" is registered`)
|
||||
|
||||
// `template_keys` is { channel: templateKey }. Phase 5 owns templates, so the
|
||||
// KEYS are checked for shape and not for existence - a rule may legitimately
|
||||
// name a template that has not been authored yet, and Phase 5's editor is where
|
||||
// that becomes resolvable.
|
||||
const templateKeys = {}
|
||||
if (raw.templateKeys !== undefined && !isPlainObject(raw.templateKeys)) {
|
||||
errors.push('templateKeys must be an object of { channel: templateKey }')
|
||||
} else {
|
||||
for (const [channel, key] of Object.entries(raw.templateKeys || {})) {
|
||||
if (!wanted.includes(channel)) {
|
||||
errors.push(`templateKeys names "${channel}", which is not one of this rule's channels`)
|
||||
continue
|
||||
}
|
||||
if (typeof key !== 'string' || !/^[a-z0-9][a-z0-9-]{0,63}$/.test(key)) {
|
||||
errors.push(`templateKeys.${channel} is not a valid template key`)
|
||||
continue
|
||||
}
|
||||
templateKeys[channel] = key
|
||||
}
|
||||
}
|
||||
|
||||
const numbers = [
|
||||
['cooldownSeconds', 'cooldown_seconds', MAX_COOLDOWN_SECONDS, 0],
|
||||
['delaySeconds', 'delay_seconds', MAX_DELAY_SECONDS, 0],
|
||||
['maxSendsPerHour', 'max_sends_per_hour', MAX_SENDS_PER_HOUR, 1],
|
||||
]
|
||||
const scalars = {}
|
||||
for (const [key, column, max, min] of numbers) {
|
||||
const supplied = raw[key]
|
||||
const fallback = existing ? existing[column] : column === 'max_sends_per_hour' ? 100 : 0
|
||||
const value = supplied === undefined || supplied === null ? fallback : Number(supplied)
|
||||
if (!Number.isInteger(value) || value < min || value > max) {
|
||||
errors.push(`${key} must be an integer between ${min} and ${max}`)
|
||||
} else scalars[column] = value
|
||||
}
|
||||
|
||||
// `cancel_on` names trigger ids, and they are NOT checked for registration for
|
||||
// the dormancy reason (§7.3): a resolving event whose module is temporarily
|
||||
// absent should stop cancelling, not make the rule unsaveable.
|
||||
const cancelOn = Array.isArray(raw.cancelOn) ? [...new Set(raw.cancelOn.filter((t) => typeof t === 'string'))] : []
|
||||
if (cancelOn.length && !scalars.delay_seconds) {
|
||||
// Not an error - it is a rule that will never cancel anything, because there
|
||||
// is no window in which to do it. Worth saying out loud rather than silently
|
||||
// accepting a setting that cannot take effect.
|
||||
errors.push('cancelOn has no effect without a delaySeconds grace window')
|
||||
}
|
||||
|
||||
const checked = conditions.validate(declaration, raw.conditions === undefined ? existing?.conditions : raw.conditions)
|
||||
if (!checked.ok) errors.push(...checked.errors)
|
||||
|
||||
// ── The audience, and the one check that is a security boundary ──────────
|
||||
let audience = typeof raw.audience === 'string' ? raw.audience : existing?.audience || declaration?.audience
|
||||
let segmentId = raw.audienceSegmentId === undefined ? existing?.audience_segment_id ?? null : raw.audienceSegmentId
|
||||
segmentId = segmentId === null || segmentId === '' ? null : Number(segmentId)
|
||||
|
||||
let effectiveCeiling = null
|
||||
if (segmentId !== null) {
|
||||
if (!Number.isInteger(segmentId)) errors.push('audienceSegmentId must be an integer')
|
||||
else {
|
||||
const segment = await segmentsDb.getById(segmentId)
|
||||
if (!segment) errors.push(`no audience segment ${segmentId} exists`)
|
||||
else {
|
||||
// The segment's STORED ceiling, derived when it was saved by
|
||||
// `segments.validate` from the narrowest audience it contains. A rule
|
||||
// pointing at a segment takes that as its reach; the `audience` column
|
||||
// is retained for display and is not what the engine resolves.
|
||||
effectiveCeiling = segment.ceiling
|
||||
audience = segment.ceiling
|
||||
}
|
||||
}
|
||||
} else if (!ceilings.isCeiling(audience)) {
|
||||
errors.push(`audience must be one of ${ceilings.CEILINGS.join(', ')}`)
|
||||
} else {
|
||||
effectiveCeiling = audience
|
||||
}
|
||||
|
||||
if (declaration && effectiveCeiling && !ceilings.permits(declaration.ceiling, effectiveCeiling)) {
|
||||
errors.push(
|
||||
`audience "${effectiveCeiling}" is wider than trigger "${triggerId}" permits (ceiling "${declaration.ceiling}")`,
|
||||
)
|
||||
}
|
||||
|
||||
if (errors.length) return { ok: false, errors }
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
rule: {
|
||||
trigger_id: triggerId,
|
||||
name,
|
||||
enabled: raw.enabled === undefined ? Boolean(existing?.enabled) : Boolean(raw.enabled),
|
||||
audience,
|
||||
audience_segment_id: segmentId,
|
||||
max_sends_per_hour: scalars.max_sends_per_hour,
|
||||
channels: wanted,
|
||||
template_keys: templateKeys,
|
||||
conditions: checked.conditions,
|
||||
cooldown_seconds: scalars.cooldown_seconds,
|
||||
delay_seconds: scalars.delay_seconds,
|
||||
cancel_on: cancelOn,
|
||||
updated_by: Number.isInteger(raw.updatedBy) ? raw.updatedBy : null,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
async function create(input) {
|
||||
const checked = await validate(input)
|
||||
if (!checked.ok) return checked
|
||||
const id = await db.insert(checked.rule)
|
||||
return { ok: true, rule: await db.getById(id) }
|
||||
}
|
||||
|
||||
async function update(id, input) {
|
||||
const existing = await db.getById(id)
|
||||
if (!existing) return { ok: false, errors: [`no rule ${id} exists`], notFound: true }
|
||||
const checked = await validate(input, { existing })
|
||||
if (!checked.ok) return checked
|
||||
await db.update(id, checked.rule)
|
||||
return { ok: true, rule: await db.getById(id) }
|
||||
}
|
||||
|
||||
/**
|
||||
* List every rule, each annotated with whether it can currently fire.
|
||||
*
|
||||
* Dormancy is computed rather than stored (§7.3): a rule whose trigger or
|
||||
* segment is not registered right now is listed, flagged, and left alone. The
|
||||
* alternative - deleting or disabling it on uninstall - destroys an operator's
|
||||
* configuration on the strength of a module being temporarily absent.
|
||||
*/
|
||||
async function listAnnotated() {
|
||||
const rows = await db.list()
|
||||
const segments = new Map((await segmentsDb.list()).map((s) => [s.id, s]))
|
||||
return rows.map((rule) => {
|
||||
const reasons = []
|
||||
if (!registries.eventTrigger(rule.trigger_id)) reasons.push(`trigger "${rule.trigger_id}" is not registered`)
|
||||
if (rule.audience_segment_id && !segments.has(rule.audience_segment_id)) {
|
||||
reasons.push('its audience segment no longer exists')
|
||||
}
|
||||
for (const c of rule.channels || []) if (!channels.has(c)) reasons.push(`channel "${c}" is not registered`)
|
||||
return { ...rule, dormant: reasons.length > 0, dormantReasons: reasons }
|
||||
})
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
validate,
|
||||
create,
|
||||
update,
|
||||
listAnnotated,
|
||||
MAX_COOLDOWN_SECONDS,
|
||||
MAX_DELAY_SECONDS,
|
||||
MAX_SENDS_PER_HOUR,
|
||||
}
|
||||
37
server/src/model/engagement/engagementSegments.db.js
Normal file
37
server/src/model/engagement/engagementSegments.db.js
Normal file
@@ -0,0 +1,37 @@
|
||||
const { query } = require('../../utils/db')
|
||||
const { parseJson } = require('./engagementRules.db')
|
||||
|
||||
const hydrate = (row) => row && { ...row, expression: parseJson(row.expression, null) }
|
||||
|
||||
const list = async () =>
|
||||
(await query('SELECT * FROM engagement_audience_segments ORDER BY name, id')).map(hydrate)
|
||||
|
||||
const getById = async (id) => {
|
||||
const [row] = await query('SELECT * FROM engagement_audience_segments WHERE id = ?', [id])
|
||||
return hydrate(row)
|
||||
}
|
||||
|
||||
/**
|
||||
* `ceiling` is written by the caller from `segments.deriveCeiling`, never taken
|
||||
* from an operator. It is a stored column rather than a runtime computation so
|
||||
* an audit can read what a rule was ALLOWED to reach without re-resolving it,
|
||||
* and so a module that later widens its own audience's ceiling cannot
|
||||
* retroactively widen a segment that was saved under the old one.
|
||||
*/
|
||||
const insert = async (segment) => {
|
||||
const result = await query(
|
||||
'INSERT INTO engagement_audience_segments (name, expression, ceiling, updated_by) VALUES (?, ?, ?, ?)',
|
||||
[segment.name, JSON.stringify(segment.expression), segment.ceiling, segment.updated_by ?? null],
|
||||
)
|
||||
return result.insertId
|
||||
}
|
||||
|
||||
const update = (id, segment) =>
|
||||
query(
|
||||
'UPDATE engagement_audience_segments SET name = ?, expression = ?, ceiling = ?, updated_by = ? WHERE id = ?',
|
||||
[segment.name, JSON.stringify(segment.expression), segment.ceiling, segment.updated_by ?? null, id],
|
||||
)
|
||||
|
||||
const remove = (id) => query('DELETE FROM engagement_audience_segments WHERE id = ?', [id])
|
||||
|
||||
module.exports = { list, getById, insert, update, remove }
|
||||
87
server/src/model/engagement/engagementSegments.model.js
Normal file
87
server/src/model/engagement/engagementSegments.model.js
Normal file
@@ -0,0 +1,87 @@
|
||||
// ── Audience segments — the save path ──────────────────────────────────────
|
||||
//
|
||||
// ENGAGEMENT.md §5.1a, Phase 4a. The thin model over `segments.js`: it validates,
|
||||
// derives the ceiling, and writes. The composition UI is Phase 4b's; this is what
|
||||
// it will call, and what any other route in must go through.
|
||||
//
|
||||
// The `ceiling` column is never taken from the caller. It is derived from the
|
||||
// expression by `segments.validate` as the narrowest ceiling in the tree, and
|
||||
// stored so an audit can read what a rule was ALLOWED to reach without
|
||||
// re-resolving it.
|
||||
|
||||
const db = require('./engagementSegments.db')
|
||||
const rulesDb = require('./engagementRules.db')
|
||||
const registries = require('../../modules/registries')
|
||||
const segments = require('../../engagement/segments')
|
||||
|
||||
async function save(input, { id = null } = {}) {
|
||||
const errors = []
|
||||
const name = typeof input?.name === 'string' ? input.name.trim() : ''
|
||||
if (!name) errors.push('name is required')
|
||||
else if (name.length > 160) errors.push('name is longer than 160 characters')
|
||||
|
||||
const checked = segments.validate(input?.expression)
|
||||
if (!checked.ok) errors.push(...checked.errors)
|
||||
if (errors.length) return { ok: false, errors }
|
||||
|
||||
const row = {
|
||||
name,
|
||||
expression: checked.expression,
|
||||
ceiling: checked.ceiling,
|
||||
updated_by: Number.isInteger(input?.updatedBy) ? input.updatedBy : null,
|
||||
}
|
||||
|
||||
if (id) {
|
||||
const existing = await db.getById(id)
|
||||
if (!existing) return { ok: false, errors: [`no segment ${id} exists`], notFound: true }
|
||||
await db.update(id, row)
|
||||
return { ok: true, segment: await db.getById(id) }
|
||||
}
|
||||
const newId = await db.insert(row)
|
||||
return { ok: true, segment: await db.getById(newId) }
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete a segment, refusing while a rule still points at it.
|
||||
*
|
||||
* There is deliberately no foreign key doing this (schema.sql): the database
|
||||
* options are CASCADE, which would delete an operator's rules, and SET NULL,
|
||||
* which would silently fall the rule back to its plain `audience` column and mail
|
||||
* a DIFFERENT set of people. Refusing here, with the count, is the third option
|
||||
* and the only safe one.
|
||||
*/
|
||||
async function remove(id) {
|
||||
const inUse = await rulesDb.countUsingSegment(id)
|
||||
if (inUse > 0) {
|
||||
return {
|
||||
ok: false,
|
||||
inUse,
|
||||
errors: [`${inUse} rule${inUse === 1 ? '' : 's'} still use this segment`],
|
||||
}
|
||||
}
|
||||
await db.remove(id)
|
||||
return { ok: true }
|
||||
}
|
||||
|
||||
/**
|
||||
* Every segment, each annotated with whether it can currently resolve.
|
||||
*
|
||||
* A segment naming an audience whose module has been uninstalled is DORMANT, not
|
||||
* broken: it is listed, it resolves to nobody, and it starts working again when
|
||||
* the module comes back (§5.1a rule 4).
|
||||
*/
|
||||
async function listAnnotated() {
|
||||
const rows = await db.list()
|
||||
return rows.map((segment) => {
|
||||
const missing = []
|
||||
const walk = (node) => {
|
||||
if (!node || typeof node !== 'object') return
|
||||
if (node.op) (node.nodes || []).forEach(walk)
|
||||
else if (!registries.audience(node.audienceId)) missing.push(node.audienceId)
|
||||
}
|
||||
walk(segment.expression)
|
||||
return { ...segment, dormant: missing.length > 0, missingAudiences: [...new Set(missing)] }
|
||||
})
|
||||
}
|
||||
|
||||
module.exports = { save, remove, listAnnotated }
|
||||
73
server/src/model/engagement/engagementSends.db.js
Normal file
73
server/src/model/engagement/engagementSends.db.js
Normal file
@@ -0,0 +1,73 @@
|
||||
const { query } = require('../../utils/db')
|
||||
|
||||
/**
|
||||
* Record one attempt's outcome. G15: "did user X get the mail?" has never been
|
||||
* answerable on this deployment, and this row is the answer.
|
||||
*
|
||||
* `address_hash` is a sha256 the CALLER computes, never an address. The log has
|
||||
* to correlate a bounce back to a recipient (Phase 9) and it must not become a
|
||||
* second address book, and a hash does the first without the second.
|
||||
*/
|
||||
const record = async (entry) => {
|
||||
const result = await query(
|
||||
`INSERT INTO engagement_sends
|
||||
(outbox_id, rule_id, trigger_id, user_id, channel, transport, address_hash, status, detail)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
|
||||
[
|
||||
entry.outbox_id ?? null,
|
||||
entry.rule_id ?? null,
|
||||
entry.trigger_id,
|
||||
entry.user_id ?? null,
|
||||
entry.channel,
|
||||
entry.transport ?? null,
|
||||
entry.address_hash ?? null,
|
||||
entry.status,
|
||||
entry.detail ? String(entry.detail).slice(0, 500) : null,
|
||||
],
|
||||
)
|
||||
return result.insertId
|
||||
}
|
||||
|
||||
/**
|
||||
* How many sends this rule has made in the last hour - the count the per-rule
|
||||
* ceiling (§7.1 Q3) is enforced against.
|
||||
*
|
||||
* It counts 'sent' only. A refusal that never left the building (`suppressed`)
|
||||
* and an attempt that failed are not sends, and counting them would let a broken
|
||||
* transport silently consume a rule's whole hourly budget and mute it.
|
||||
*
|
||||
* `idx_engs_rule_window (rule_id, created_at)` exists for this: it runs once per
|
||||
* rule per event, so it has to be an index range scan.
|
||||
*/
|
||||
const countSentSince = async (ruleId, since) => {
|
||||
const [row] = await query(
|
||||
"SELECT COUNT(*) AS n FROM engagement_sends WHERE rule_id = ? AND status = 'sent' AND created_at >= ?",
|
||||
[ruleId, since],
|
||||
)
|
||||
return Number(row?.n || 0)
|
||||
}
|
||||
|
||||
/** The admin send log (Phase 4b/5), newest first. */
|
||||
const list = ({ triggerId = null, userId = null, ruleId = null, limit = 100, offset = 0 } = {}) => {
|
||||
const where = []
|
||||
const params = []
|
||||
if (triggerId) {
|
||||
where.push('trigger_id = ?')
|
||||
params.push(triggerId)
|
||||
}
|
||||
if (userId) {
|
||||
where.push('user_id = ?')
|
||||
params.push(userId)
|
||||
}
|
||||
if (ruleId) {
|
||||
where.push('rule_id = ?')
|
||||
params.push(ruleId)
|
||||
}
|
||||
const clause = where.length ? `WHERE ${where.join(' AND ')}` : ''
|
||||
return query(
|
||||
`SELECT * FROM engagement_sends ${clause} ORDER BY id DESC LIMIT ? OFFSET ?`,
|
||||
[...params, limit, offset],
|
||||
)
|
||||
}
|
||||
|
||||
module.exports = { record, countSentSince, list }
|
||||
Reference in New Issue
Block a user