feat(engagement): the rules engine, cooldowns and outbox (engagement Phase 4a)
Phase 4 of docs/website/ENGAGEMENT.md, split 4a/4b at the org lead's direction. This is 4a: the engine, server only, with no HTTP surface at all. A fired trigger now produces outbox rows and send-log entries; Admin - Engagement - Rules and the segment composition UI are 4b. Five tables (rules, audience segments, cooldowns, outbox, sends), the sweep worker, audience resolution, condition evaluation, the grace window and its cancellation, and the save-path validation 4b's form will call. engagementEmit's Phase 2 log line becomes the engine call. Two settled questions this phase was blocked on: Q2 (multi-instance) - neither SKIP LOCKED nor documented single-instance: the outbox claims each row with a compare-and-set into the 'sending' state the ENUM already carried. It makes the outbox safe for two instances, not the deployment. Q4 (admin surface) - its own top-level nav group, built in 4b. Two defects in the plan's own section 4, both found by building it: The global UNIQUE(dedupe_key) was data loss. A dedupe key names the EVENT, and one event is one row per (rule, user, channel) - so a fifty-person audience would have had one row admitted and forty-nine silently ignored. Scoped. Section 4.1's single INSERT ... ON DUPLICATE KEY UPDATE cooldown claim always passes against this codebase's pool: the mariadb connector defaults foundRows:true, so a no-op update reports affectedRows 1 rather than 0. It is two statements now, with the interval guard in a WHERE clause. The second defect is why there is a second test file. The stubbed suite was green against the broken claim, because a stub can only agree with whoever wrote it; engagementEngineSql.test.js runs the raw statements against a real MariaDB and skips when there is none. Verification: 43 new tests green in engagementEngine.test.js, 12 more against MariaDB 11.8, and the whole path exercised end to end against a live database - per-subject cooldowns, conditions, the CAS claim, the send log's honest failure detail, and dormancy on uninstall. The three pre-existing Windows-only CRLF failures in the generated-artifact tests are unchanged from clean edge. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
251
server/src/engagement/conditions.js
Normal file
251
server/src/engagement/conditions.js
Normal file
@@ -0,0 +1,251 @@
|
||||
// ── Rule conditions — a predicate over a trigger's DECLARED variables ───────
|
||||
//
|
||||
// ENGAGEMENT.md §4.5, Phase 4a. `engagement_rules.conditions` is the half of a
|
||||
// rule that decides *whether* this particular firing is interesting: "only when
|
||||
// decayStatus is IDOC", "only for threads in this Team". Without it every rule is
|
||||
// all-or-nothing per trigger, and an operator's only way to narrow is to ask a
|
||||
// module author for a second trigger.
|
||||
//
|
||||
// **It is validated against the declaration, not against a payload.** A condition
|
||||
// naming a variable the trigger does not declare is refused at SAVE, with the
|
||||
// variable named, for the same reason §4.3 gives the template editor: a predicate
|
||||
// that silently reads `undefined` is a rule that silently never fires (or always
|
||||
// does), and the day you find out is the day the mail did not go.
|
||||
//
|
||||
// **The grammar is small and closed on purpose.** No arbitrary expressions, no
|
||||
// arithmetic, no regex. An operator composes and/or/not over comparisons of one
|
||||
// declared variable against a literal, and every operator here is one a rule
|
||||
// editor can render as a dropdown. Anything that needs more than this is asking
|
||||
// for a condition the module should have declared as a variable.
|
||||
//
|
||||
// Nothing in this file reaches the database or the network.
|
||||
|
||||
const registries = require('../modules/registries')
|
||||
|
||||
// Comparison operators, grouped by what they may be applied to. The grouping is
|
||||
// the whole of the type check: `gt` on a boolean and `startsWith` on an int are
|
||||
// both refused at save rather than quietly answering false forever.
|
||||
const OPERATORS = {
|
||||
eq: { label: 'is', types: ['string', 'int', 'float', 'boolean', 'datetime', 'url'], arity: 1 },
|
||||
ne: { label: 'is not', types: ['string', 'int', 'float', 'boolean', 'datetime', 'url'], arity: 1 },
|
||||
in: { label: 'is one of', types: ['string', 'int', 'float', 'url'], arity: 'list' },
|
||||
nin: { label: 'is none of', types: ['string', 'int', 'float', 'url'], arity: 'list' },
|
||||
gt: { label: 'is greater than', types: ['int', 'float', 'datetime'], arity: 1 },
|
||||
gte: { label: 'is at least', types: ['int', 'float', 'datetime'], arity: 1 },
|
||||
lt: { label: 'is less than', types: ['int', 'float', 'datetime'], arity: 1 },
|
||||
lte: { label: 'is at most', types: ['int', 'float', 'datetime'], arity: 1 },
|
||||
contains: { label: 'contains', types: ['string', 'url'], arity: 1 },
|
||||
startsWith: { label: 'starts with', types: ['string', 'url'], arity: 1 },
|
||||
// The one operator that takes no value: "the emit carried this variable at
|
||||
// all". It is the honest way to write a rule about an OPTIONAL variable, and
|
||||
// without it `ne` would have to double as a presence test and get it wrong
|
||||
// (an absent variable is not "not equal to X"; it is absent).
|
||||
present: { label: 'is present', types: ['string', 'int', 'float', 'boolean', 'datetime', 'url'], arity: 0 },
|
||||
absent: { label: 'is absent', types: ['string', 'int', 'float', 'boolean', 'datetime', 'url'], arity: 0 },
|
||||
}
|
||||
|
||||
const BOOLEAN_OPS = ['and', 'or', 'not']
|
||||
|
||||
// A list literal an operator may type. Bounded because it is stored in a JSON
|
||||
// column an admin can write, and an unbounded IN list is an unbounded predicate
|
||||
// evaluated on every event.
|
||||
const MAX_LIST = 50
|
||||
// Depth of the and/or/not tree. Three levels is more nesting than any rule
|
||||
// editor should offer; the bound is here so a hand-written JSON body cannot
|
||||
// recurse this evaluator into a stack overflow on the emit path.
|
||||
const MAX_DEPTH = 5
|
||||
|
||||
const isPlainObject = (v) => v !== null && typeof v === 'object' && !Array.isArray(v)
|
||||
|
||||
/**
|
||||
* Check one literal against the declared type of the variable it is compared to.
|
||||
*
|
||||
* `datetime` accepts anything `Date` parses and is normalised to an ISO string,
|
||||
* which is what `engagementEmit.coerce` does to the payload side — so both sides
|
||||
* of every comparison are the same representation of a moment, and a lexical
|
||||
* `<` on two ISO strings is a chronological one.
|
||||
*/
|
||||
function checkLiteral(type, raw) {
|
||||
switch (type) {
|
||||
case 'string':
|
||||
case 'url':
|
||||
return typeof raw === 'string' ? { value: raw } : { error: 'expected a string' }
|
||||
case 'int':
|
||||
return Number.isInteger(raw) ? { value: raw } : { error: 'expected an integer' }
|
||||
case 'float':
|
||||
return typeof raw === 'number' && Number.isFinite(raw)
|
||||
? { value: raw }
|
||||
: { error: 'expected a finite number' }
|
||||
case 'boolean':
|
||||
return typeof raw === 'boolean' ? { value: raw } : { error: 'expected a boolean' }
|
||||
case 'datetime': {
|
||||
const d = raw instanceof Date ? raw : new Date(raw)
|
||||
if (Number.isNaN(d.getTime())) return { error: 'expected a date' }
|
||||
return { value: d.toISOString() }
|
||||
}
|
||||
default:
|
||||
return { error: `unsupported type "${type}"` }
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate a condition tree against a trigger declaration.
|
||||
*
|
||||
* Returns `{ ok: true, conditions }` with a NEW normalised tree — literals
|
||||
* coerced, unknown keys dropped — or `{ ok: false, errors }` listing every
|
||||
* problem rather than the first, the posture `validatePayload` takes and for the
|
||||
* same reason: an operator fixing one clause at a time is an operator making six
|
||||
* round trips through a form.
|
||||
*
|
||||
* `null` and `undefined` are valid and mean "no conditions" — a rule that fires
|
||||
* on every occurrence of its trigger, which is the common case.
|
||||
*/
|
||||
function validate(declaration, raw) {
|
||||
const errors = []
|
||||
const variables = new Map((declaration?.variables || []).map((v) => [v.name, v]))
|
||||
|
||||
function walk(node, depth, path) {
|
||||
if (depth > MAX_DEPTH) {
|
||||
errors.push(`${path}: nested deeper than ${MAX_DEPTH}`)
|
||||
return null
|
||||
}
|
||||
if (!isPlainObject(node)) {
|
||||
errors.push(`${path}: expected an object`)
|
||||
return null
|
||||
}
|
||||
|
||||
if (BOOLEAN_OPS.includes(node.op)) {
|
||||
// `not` takes exactly one node; `and`/`or` take a list. Both are written
|
||||
// as `nodes` so a client walks one shape.
|
||||
const raws = Array.isArray(node.nodes) ? node.nodes : []
|
||||
if (!raws.length) {
|
||||
errors.push(`${path}: "${node.op}" has no nodes`)
|
||||
return null
|
||||
}
|
||||
if (node.op === 'not' && raws.length !== 1) {
|
||||
errors.push(`${path}: "not" takes exactly one node`)
|
||||
return null
|
||||
}
|
||||
const nodes = raws.map((child, i) => walk(child, depth + 1, `${path}.nodes[${i}]`)).filter(Boolean)
|
||||
return nodes.length === raws.length ? { op: node.op, nodes } : null
|
||||
}
|
||||
|
||||
if (node.op !== undefined) {
|
||||
errors.push(`${path}: unknown operator "${node.op}"`)
|
||||
return null
|
||||
}
|
||||
|
||||
// A leaf: { variable, cmp, value }.
|
||||
const variable = variables.get(node.variable)
|
||||
if (!variable) {
|
||||
errors.push(`${path}: "${node.variable}" is not a variable of "${declaration?.id}"`)
|
||||
return null
|
||||
}
|
||||
const operator = OPERATORS[node.cmp]
|
||||
if (!operator) {
|
||||
errors.push(`${path}: unknown comparison "${node.cmp}"`)
|
||||
return null
|
||||
}
|
||||
if (!operator.types.includes(variable.type)) {
|
||||
errors.push(`${path}: "${node.cmp}" cannot be applied to a ${variable.type}`)
|
||||
return null
|
||||
}
|
||||
|
||||
if (operator.arity === 0) return { variable: variable.name, cmp: node.cmp }
|
||||
|
||||
if (operator.arity === 'list') {
|
||||
if (!Array.isArray(node.value) || !node.value.length) {
|
||||
errors.push(`${path}: "${node.cmp}" needs a non-empty list`)
|
||||
return null
|
||||
}
|
||||
if (node.value.length > MAX_LIST) {
|
||||
errors.push(`${path}: "${node.cmp}" list is longer than ${MAX_LIST}`)
|
||||
return null
|
||||
}
|
||||
const value = []
|
||||
let bad = false
|
||||
node.value.forEach((item, i) => {
|
||||
const checked = checkLiteral(variable.type, item)
|
||||
if (checked.error) {
|
||||
errors.push(`${path}.value[${i}]: ${checked.error}`)
|
||||
bad = true
|
||||
} else value.push(checked.value)
|
||||
})
|
||||
return bad ? null : { variable: variable.name, cmp: node.cmp, value }
|
||||
}
|
||||
|
||||
const checked = checkLiteral(variable.type, node.value)
|
||||
if (checked.error) {
|
||||
errors.push(`${path}: ${checked.error}`)
|
||||
return null
|
||||
}
|
||||
return { variable: variable.name, cmp: node.cmp, value: checked.value }
|
||||
}
|
||||
|
||||
if (raw === null || raw === undefined) return { ok: true, conditions: null }
|
||||
const conditions = walk(raw, 0, 'conditions')
|
||||
return errors.length ? { ok: false, errors } : { ok: true, conditions }
|
||||
}
|
||||
|
||||
/** Compare one already-normalised leaf against a payload. */
|
||||
function evaluateLeaf(leaf, data) {
|
||||
const present = Object.prototype.hasOwnProperty.call(data, leaf.variable)
|
||||
const actual = data[leaf.variable]
|
||||
|
||||
if (leaf.cmp === 'present') return present
|
||||
if (leaf.cmp === 'absent') return !present
|
||||
// Every other comparison against an absent variable is FALSE, never true.
|
||||
// `ne` is the one that tempts otherwise — "not equal to X" reads as satisfied
|
||||
// by nothing at all — and treating it as true would make an optional variable's
|
||||
// absence fire the rule.
|
||||
if (!present) return false
|
||||
|
||||
switch (leaf.cmp) {
|
||||
case 'eq': return actual === leaf.value
|
||||
case 'ne': return actual !== leaf.value
|
||||
case 'in': return leaf.value.includes(actual)
|
||||
case 'nin': return !leaf.value.includes(actual)
|
||||
case 'gt': return actual > leaf.value
|
||||
case 'gte': return actual >= leaf.value
|
||||
case 'lt': return actual < leaf.value
|
||||
case 'lte': return actual <= leaf.value
|
||||
case 'contains': return typeof actual === 'string' && actual.includes(leaf.value)
|
||||
case 'startsWith': return typeof actual === 'string' && actual.startsWith(leaf.value)
|
||||
default: return false
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Does this event's payload satisfy the rule's conditions?
|
||||
*
|
||||
* `null` conditions are satisfied — a rule with no conditions fires on every
|
||||
* occurrence. A tree this evaluator does not recognise answers **false**, which
|
||||
* is the fail-closed direction: a stored condition that no longer parses (a rule
|
||||
* saved against an older trigger version, say) must stop the mail rather than
|
||||
* become "no conditions" and mail everyone.
|
||||
*/
|
||||
function evaluate(conditions, data = {}) {
|
||||
if (conditions === null || conditions === undefined) return true
|
||||
if (!isPlainObject(conditions)) return false
|
||||
|
||||
if (conditions.op === 'and') return (conditions.nodes || []).every((n) => evaluate(n, data))
|
||||
if (conditions.op === 'or') return (conditions.nodes || []).some((n) => evaluate(n, data))
|
||||
if (conditions.op === 'not') return !evaluate((conditions.nodes || [])[0], data)
|
||||
if (conditions.op !== undefined) return false
|
||||
|
||||
return evaluateLeaf(conditions, data)
|
||||
}
|
||||
|
||||
/**
|
||||
* The operator vocabulary a rule editor renders, with the variable types each
|
||||
* one applies to. Served with the rule surface in Phase 4b rather than hardcoded
|
||||
* in the client, on the same argument the ceiling vocabulary is served with the
|
||||
* trigger catalog: a second copy of a rule is a copy that drifts.
|
||||
*/
|
||||
const vocabulary = () =>
|
||||
Object.entries(OPERATORS).map(([cmp, o]) => ({ cmp, label: o.label, types: o.types, arity: o.arity }))
|
||||
|
||||
/** Convenience for a caller holding only a trigger id. */
|
||||
const validateFor = (triggerId, raw) => validate(registries.eventTrigger(triggerId), raw)
|
||||
|
||||
module.exports = { validate, validateFor, evaluate, vocabulary, OPERATORS, MAX_LIST, MAX_DEPTH }
|
||||
Reference in New Issue
Block a user