feat(engagement): the rules engine, cooldowns and outbox (engagement Phase 4a)
All checks were successful
PR Checks / bot-tests (pull_request) Successful in 27s
PR Checks / client-build (pull_request) Successful in 30s
PR Checks / server-tests (pull_request) Successful in 2m37s

Phase 4 of docs/website/ENGAGEMENT.md, split 4a/4b at the org lead's direction.
This is 4a: the engine, server only, with no HTTP surface at all. A fired trigger
now produces outbox rows and send-log entries; Admin - Engagement - Rules and the
segment composition UI are 4b.

Five tables (rules, audience segments, cooldowns, outbox, sends), the sweep
worker, audience resolution, condition evaluation, the grace window and its
cancellation, and the save-path validation 4b's form will call. engagementEmit's
Phase 2 log line becomes the engine call.

Two settled questions this phase was blocked on:

  Q2 (multi-instance) - neither SKIP LOCKED nor documented single-instance: the
  outbox claims each row with a compare-and-set into the 'sending' state the ENUM
  already carried. It makes the outbox safe for two instances, not the deployment.

  Q4 (admin surface) - its own top-level nav group, built in 4b.

Two defects in the plan's own section 4, both found by building it:

  The global UNIQUE(dedupe_key) was data loss. A dedupe key names the EVENT, and
  one event is one row per (rule, user, channel) - so a fifty-person audience
  would have had one row admitted and forty-nine silently ignored. Scoped.

  Section 4.1's single INSERT ... ON DUPLICATE KEY UPDATE cooldown claim always
  passes against this codebase's pool: the mariadb connector defaults
  foundRows:true, so a no-op update reports affectedRows 1 rather than 0. It is
  two statements now, with the interval guard in a WHERE clause.

The second defect is why there is a second test file. The stubbed suite was green
against the broken claim, because a stub can only agree with whoever wrote it;
engagementEngineSql.test.js runs the raw statements against a real MariaDB and
skips when there is none.

Verification: 43 new tests green in engagementEngine.test.js, 12 more against
MariaDB 11.8, and the whole path exercised end to end against a live database -
per-subject cooldowns, conditions, the CAS claim, the send log's honest failure
detail, and dormancy on uninstall. The three pre-existing Windows-only CRLF
failures in the generated-artifact tests are unchanged from clean edge.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-08-29 08:07:27 -05:00
parent 447c9113d3
commit 2079aaf667
18 changed files with 3322 additions and 11 deletions

View File

@@ -1686,3 +1686,164 @@ CREATE TABLE IF NOT EXISTS notification_channel_prefs (
-- has no digest mode to be asked into instead.
INSERT IGNORE INTO notification_channel_prefs (user_id, stream_id, channel, mode)
SELECT user_id, stream_id, 'push', 'instant' FROM notification_subscriptions;
-- ── The engagement engine (ENGAGEMENT.md §4.1, §4.2a, §4.5 — Phase 4a) ──────
--
-- Five tables and no delivery. A rule says "when this trigger fires, for these
-- people, on these channels, no more often than this"; the outbox is the queue
-- the grace window needs; the cooldown table is what makes "once per house" mean
-- once per house; and the send log is the first answer this deployment has ever
-- had to "did user X get the mail?".
--
-- Nothing here sends anything. Core seeds no rules and `enabled` defaults to 0,
-- so on a real deployment these five tables stay empty until an operator turns a
-- rule on from the screen Phase 4b builds.
-- What an operator actually configures: trigger -> audience -> template -> timing.
--
-- `trigger_id` deliberately has NO foreign key and no existence check: a trigger
-- is DECLARED IN CODE (§4.3), so the set of them is whatever registered on this
-- boot. A rule naming a trigger no module currently registers is DORMANT — it is
-- listed, it never fires, and it starts working again when the module comes back
-- (§7.3). Deleting it on uninstall would silently destroy an operator's
-- configuration on the strength of a module being temporarily absent.
CREATE TABLE IF NOT EXISTS engagement_rules (
id INT AUTO_INCREMENT PRIMARY KEY,
trigger_id VARCHAR(96) NOT NULL,
name VARCHAR(160) NOT NULL,
-- OFF by default (§7.1 Q3). A rule arrives inert and an operator turns it on,
-- so no import, seed or restore can start mailing on its own.
enabled TINYINT(1) NOT NULL DEFAULT 0,
audience VARCHAR(32) NOT NULL DEFAULT 'owner',
audience_segment_id INT NULL,
-- §7.1 Q3: the hard stop that makes operator-editable rules safe to choose over
-- code-registered ones. Counted in engagement_sends, enforced before the outbox
-- row is written, never overridable from the rule editor beyond this column.
max_sends_per_hour INT NOT NULL DEFAULT 100,
channels JSON NOT NULL,
template_keys JSON NOT NULL,
conditions JSON NULL,
cooldown_seconds INT NOT NULL DEFAULT 0,
delay_seconds INT NOT NULL DEFAULT 0,
cancel_on JSON NULL,
updated_by INT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
CONSTRAINT fk_engr_user FOREIGN KEY (updated_by) REFERENCES users(id) ON DELETE SET NULL,
INDEX idx_engr_trigger (trigger_id, enabled)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- §5.1a: an operator-composed segment over module-declared audiences. Stored as a
-- boolean tree of audience ids + params; `ceiling` is DERIVED at save time as the
-- NARROWEST ceiling in the tree (ceilings.meetAll) and re-checked against the
-- trigger's own ceiling, so composition can never widen. It is a column rather
-- than a runtime computation so an audit can read what a rule was allowed to
-- reach without re-resolving it — and so a module that has since changed its
-- audience's ceiling cannot retroactively widen a saved segment.
--
-- `engagement_rules.audience_segment_id` above points here with NO foreign key,
-- on purpose and for the same reason `trigger_id` has none: a rule whose segment
-- has been deleted must go DORMANT, not silently fall back to its plain
-- `audience` column. ON DELETE SET NULL would be exactly that silent fallback,
-- and the fallback reaches a DIFFERENT set of people (§5.1a rule 4).
CREATE TABLE IF NOT EXISTS engagement_audience_segments (
id INT AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(160) NOT NULL,
expression JSON NOT NULL,
ceiling VARCHAR(32) NOT NULL,
updated_by INT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
CONSTRAINT fk_engseg_user FOREIGN KEY (updated_by) REFERENCES users(id) ON DELETE SET NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- §4.1. NOT `settings`: cooldown state is high-cardinality (recipients x rules x
-- subjects), written on every fire, and asked "is this one pair still cooling?".
-- A JSON blob under one settings key would be a read-modify-write of the whole
-- deployment's cooldown state per event, with a lost-update race between two
-- concurrent triggers.
--
-- `subject_key` is what makes "one IDOC mail per player per day" the right rule
-- instead of the wrong one: a player with four houses decaying should hear about
-- all four, once each. Cooling per (rule, user) alone silently drops three.
CREATE TABLE IF NOT EXISTS engagement_cooldowns (
rule_id INT NOT NULL,
user_id INT NOT NULL,
-- The SUBJECT the cooldown is about, opaque to core: a house serial, a vendor
-- id, ''. NOT NULL with a '' default, because this is a PRIMARY KEY column and
-- MariaDB would coerce a NULL one anyway. '' is "this rule cools per user, not
-- per subject".
subject_key VARCHAR(190) NOT NULL DEFAULT '',
last_fired_at DATETIME NOT NULL,
fire_count INT NOT NULL DEFAULT 1,
PRIMARY KEY (rule_id, user_id, subject_key),
CONSTRAINT fk_engc_rule FOREIGN KEY (rule_id) REFERENCES engagement_rules(id) ON DELETE CASCADE,
CONSTRAINT fk_engc_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
-- So a prune worker can drop rows older than the longest configured cooldown.
-- Without it this table grows without bound, which is the failure mode
-- teamActivityPrune was written for.
INDEX idx_engc_sweep (last_fired_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- §4.2a. Modelled on announce_jobs / announce_job_legs. One row per
-- (rule, user, channel) occurrence of an event.
CREATE TABLE IF NOT EXISTS engagement_outbox (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
rule_id INT NOT NULL,
trigger_id VARCHAR(96) NOT NULL, -- denormalized; survives a rule edit
user_id INT NOT NULL,
channel VARCHAR(32) NOT NULL, -- VARCHAR, never ENUM: the channel set is data
subject_key VARCHAR(190) NOT NULL DEFAULT '',
payload JSON NOT NULL, -- the declared variables, snapshotted at emit
dedupe_key VARCHAR(190) NULL,
status ENUM('scheduled','sending','sent','failed','cancelled','suppressed') NOT NULL DEFAULT 'scheduled',
due_at DATETIME NOT NULL,
attempts SMALLINT NOT NULL DEFAULT 0,
last_error TEXT NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
sent_at DATETIME NULL,
CONSTRAINT fk_engo_rule FOREIGN KEY (rule_id) REFERENCES engagement_rules(id) ON DELETE CASCADE,
CONSTRAINT fk_engo_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE,
-- **Scoped to the row's identity, and §4.2a's global `UNIQUE (dedupe_key)` is
-- a defect this phase found while building it.** A dedupe key names the EVENT
-- ("house 0x4001 entered IDOC"), and one event legitimately becomes many rows:
-- an audience of fifty users is fifty rows, a rule spanning email and in-app
-- doubles that, and two rules on one trigger double it again. Under a global
-- unique index the FIRST of those inserts wins and every other one is silently
-- ignored — ninety-nine recipients dropped by the mechanism meant to stop a
-- replayed event becoming a second mail. Scoping it to (rule, user, channel)
-- keeps exactly that guarantee and nothing more.
UNIQUE KEY uq_engo_dedupe (rule_id, user_id, channel, dedupe_key),
INDEX idx_engo_due (status, due_at),
-- What a RESOLVING event queries: a house repaired back to LikeNew cancels
-- every scheduled row for that (rule, user, house).
INDEX idx_engo_cancel (rule_id, user_id, subject_key, status)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- G15: the per-message record. Today "did user X get the mail?" is unanswerable.
--
-- It is deliberately NOT a second address book: the address is stored as a
-- sha256, which is enough to correlate a bounce (Phase 9) and useless as a
-- mailing list. `user_id` is SET NULL rather than CASCADE so the log survives an
-- account deletion — an audit of what this deployment sent must not be erasable
-- by deleting the recipient.
CREATE TABLE IF NOT EXISTS engagement_sends (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
outbox_id BIGINT NULL,
rule_id INT NULL,
trigger_id VARCHAR(96) NOT NULL,
user_id INT NULL,
channel VARCHAR(32) NOT NULL,
transport VARCHAR(32) NULL,
address_hash CHAR(64) NULL,
status ENUM('sent','failed','suppressed','bounced','complained') NOT NULL,
detail VARCHAR(500) NULL,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT fk_engs_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE SET NULL,
INDEX idx_engs_trigger (trigger_id, created_at),
INDEX idx_engs_user (user_id, created_at),
-- The per-rule hourly ceiling (§7.1 Q3) is counted here, so the count has to be
-- an index range scan rather than a table scan: it runs once per rule per event.
INDEX idx_engs_rule_window (rule_id, created_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;