3 Commits

Author	SHA1	Message	Date
wtclaude	5779d15150	feat(engagement): retention — three sweeps and one recorded refusal ... ENGAGEMENT.md Phase 14, the last phase of the workstream. Four engagement tables grew on every fire and nothing had ever deleted from any of them. Three of them now have a horizon, swept nightly by one worker (utils/engagementRetentionPrune.js — setInterval + unref + stop(), batched 1000 x 50, each table's failure caught on its own so a lock timeout on one does not leave the other two unbounded): engagement_sends 180 days engagement_sends_retain_days (7-3650) engagement_cooldowns 30 days engagement_cooldowns_retain_days (2-3650) engagement_outbox 30 days engagement_outbox_retain_days (2-3650) The fourth, engagement_suppressions, does not expire, and that is the recorded decision rather than an omission: a suppression is a standing decision, and ageing out a hard bounce re-mails an address that already bounced. The way out stays deliberate, and is now reachable per row. Six decisions were settled by the org lead before any code. Two of them widened the phase past what was offered: * the send-log horizon is admin-configurable, so retention got a SCREEN (Admin -> Engagement -> Retention) where team_activity and user_notifications keep theirs in invisible settings rows. The send-log horizon changes what an operator-facing page is able to show, so it has to be visible; the other two came with it, because "what does this deployment keep" is one question. * the suppression purge, which cost a Phase 9 decision. The list deliberately stripped address_hash from every row, so the only way out was a window.prompt asking the operator to retype an address the screen has never shown them. The row had no handle at all. The hash is now returned: this route is admin-only and an admin can already suppress and unsuppress any address they can name, so it grants no capability they lack. GET /sends still strips its own. The outbox sweep is TERMINAL-ONLY and that is a correctness rule: a scheduled row is a send this deployment still intends to make (delay_seconds can put one a day out) and a sending row may be mid-flight. One shipped defect had to be fixed for the sweep to be a bound at all. reclaimStale returned every stale sending row to scheduled, and MAX_ATTEMPTS is consulted only on a graceful retry outcome — so a send that killed the process mid-flight cycled sending -> scheduled -> sending forever, never terminal, therefore never eligible for any sweep. It now fails an exhausted row BEFORE reclaiming the rest; the order is the fix. Two indexes (idx_engo_sweep, idx_engs_sweep): every existing index on those tables has created_at in second position, which serves a per-rule window and is useless to a whole-table horizon. Proved twice: engagementRetentionSql.test.js against a real MariaDB (7 tests, incl. the acceptance case and the wrong reclaim order run deliberately), and the live stack, where a 90-day-old cancelled row was swept and a 90-day-old scheduled row survived. Co-Authored-By: Claude <noreply@anthropic.com>	2026-09-01 15:40:53 -05:00
wtclaude	065bec7ad8	feat(engagement): the email channel on the engine, and the Teams migration (engagement Phase 6) ... Email becomes a DeliveryChannel driven by rules, and the Team pipeline stops being its own thing. `teamNotify.forumPost` now emits an event; a rule decides who is mailed, through which template, and how often at most. One walk goes forum write -> events.emit -> rule -> outbox -> worker -> email channel -> template -> SMTP. Seven decisions settled by the org lead before any code: - email only moves; the push tickle and the Discord bridge stay direct calls - the EVENT carries its access-checked audience, and `members` resolves to it - the four Team rules are seeded DISABLED, with an admin banner and a note - team_notification_prefs stays, read by the engine as a scoped preference - the payload wins and a structural projection fills the gaps - the digest keeps computing at send time; only its state generalizes - an unsubscribe token turns off the channel it names, and nothing else Three defects found while building it: - `email.button` never absolutized its href, while image and itemList both did. Every rule-driven CTA would have been a dead relative link, because a trigger's url variables are validated site-relative by construction. - Phase 4a enqueued digest-mode recipients for a drain that Phase 6 decided not to build. An outbox row snapshots the payload and so has none of the three properties the digest design exists for, including the security one. - the digest's send-log row carried no address_hash while the instant row beside it did, which would have made half the mail uncorrelatable in Phase 9. Also: engagement_digest_state + a replay-safe backfill, engagement_outbox.scope_key, a v2 unsubscribe token that still verifies v1 forever, and the canonical /public/engagement/unsubscribe pair with the old /public/teams path kept permanently — mail is not editable once sent. Verified with 1464 server tests, 324 client tests, and a live rig (MariaDB + Mailpit + a real Team) covering the instant mail, the digest, the generic template, a pre-migration unsubscribe link and the backfill's replay-safety. Docs: RunicGateway/docs#TBD Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-29 20:11:54 -05:00
wtclaude	2079aaf667	feat(engagement): the rules engine, cooldowns and outbox (engagement Phase 4a) ... Phase 4 of docs/website/ENGAGEMENT.md, split 4a/4b at the org lead's direction. This is 4a: the engine, server only, with no HTTP surface at all. A fired trigger now produces outbox rows and send-log entries; Admin - Engagement - Rules and the segment composition UI are 4b. Five tables (rules, audience segments, cooldowns, outbox, sends), the sweep worker, audience resolution, condition evaluation, the grace window and its cancellation, and the save-path validation 4b's form will call. engagementEmit's Phase 2 log line becomes the engine call. Two settled questions this phase was blocked on: Q2 (multi-instance) - neither SKIP LOCKED nor documented single-instance: the outbox claims each row with a compare-and-set into the 'sending' state the ENUM already carried. It makes the outbox safe for two instances, not the deployment. Q4 (admin surface) - its own top-level nav group, built in 4b. Two defects in the plan's own section 4, both found by building it: The global UNIQUE(dedupe_key) was data loss. A dedupe key names the EVENT, and one event is one row per (rule, user, channel) - so a fifty-person audience would have had one row admitted and forty-nine silently ignored. Scoped. Section 4.1's single INSERT ... ON DUPLICATE KEY UPDATE cooldown claim always passes against this codebase's pool: the mariadb connector defaults foundRows:true, so a no-op update reports affectedRows 1 rather than 0. It is two statements now, with the interval guard in a WHERE clause. The second defect is why there is a second test file. The stubbed suite was green against the broken claim, because a stub can only agree with whoever wrote it; engagementEngineSql.test.js runs the raw statements against a real MariaDB and skips when there is none. Verification: 43 new tests green in engagementEngine.test.js, 12 more against MariaDB 11.8, and the whole path exercised end to end against a live database - per-subject cooldowns, conditions, the CAS claim, the send log's honest failure detail, and dormancy on uninstall. The three pre-existing Windows-only CRLF failures in the generated-artifact tests are unchanged from clean edge. Co-Authored-By: Claude <noreply@anthropic.com>	2026-08-29 08:07:27 -05:00