5779d1515059b816153e0f117911b89dc426cf19
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 5779d15150 |
feat(engagement): retention — three sweeps and one recorded refusal
ENGAGEMENT.md Phase 14, the last phase of the workstream. Four engagement
tables grew on every fire and nothing had ever deleted from any of them.
Three of them now have a horizon, swept nightly by one worker
(utils/engagementRetentionPrune.js — setInterval + unref + stop(), batched
1000 x 50, each table's failure caught on its own so a lock timeout on one
does not leave the other two unbounded):
engagement_sends 180 days engagement_sends_retain_days (7-3650)
engagement_cooldowns 30 days engagement_cooldowns_retain_days (2-3650)
engagement_outbox 30 days engagement_outbox_retain_days (2-3650)
The fourth, engagement_suppressions, does not expire, and that is the
recorded decision rather than an omission: a suppression is a standing
decision, and ageing out a hard bounce re-mails an address that already
bounced. The way out stays deliberate, and is now reachable per row.
Six decisions were settled by the org lead before any code. Two of them
widened the phase past what was offered:
* the send-log horizon is admin-configurable, so retention got a SCREEN
(Admin -> Engagement -> Retention) where team_activity and
user_notifications keep theirs in invisible settings rows. The send-log
horizon changes what an operator-facing page is able to show, so it has
to be visible; the other two came with it, because "what does this
deployment keep" is one question.
* the suppression purge, which cost a Phase 9 decision. The list
deliberately stripped address_hash from every row, so the only way out
was a window.prompt asking the operator to retype an address the screen
has never shown them. The row had no handle at all. The hash is now
returned: this route is admin-only and an admin can already suppress and
unsuppress any address they can name, so it grants no capability they
lack. GET /sends still strips its own.
The outbox sweep is TERMINAL-ONLY and that is a correctness rule: a
scheduled row is a send this deployment still intends to make (delay_seconds
can put one a day out) and a sending row may be mid-flight.
One shipped defect had to be fixed for the sweep to be a bound at all.
reclaimStale returned every stale sending row to scheduled, and MAX_ATTEMPTS
is consulted only on a graceful retry outcome — so a send that killed the
process mid-flight cycled sending -> scheduled -> sending forever, never
terminal, therefore never eligible for any sweep. It now fails an exhausted
row BEFORE reclaiming the rest; the order is the fix.
Two indexes (idx_engo_sweep, idx_engs_sweep): every existing index on those
tables has created_at in second position, which serves a per-rule window and
is useless to a whole-table horizon.
Proved twice: engagementRetentionSql.test.js against a real MariaDB (7
tests, incl. the acceptance case and the wrong reclaim order run
deliberately), and the live stack, where a 90-day-old cancelled row was
swept and a 90-day-old scheduled row survived.
Co-Authored-By: Claude <noreply@anthropic.com>
|
|||
| c208543044 |
feat(engagement): deliverability — suppression, bounces and the verification gate
ENGAGEMENT.md Phase 9, closing gap G16. Two mechanisms decide that somebody in a
rule's audience does not get the mail, and they sit at deliberately different
points in the pipeline.
`engagement_suppressions` is checked at DELIVERY: an outbox row can sit through a
rule's `delay_seconds` grace window and an address can bounce inside it, so the
only correct check is the one taken immediately before the transport call — which
is also what produces the `status='suppressed'` row with no transport call at all.
The Phase 1b verification gate is applied at ENQUEUE, through a new optional
`registerDeliveryChannel({ eligible })` that only `email` declares. Filtering the
shared audience would have silenced the wrong sink: a rule spanning email and
in-app must still put an item in an unverified user's inbox. The excluded counts
reach `summary.ineligible` and the admin reach preview, which until now reported
an audience size that was never the number of people who would be mailed.
`bounceClassify.js` is the only thing that may write a `bounce` row, and it is
deliberately NOT `mailer.PERMANENT_CODES`. That set answers "is retrying
pointless?" and contains EAUTH and 554 — an auth failure and a relay-wide policy
refusal, neither of which is a fact about the recipient. Reusing it would mean one
stale SMTP password suppressing every address the worker touched, silently. The
classifier reads the RFC 3463 enhanced status first, falls back to a phrase match
only past a veto list and only for 550/551/553, and does not suppress anything it
is unsure about.
Scope is engagement rules only: resets, invites, verification and the contact form
still attempt, matching the posture passwordReset.controller.js already stated.
Found on the live rig, against a real MariaDB and a real SMTP conversation: a hard
bounce was being recorded as `failed`, so the Send Log's "Bounced" filter — a
status `engagement_sends` has carried since §4.5 — matched nothing and always
would have. It is now its own outcome; the outbox row stays `failed`, since that
ENUM has no `bounced` and a bounced row is one that finished unsuccessfully.
`address_masked` is this phase's one addition to §4.5's DDL. A hash-only table
cannot be operated — an operator cannot tell three typos from a whole domain
refusing mail — and the domain survives while the local part is destroyed, so the
column can never be read back as an address book.
- schema: `engagement_suppressions` (+ `address_masked`, `created_by`)
- `GET/POST/DELETE /api/v1/admin/engagement/suppressions`, and Admin → Engagement
→ Suppressions, the only way out of the list
- `sendNotification` returns `smtp: { code, responseCode, response }`
- 26 new tests; swagger, routes manifest and guards regenerated
Docs: RunicGateway/docs#191.
Co-Authored-By: Claude <noreply@anthropic.com>
|
|||
| 3f90070566 |
feat(engagement): the template editor, the trigger catalog and the send log (engagement Phase 5b)
Phase 5a gave templates a table, a renderer and nine seeded rows; nothing could
change one. This is the screen that lets an operator change one without being able
to break the mail the system depends on — plus the two screens Q4 promised Phase 5:
Triggers (read-only, from the registries) and the Send Log, which closes G15.
The shape follows from one fact: a mail body is rendered by the SERVER, so the
preview is too, and framed rather than redrawn in React. A client-side renderer
would be a second implementation of the one artifact that matters, agreeing with
the send path on the day it was written and drifting from the first Outlook fix on.
Settled with the org lead before any code: a shipped default is edited IN PLACE
(`protected` blocks deletion and nothing else, `customized = 1` keeps the edit);
duplicate is the only way to a new template; `renderByKey` now requires
`published`; a test send is logged under a synthetic `core.admin.test-send`; and a
template a rule points at refuses deletion with a 409 naming the rules.
Three things the plan did not know, found by building it:
- The undeclared-variable check cannot be a token scan. `email.itemList.variable`
holds a BARE name, so a digest pointed at `itmes` would have saved clean and
arrived empty. Blocks now declare `variables(props)`; the editor makes that
field a select over the trigger's list variables so the typo is unavailable.
- A duplicate that drops `seed_key` loses its variable palette, so duplicating
`notify.event` would have been refused for the tokens it was copied with — the
one action §4.6.2 offers, refusing itself. The copy inherits it; `customized`
is what the seeder actually reads.
- `validateEmailBlocks` returns `{ valid, errors }`, not an array, and the first
version tested it with `.length` — so block validation never ran at all.
Also fixes a Phase 4a defect the live walk found, with the org lead's approval: a
rule's template key was checked against a pattern with no dot in it, so no rule
could name any template that exists — §4.6.2's whole duplicate-and-point-a-rule-at-it
workflow was unreachable. Both models now read one pattern.
Verified against the running stack: real multipart mail into a mailpit catcher
including an unsaved draft, the draft/published arms both ways through the real
mailer path, every refusal, and the end-to-end duplicate → rule → 409 walk.
Server 1428 tests green, client 324.
Co-Authored-By: Claude <noreply@anthropic.com>
|
|||
| 4b45eddb5d |
feat(engagement): Admin - Engagement - Rules and Audiences (engagement Phase 4b)
The admin surface over the Phase 4a engine: two screens, twelve routes and the
reach preview. Nothing in the engine changed; what changed is that an operator
can now reach it.
Four decisions settled by the org lead before any code:
- segments get their OWN nav entry, "Audiences", not a tab of the rules screen
- the on/off switch is its own PATCH route, not a full PUT
- the reach preview is a count only, on demand
- a rule can be hard-deleted; the send log survives it
The switch is the one with real content in it. A PUT re-validates against the
registries as they are NOW, so the rules a re-validating toggle cannot switch
off are exactly the three an operator most wants stopped: a rule whose module
was uninstalled, one naming a channel that is gone, and one whose trigger has
since narrowed its ceiling under a saved audience. PATCH .../enabled writes one
column and always works. Switching ON unvalidated is safe because the engine
re-checks the ceiling at send time.
The preview calls the engine's own resolver rather than a second query that
agrees with it today, and answers a count and nothing else - the resolver's
output for a module-declared segment is a set of players derived from game data.
It reports `capped` at the 5000-row bound (the count is a floor, not a total),
`reason` for an `owner` audience (which resolves per event and has no advance
answer), and `permitted` so the editor cannot show a healthy number beside a
save the server will refuse.
Two defects found by walking it against a live server, both in Phase 4a's code:
1. A rule pointing at a DORMANT segment read as healthy. listAnnotated asked
only whether the segment ROW existed. The other shape of the same failure
is a segment sitting exactly where it was whose every audience belongs to
an uninstalled module: same outcome, nothing deleted. Uninstalling a module
under an enabled rule produced a rule the screen showed as on and firing.
The expression walk now lives in engagement/segments.js as
`missingAudiences` and both lists ask it.
2. "1 rule still use this segment" - the delete refusal pluralised the noun
and not the verb, in the sentence an operator reads when told no.
Also: a rule's trigger is now a stated rule rather than an omission in the
UPDATE statement (its cooldowns, queued sends and history are all about one
trigger id); a condition tree the editor cannot render is shown read-only rather
than flattened, because flattening changes which events fire the rule; and
literals are coerced client-side to the type the trigger declared, with anything
that does not parse passed through unchanged so the server's refusal names the
variable.
Tests: 21 new server tests (test/engagementAdmin.test.js) and 25 client ones
(client/test/engagementRules.test.js), all green. The single failure in the
server suite (`the committed manifest matches the declarations in the tree`) is
the known Windows CRLF artifact and fails identically on clean edge.
Companion docs PR: docs#184.
- [x] AI-assisted: written with Claude Code (Opus)
Co-Authored-By: Claude <noreply@anthropic.com>
|
|||
| 563199a096 |
feat(modules): event triggers, audiences and the ceiling lattice (engagement Phase 2)
The contract half of the engagement system: a module (and core) can DECLARE an
event with a payload contract and fire it. Nothing delivers yet — `emit`
validates, logs and stops, and Phase 4 replaces that log line with the engine.
`api.registerEventTriggers` and `api.registerAudiences` ride the existing
stage()/apply() validate-then-commit discipline, so a registrant that throws
halfway leaves nothing behind. `ctx.events.emit` is fire-and-forget and binds
the owner from the calling module — a module fires its own triggers and no one
else's. `ctx.inbox.push` is present and throws until Phase 7, the shape 1.6.0
settled on for a member that arrives a phase late.
MODULE_API_VERSION 1.7.0 on both halves. Additions only; module-uo's
`coreApi: "^1.3.0"` still resolves.
Three design decisions, approved by the org lead before any code:
ONE NAMESPACE for trigger ids and notification-stream ids (ENGAGEMENT.md §7.2,
against the recommendation in the text). A trigger is a payload contract
attached to an id that may also carry a subscription toggle, so an id has
exactly one owner across both facets, checked in both directions. Core's five
trigger ids ARE its five stream ids, so the same-owner upgrade case is
exercised on every boot rather than only by a module. It keeps
notification_channel_prefs single-keyed in Phase 3, where two namespaces would
have forced a `kind` discriminator into its primary key.
Two knock-on effects appeared only once it was implemented. The id grammar had
to be RELAXED to admit `_` inside a segment — §4.3's own worked example is
`uo.house.idoc_warning`, and two grammars over one namespace would mean an id
legal as a trigger and illegal as the stream it is the same event as. And the
seven grandfathered `uo.*` ids had to share their legacy allowlist with
triggers, because under one namespace `idoc.warning` is a single id. The push
catalog is untouched either way: allStreams() still serves the stream facet
only, so the shipped Android client sees exactly what it saw before.
THE CEILING LATTICE (G24), which the plan named everywhere and defined nowhere.
It is containment, not size: everyone ⊃ authenticated ⊃ {subscribers, members,
staff, owner}, with the four leaves mutually incomparable. The flat total order
the plan's wording invites would let a `staff`-ceilinged trigger be given an
`owner` audience — a rule that mails cheat detection to the player it detected.
Fewer people is not less exposure. Two incomparable ceilings have no meet at
all, so a composition is refused rather than guessed; union-widens is the
intuitive implementation and it is the wrong one.
`kind: 'event' | 'scheduled'` is declarable now and no evaluator exists (§7.1
Q6). Registration accepts `scheduled` and emit refuses to fire one, so `kind`
means something from the moment it can be written rather than from the moment
it is honoured.
Also: `GET /admin/engagement/{triggers,audiences}`, served from the registries
rather than a table so an uninstalled module simply stops appearing;
`npm run engagement:manifest` plus its CI `--check`, the twin of the route
manifest, because renaming a variable breaks stored templates silently, at send
time, in mail someone already received.
Co-Authored-By: Claude <noreply@anthropic.com>
|
