RunicGateway/docs
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(engagement): Phase 13 as built, and Phase 14 -- retention

Browse Source 
Phase 13 was the only phase in the workstream with no as-built section, which
6.0b makes a phase's own deliverable. It has one now, and it has something to
record beyond "it merged":

  * the window missed steps 5 and 7 (Integration-kit, runicgateway.com); they
    landed afterwards, with a third PR clearing a leftover from step 4;
  * Gitea's `raw` API route is CDN-cached for six hours, and a stale read fails
    BOTH ways -- it failed a correct site AND reported an already-republished
    bundle as current. All three cross-repo checks read `contents` now;
  * the Integration-kit equality check never goes red on its own. It clones the
    ref the kit itself pins, so the kit sat three minor versions behind, green,
    for the whole workstream. Step 5 is the only thing that makes anyone look;
  * `registerEngagementSeeds` does not validate the body it seeds, and the kit's
    own example was malformed. Only a run through core's REAL registry found it,
    which the kit's CI does not do and cannot;
  * every `edge` was deleted rather than fast-forwarded. Recorded as the
    convention, because a branch that does not exist cannot be branched from
    stale -- which was 6.0a's actual failure.

Phase 12's second finding becomes **Phase 14**, scoped and not started: four
tables with no retention sweep, and they are not one problem with one horizon.
`engagementCooldowns.db.prune()` already exists with no caller; the send log has
two live readers (the hourly cap's index and the operator screen) that bound how
short its horizon may be; and suppressions probably should NOT expire at all,
because ageing out a bounce re-mails an address that already hard-bounced. The
phase owes `/privacy` and the Play answers, which today say "nothing here expires
on its own" -- true now, false the moment it lands.

Also: ticks on the phase headings that were done and unmarked (4, 9, 10, 11, 12,
13), a status header that still said everything from Phase 10 on was design, and
three status lines that outlived the cutover -- `link/v5.md` and this README's
index row both said protocol 5 was built but unreleased, and INTEGRATION.md named
the v2.0.0/v1.0.0 pairing as the shipped one.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
claude
2026-09-01 13:17:08 -05:00
parent bc74a507b3
commit 5f5ca41d9e
4 changed files with 149 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

178
website/ENGAGEMENT.md
View File

@@ -1,30 +1,33 @@
# The Engagement System — findings and plan
**Status:** design of record. **Phases 1, 1a, 1b, 2, 3, 4a, 4b, 5a, 5b, 6, 7, 8 and 9 are built**
(Phase 1: website#165 + docs#178, with website#164 as its prerequisite; Phase 1a: website#166 +
docs#179; Phase 1b: website#167 + docs#180; Phase 2: website#168 + docs#181; Phase 3: website#169 +
docs#182; Phase 4a: website#170 + docs#183; Phase 4b: website#171 + docs#184; Phase 5a: website#172 +
docs#185; Phase 5b: website#173 + docs#186 + runicgateway.com#22; Phase 6: website#174 + docs#187 +
runicgateway.com#23; Phase 7: website#175 + docs#188 + runicgateway.com#24; Phase 8: the Android
half, Android-app#42 + docs#190; Phase 9: website#176 + docs#191); everything from Phase 10 on is
still design. **PHASE 10 IS BUILT (2026-08-31)** — the protocol bump; see its as-built below and
[`../link/v5.md`](../link/v5.md). **Phases 10 and 11 were both widened on 2026-08-31, by the org lead, before any code:**
the protocol bump carries three wire enrichments rather than one, and Phase 11 ships **every ✅ row of
§8.6** rather than a single rule. **Phase 11 was then scoped on the same day** — six decisions, three
carve-outs, a seventh value in the ceiling lattice and an 11a/11b split; see its own decision block.
The scope decisions
below are settled; **all nine questions in §7.1 are answered** - Q1, Q3, Q5 and Q7 on
2026-08-28, Q6 on 2026-08-29 at the start of Phase 2 (which also settled §7.2's namespace question),
**Q2 and Q4 on 2026-08-29 at the start of Phase 4**, and **Q8 on 2026-08-31 at the start of Phase
8**. Q1's answer added a whole phase (**Phase 1b**, unique email addresses); Q4's answer and the
phase's size split **Phase 4 into 4a and 4b**; **Q9** (core's own `news.post` emitter) was answered
**2026-08-31 at the start of Phase 11**, along with five further decisions that changed what that
phase ships — see its own decision block. Per CLAUDE.md § Conventions, no
implementation starts without the org lead's approval of the phase it belongs to.
**Status:** design of record. **The workstream is BUILT and CUT OVER (2026-09-01).** Phases 0
through 13 are done and on `main` in every repository; **Phase 14 (retention) is scoped and not
started**, and Phase 13's own acceptance walk has not been run. Each phase carries its own as-built
section below, and the PRs are named there rather than here.
**Branching:** every phase lands on **`edge`** in its repo; `main` is touched once, by the cutover
(Phase 13). §6.0a records the blocking precondition — six `edge` branches are stale and two repos have
none.
The cutover cut `link` **v2.1.0**, the plugin overlay **v1.1.0**, `module-uo` **v1.1.0** and the
paired bundle **2026.09.01** (protocol **5**); `MODULE_API_VERSION` is **1.9.0**. Two of Phase 13's
seven steps were missed in the window and landed afterwards — see its as-built, which is also where
the four findings of the cutover itself are recorded.
**Phases 10 and 11 were both widened on 2026-08-31, by the org lead, before any code:** the protocol
bump carries three wire enrichments rather than one, and Phase 11 ships **every ✅ row of §8.6**
rather than a single rule. **Phase 11 was then scoped on the same day** — six decisions, three
carve-outs, a seventh value in the ceiling lattice and an 11a/11b split; see its own decision block.
The scope decisions below are settled; **all nine questions in §7.1 are answered** — Q1, Q3, Q5 and
Q7 on 2026-08-28, Q6 on 2026-08-29 at the start of Phase 2 (which also settled §7.2's namespace
question), **Q2 and Q4 on 2026-08-29 at the start of Phase 4**, and **Q8 on 2026-08-31 at the start
of Phase 8**. Q1's answer added a whole phase (**Phase 1b**, unique email addresses); Q4's answer and
the phase's size split **Phase 4 into 4a and 4b**; **Q9** (core's own `news.post` emitter) was
answered **2026-08-31 at the start of Phase 11**, along with five further decisions that changed what
that phase ships — see its own decision block. Per CLAUDE.md § Conventions, no implementation starts
without the org lead's approval of the phase it belongs to, **and that still holds for Phase 14.**
**Branching:** every phase landed on **`edge`** in its repo, and `main` was touched once, by the
cutover (Phase 13). §6.0a records the blocking precondition it opened with — six `edge` branches
stale, three repos with none — and Phase 13's as-built records how it closed: **every `edge` was
deleted on merge rather than fast-forwarded**, which is the convention now. Phase 14 comes after the
cutover and so goes to `main` through an ordinary feature branch.
**Scope decisions, settled by the org lead (2026-08-28):**
@@ -1388,7 +1391,8 @@ change is not complete until `docs/` reflects it" — is the floor; this table i
| **11a** module-uo triggers | `modules/uo/API.md` — **the full trigger catalogue, its audiences and its ceilings**, not one entry · `modules/uo/README.md` · `website/ENGAGEMENT.md` §8.6 kept true as rows ship · `website/MODULE_API.md` §1.1 (**1.8.0**) and the ceiling vocabulary wherever it is enumerated · `BACKEND_DESIGN.md` — the `news.post` publish path now runs through the engine | `module-uo/README.md` |
| **11b** Seeded rules + templates | `website/ENGAGEMENT.md` this phase as built · a release note naming **the seeded-disabled `news.post` rule as an upgrade step** (decision 5) — without it a deployment loses news push silently | **`runicgateway.com`**: `capabilities.mjs` and the notifications page — "one rule" and "the whole catalogue" are different marketing claims |
| **12** Public site | `website/ENGAGEMENT.md` this phase as built, incl. the 12a/12b split and Phase 13's fill-in step | **`runicgateway.com`**, in full — see the phase. **Two PRs**: 12a (#26) mergeable now, 12b (#27) a draft held for the cutover window |
| **13** Cutover | `README.md` index rows · every doc's status line | `.profile/README.md` if this is a headline capability |
| **13** Cutover ✅ | `README.md` index rows · every doc's status line · **this phase as built**, incl. the two steps the window missed | `.profile/README.md` — the org lead called this a headline capability, so it landed |
| **14** Retention | `website/ENGAGEMENT.md` this phase as built · `BACKEND_DESIGN.md` table inventory · the settings keys it adds | **`runicgateway.com`**: `/privacy` + **`PLAY_DATA_SAFETY.md`**, both generated from `src/data/collection.mjs`, whose `deploy-engagement` row says "nothing here expires on its own" and stops being true |
**One thing this table is protecting against.** `runicgateway.com` appears in eight rows, and it is the
only repo here whose checks are *fetching* these values rather than being told them — see Phase 12.
@@ -1843,7 +1847,7 @@ see something a preference actually governs.
---
### Phase 4 — The engine: rules, cooldowns, outbox
### Phase 4 — The engine: rules, cooldowns, outbox ✅
**Split into 4a and 4b** at the start of the phase, on the same argument that split Phase 5: the half
that first makes this system capable of sending is worth reviewing without a React screen in the same
@@ -2914,7 +2918,7 @@ just using, on a train — and fails for the case where the app was killed first
---
### Phase 9 — Deliverability: suppression and bounces
### Phase 9 — Deliverability: suppression and bounces ✅
`engagement_suppressions` and bounce/complaint capture per transport (SMTP has none — this is where the
API-based providers earn their place).
@@ -3028,7 +3032,7 @@ a bounce handler today.
---
### Phase 10 — Protocol bump: three wire enrichments *(parallel from day one)*
### Phase 10 — Protocol bump: three wire enrichments ✅ *(parallel from day one)*
**Scope widened 2026-08-31, by the org lead, before any code.** The phase was written as the
`house.decay` enrichment alone. It now carries **three** enrichments, for one reason that is a property
@@ -3147,7 +3151,7 @@ once.
---
### Phase 11 — module-uo's triggers: the full catalogue, and core's `news.post`
### Phase 11 — module-uo's triggers: the full catalogue, and core's `news.post` ✅
**Scope widened 2026-08-31, by the org lead, before any code.** This phase used to say `module-uo`
registers `uo.house.idoc_warning` "(and siblings)", and Part 8's preamble used to say "Phase 11 ships
@@ -3590,7 +3594,7 @@ takes a fixed verb set, so `[p5probe` cannot be typed at a headless shard at all
---
### Phase 12 — `runicgateway.com`: the public site and the docs journey
### Phase 12 — `runicgateway.com`: the public site and the docs journey ✅
**This phase is not optional polish, and it is not "update the marketing copy". Two of its checks
will fail the build on their own**, and they read from `main`, which fixes exactly when this has to
@@ -3707,7 +3711,7 @@ has to sit outside the code span or the braces reach the published page.
---
### Phase 13 — The cutover: `edge` → `main`, in order
### Phase 13 — The cutover: `edge` → `main`, in order ✅
One PR per repo, all in one window. **The order is not cosmetic** — a `main` holding a v5 sidecar and a
v4 overlay is a shard that cannot pair, and the installer refuses it by design.
@@ -3742,6 +3746,113 @@ else, and `runicgateway.com` builds green with no fact disagreeing with its auth
is then fast-forwarded to `main` again so the next workstream starts from a clean one — the step
§6.0a found had been skipped after every previous cutover.
#### As built (2026-09-01)
**The window ran in the order above and missed two steps.** Steps 1 (docs#200), 2a
(servuo-plugins#20), 2b (link#35), 3 (website#180), 4 (Module-uo#26) and 6 (Android-app#43) merged
together; **5 and 7 did not**, and were done afterwards as **Integration-kit#9** and
**runicgateway.com#28**, with **Module-uo#27** clearing a leftover from step 4. What the window cut:
`link` **v2.1.0**, the plugin overlay **v1.1.0**, `module-uo` **v1.1.0**, and the paired bundle
**2026.09.01** (protocol 5). `website` releases nothing by design.
**Every `edge` was deleted rather than fast-forwarded**, in seven repos. The acceptance line above
asks for the fast-forward so the next workstream starts from a clean branch; deleting reaches the
same place more bluntly, since §6.0a's Phase -1 cuts `edge` fresh anyway — and a branch that does not
exist cannot be the stale one somebody branches from, which was the actual failure. Only
`Integration-kit` ended up in the state §6.0a warns about, one commit behind `main` once its own step
landed late; fast-forwarded on the day. **The convention is now delete-on-merge**, and Phase -1 cuts
`edge` rather than trusting one it finds.
**1. Gitea's `raw` API route is CDN-cached for six hours, and a stale read fails BOTH ways.**
`checkFacts.mjs` failed `runicgateway.com` for a `moduleApi` that was correct: it had been served
`website`'s `version.js` from a fortnight earlier (`Cache-Control: public, max-age=21600`,
`cf-cache-status: HIT`, `Age: 15713`), and no edit in that repository could have made it pass. The
false red is the cheap half. **The same run reported the already-republished bundle triple as still
current** — a stale read is just as able to say "nothing has moved", and that repo's whole bargain is
that it goes red when the platform moves. All three cross-repository checks now read the `contents`
endpoint, which answers `private, must-revalidate` and is not cached; a request `Cache-Control:
no-cache` header does **not** bust the CDN, and a cache-busting query param was rejected as
papering over the mechanism rather than choosing the right one.
**2. The Integration-kit equality check never goes red on its own, and step 5 is the only thing that
makes anyone look.** It clones the ref **the kit itself pins**, so a core that moves past that pin
changes nothing there. The kit sat three minor versions behind the platform for the whole
workstream — teaching a 1.6.0 contract with no triggers, audiences or seeds in it — and was green
throughout. Between cutovers the kit is not wrong, it is **dated**, and `ci/core-ref.json` is where
the date is written down. Read step 5's obligation as *someone re-reads the chapters*, never as
*CI will tell us*.
**3. `registerEngagementSeeds` does not validate the body it seeds, and the kit's own example was
malformed.** The call checks that `blocks` is a non-empty array and stops; the body is validated by
the block registry, which runs in the template editor and in the renderer and nowhere else. The
draft template shipped a heading `level: 2` — the registry takes `'h1' | 'h2' | 'h3'` — and no block
`id`s at all, so it would have registered cleanly, seeded cleanly, and failed the first time an
operator opened it. It was found by running the template's `register()` through **core's real
`stage()` validators** at the pinned ref, which the kit's CI does not do and cannot: it runs the
template against `test/_fakes.js`, and a fake accepts what core refuses. Two smaller corrections came
out of the same run — core **does** validate `subjectKey` against the declared variables and refuses
the module, and `ctx.events.emit` **throws outside production** rather than only dropping and
logging. Both had been written into the chapter the other way round.
The general lesson is §5.3's, arriving from the other side: a registration surface is only as
teachable as the thing that checks it. Where the registry validates, the book can describe the
error; where it does not — and `blocks` is the one place it does not — the book has to say so, and
the template needs a test of its own. It has one now, verified by breaking it.
**4. `runicgateway.com`'s `pr-checks.yml` triggers on PRs into `main` only**, so not one of the seven
site PRs in this workstream was gated, and the cutover was their first CI run in eight phases. Same
shape as `android-app`'s trigger before Q8 fixed it (§6.0a), and left alone rather than fixed: that
repo's checks read the *source repos' `main`*, so running them on an `edge` PR would have been red
for the whole window by design. Phase 12's split exists for the same reason. What it costs is real
though — 12b's fill-in was verified locally and merged on that evidence alone.
**Still open after this phase:** the acceptance walk above — the clean install from `main`, and the
`Greatly` transition producing exactly one email and one in-app item to the linked owner and nothing
to anyone else — has **not** been run. `.profile/README.md` (step 8) landed: the org lead called the
engagement system a headline capability. And Phase 12's second finding became **Phase 14**.
---
### Phase 14 — Retention: the engagement schema has no sweep
**Status: scoped, not started.** Phase 12 found this and recorded it without fixing it (finding 2 of
its as-built), on the grounds that a sweep is a `website` change and outside a documentation phase.
Nothing after it picked the finding up, so it is stated here as the phase it always was.
**Four tables grow without bound**, and they are not one problem with one horizon:
| Table | What accumulates | The constraint on a horizon |
| --- | --- | --- |
| `engagement_cooldowns` | one row per (rule, user, subject, channel) per fire, read once per fire | **`engagementCooldowns.db.prune(olderThan)` already exists and has no caller.** The horizon must exceed the longest `cooldown_seconds` on any enabled rule, or a pruned row makes the next fire a *first* fire — the model says so and leaves it to the caller |
| `engagement_outbox` | every enqueued delivery, including terminal `sent` / `failed` / `cancelled` / `suppressed` rows | only terminal rows are eligible; a `scheduled` row may be days out by design (`delay_seconds`) |
| `engagement_sends` (the send log) | one row per delivery attempt | **two live readers.** The per-rule hourly cap reads `idx_engs_rule_window (rule_id, created_at)`, so a horizon under an hour breaks Q3's ceiling; Admin → Engagement → Send Log is the operator's only answer to "was this person told", so a short one blinds the screen that exists to be looked at |
| `engagement_suppressions` | one row per suppressed address, forever | **the odd one out, and probably correct as it is.** A suppression is a standing decision; ageing out a `bounce` row means the next send re-mails an address that already hard-bounced, which is exactly how a sender loses a domain's reputation. If anything expires here it is `unverified`, and that is a decision to take rather than a default to assume |
**The shape is already in the tree, twice.** `utils/teamActivityPrune.js` and
`utils/userNotificationsPrune.js` are the same worker — `setInterval` + `unref` + `stop()`, a horizon
read from a `*_retain_days` setting, started and stopped in `server.js`. Phase 7 wrote the second one
for `user_notifications` after finding that table had no policy either, with one policy difference
worth carrying: it deletes **read items only**, because age alone would destroy the evidence for "I
was never told". The same question has to be answered per table here rather than assumed.
**Documentation this owes.** `runicgateway.com` again, and for Phase 12's reason: `/privacy` and the
generated **Play Data Safety** answers currently publish, for the `deploy-engagement` row, *"kept
until the operator removes them; nothing here expires on its own"*. That is the true answer today and
becomes false the moment this lands, and it is generated from one inventory (`src/data/collection.mjs`)
rather than written twice. Also `BACKEND_DESIGN.md`'s table inventory, and an operator-facing note for
whichever settings keys this adds.
**For the org lead, before any code:** the four horizons (or three, plus "suppressions do not
expire"), and whether the send log's retention is allowed to be shorter than the operator screen's
usefulness — the alternative is that the screen learns to say "older entries have been swept" rather
than showing a silently truncated history.
**Acceptance:** every one of the four tables has a stated policy — a sweep with a horizon, or a
recorded decision that it does not expire and why; the cooldown horizon is checked against the
longest enabled rule's cooldown rather than picked; `/privacy` and the Play answers are regenerated
from the inventory; and a rig run shows the sweep deleting terminal rows while leaving a `scheduled`
outbox row and an in-window send-log row alone.
---
### Sequencing
@@ -3756,9 +3867,11 @@ Stage C └─ 7 ── 8 (8 = app-store caden
Stage D └─ 9
Stage E 10 ──────────────────────────────────── 11a ── 11b (10 parallel from day one; 11 needs 6 + 10)
Stage F 12 ── 13 (12 written before 13, merged in its window)
Stage G └─ 14 (retention; after the cutover, so straight to `main`)
── all of the above onto `edge` ──
13 is the only thing that touches `main`
...and 14, which comes after it
```
**Phase -1 is blocking and takes minutes.** Every `edge` is 0 ahead / 3–16 behind `main` (§6.0a), so
@@ -3996,7 +4109,8 @@ Nothing here is "documentation to do at the end" — a phase is not done until i
web channel that does not exist until Phase 7) · new admin pages for rules, templates and per-channel
preferences · **`PLAY_DATA_SAFETY.md` + `/privacy`**, both generated from one inventory that an
engagement mailer materially changes.
- `.profile/README.md` — only if this lands as a headline capability.
- `.profile/README.md` — only if this lands as a headline capability. **It did:** the org lead
called it one, and the org landing page names the engagement system in the Phase 13 window.
---