docs: the engagement workstream — cutover 1 of 7 (edgemain) #200

Merged
whitlocktech merged 50 commits from edge into main 2026-09-01 13:54:15 +00:00
3 changed files with 180 additions and 11 deletions
Showing only changes of commit 7b7a26f0ac - Show all commits

View File

@@ -1250,6 +1250,67 @@ The ntfy relay is treated as **untrusted infrastructure**, and the design makes
> `fix/notifications-empty-subscriptions`). The same trap applies to any "replace the full set" > `fix/notifications-empty-subscriptions`). The same trap applies to any "replace the full set"
> `PUT`/`POST` whose empty value equals a DTO default — prefer no default on required request fields. > `PUT`/`POST` whose empty value equals a DTO default — prefer no default on required request fields.
### Per-channel preferences — the superset endpoint (engagement phase 3, 2026-08-29)
Push is no longer the only channel a preference can name. `docs/website/ENGAGEMENT.md` phase 3 added
`notification_channel_prefs` and, with it, `GET · PUT /auth/me/notifications/channels`.
**Nothing above changed.** `/notifications/streams` and `/notifications/subscriptions` keep their
exact wire shapes, including the `{"streams":[]}` gotcha, and the shipped APK needs no update to keep
working — `notification_subscriptions` is now the **push projection** of the new table, and every
write to either fans out to the other. That was the acceptance criterion the phase was built against,
with the empty-array case tested explicitly.
**What the new endpoint adds, for whenever the app adopts it:**
```jsonc
// GET /auth/me/notifications/channels
{
"channels": [ // the delivery-channel registry
{ "id": "push", "label": "Push", "carriesContent": false,
"defaultMode": "off", "supportsDigest": false, "modes": ["off", "instant"] },
{ "id": "email", "label": "Email", "carriesContent": true,
"defaultMode": "off", "supportsDigest": true, "modes": ["off", "instant", "digest"] },
{ "id": "inapp", "label": "On the site", "carriesContent": true,
"defaultMode": "off", "supportsDigest": false, "modes": ["off", "instant"] }
],
"items": [ // every subscribable id, streams AND triggers
{ "id": "news.post", "label": "News posts", "description": "…",
"personal": false, "requiresLinkedAccount": false, "ceiling": "authenticated",
"channels": ["push", "email", "inapp"],
"modes": { "push": "instant", "email": "off", "inapp": "off" } }
]
}
```
Four properties the UI should be built on rather than around:
- **`items` is the union of streams and triggers**, one namespace. An id can be a push stream, an
event trigger with a payload contract, or both. A trigger-only id (`uo.house.idoc_warning`) carries
no `push` in its `channels` and no `push` key in `modes` — there is nothing registered to push it —
so **render the toggles from `channels`, never from a hardcoded three**.
- **`modes` is the *effective* mode, not the stored one.** Where the user has expressed nothing, the
server substitutes that channel's `defaultMode`. The client never has to know which it is looking
at, and must not re-implement the defaulting.
- **The PUT is sparse, and this is the one place it diverges from every other `/auth/me` PUT.** Send
only the pairs you changed: `{"prefs":[{"id":"news.post","channel":"email","mode":"digest"}]}`.
Everything not named is left alone, so the notifications screen can save one toggle without holding
the whole table. `off` is a mode, never an omission — **so the empty-array gotcha above does not
apply here at all**: there is no "clearing the last one" case, because turning something off is a
row like any other. `prefs` is still required, so a DTO field with no default is still the right
shape.
- **Entries the server cannot accept are dropped, not refused** — an unknown id, a channel that does
not apply to that id, a `digest` on a channel that cannot batch. The response is the full stored
state, so re-render from it rather than assuming the request took.
**One id may be missing from `items` that the app expects.** A trigger whose declared audience
`ceiling` is `staff` is not offered to a non-staff caller — it can never reach them, and listing it
would disclose that the event exists. `GET /notifications/streams` is unfiltered and unchanged.
**Phase 8** (`ENGAGEMENT.md`) is where the app grows the in-app inbox and this screen gains the
per-channel toggles. Until then the existing per-stream screen keeps working against
`/notifications/subscriptions` unmodified.
## 12. Build & CI (Gitea Actions) ## 12. Build & CI (Gitea Actions)
Builds run on the org's existing self-hosted runners (`runs-on: ubuntu-latest`, same label the other Builds run on the org's existing self-hosted runners (`runs-on: ubuntu-latest`, same label the other

View File

@@ -461,6 +461,36 @@ A DB read never yields a usable reset link. See §4 `/auth/password/*`.
`PRIMARY KEY(user_id, stream_id)`. Subscriptions are per-user (applied to every device); a PUT `PRIMARY KEY(user_id, stream_id)`. Subscriptions are per-user (applied to every device); a PUT
replaces the whole set. Nothing is pushed unless the user subscribed. replaces the whole set. Nothing is pushed unless the user subscribed.
Engagement phase 3 made this the **push projection** of `notification_channel_prefs` below. It keeps
its exact shape and stays what `utils/pushDispatch` reads — the shipped Android client cannot be
changed from this side — and the general table carries the channel dimension it lacks.
### notification_channel_prefs — which channel, in which mode (engagement phase 3)
| col | type | notes |
|---|---|---|
| user_id | INT NOT NULL FK→users(id) ON DELETE CASCADE | |
| stream_id | VARCHAR(64) NOT NULL | a stream id **or** a trigger id — **one namespace** ([`ENGAGEMENT.md`](ENGAGEMENT.md) §7.2), which is what keeps this key single-column |
| channel | VARCHAR(32) NOT NULL | `email` / `push` / `inapp`, from the delivery-channel registry (`src/engagement/channels.js`) |
| mode | ENUM('off','instant','digest') NOT NULL DEFAULT 'off' | `digest` only where the channel declares `supportsDigest` |
| updated_at | DATETIME | |
`PRIMARY KEY(user_id, stream_id, channel)`, `INDEX(channel, mode)`.
**A row exists only where the user has expressed something, and absence is the *channel's* default,
not `off`.** That default lives in the channel registry and nowhere else (§3.1, G9: push, email and
in-app do not agree on it). All three currently declare `off`, so absence and off happen to coincide
today — a fact about the declarations, not about this table, and code must not assume it. The column
`DEFAULT` is the value a write with no mode takes, not the meaning of a missing row.
**It is a superset of `notification_subscriptions`, which becomes its push projection.** The shipped
Android client's wire shape is frozen (`{streams:[…]}`), so the old table stays exactly what
`utils/pushDispatch` reads and every write to either fans out to the other. The invariant both
directions maintain: **a `push` row with `mode <> 'off'` ⟺ a `notification_subscriptions` row.** An
explicit `off` is *stored* rather than deleted — folding "I turned this off" back into "I never said"
is only harmless while the default is off. Existing subscriptions are carried across by an
`INSERT IGNORE … SELECT` backfill in `schema.sql`, replay-safe on every boot like the
`announce_jobs → announce_job_legs` one it copies.
### mobile_auth_sessions / mobile_auth_codes — mobile SSO bridge (M9) ### mobile_auth_sessions / mobile_auth_codes — mobile SSO bridge (M9)
Two short-lived, self-pruning tables that bridge a browser SSO redirect flow to a native client. They Two short-lived, self-pruning tables that bridge a browser SSO redirect flow to a native client. They
@@ -837,6 +867,7 @@ their own router level, and `/sso/:provider/link` carries `requireAuth` per rout
| GET | `/me/devices` · DELETE `…/:id` | cookie / bearer | — | list / unregister own push devices | | GET | `/me/devices` · DELETE `…/:id` | cookie / bearer | — | list / unregister own push devices |
| GET | `/me/notifications/streams` | cookie / bearer | — | the subscribable catalog (`personal`/`requiresLinkedAccount` flags) | | GET | `/me/notifications/streams` | cookie / bearer | — | the subscribable catalog (`personal`/`requiresLinkedAccount` flags) |
| GET · PUT | `/me/notifications/subscriptions` | cookie / bearer | `{streams:[id]}` on PUT | get / replace own opted-in streams (unknown ids dropped) | | GET · PUT | `/me/notifications/subscriptions` | cookie / bearer | `{streams:[id]}` on PUT | get / replace own opted-in streams (unknown ids dropped) |
| GET · PUT | `/me/notifications/channels` | cookie / bearer | `{prefs:[{id,channel,mode}]}` on PUT | get / update own **per-channel** preferences ([`ENGAGEMENT.md`](ENGAGEMENT.md) §4.5, phase 3). Returns the delivery-channel registry (`email`/`push`/`inapp`, each with `defaultMode`, `supportsDigest`, `modes`) plus one item per subscribable id — the **union** of push streams and event triggers, one namespace (§7.2) — carrying the **effective** mode on each channel that applies to it. A trigger-only id has no `push` toggle; a mode with no stored row reads as that channels default, so a client never sees which is which. The PUT is **sparse**: only the `(id, channel)` pairs listed are written and every other pair is untouched, so setting `email` cannot disturb `push`. `off` is a mode, never an omission — which is why this endpoint has no required-empty-array case. Entries naming an unknown id, an inapplicable channel or a mode that channel does not accept are **dropped, not refused**; the full stored state is echoed back. A `push` entry is mirrored into `/me/notifications/subscriptions`, whose wire shape is unchanged |
| GET · PUT | `/me/notifications/teams` | cookie / bearer | `{teams:[{teamId,muted,emailMode}]}` on PUT | get / replace own **per-Team** preferences (phase 6, [`TEAMS.md`](TEAMS.md) §6.3). One entry per Team the caller could be notified about — active membership or an active forum grant — plus any Team they already hold a preference for; server-side defaults applied. An entry naming a Team the caller has no access to is **dropped, not refused**: a Team left between loading the screen and saving it is a race, not a client bug. The array is required even when empty (`../android/PLAN.md` §11) | | GET · PUT | `/me/notifications/teams` | cookie / bearer | `{teams:[{teamId,muted,emailMode}]}` on PUT | get / replace own **per-Team** preferences (phase 6, [`TEAMS.md`](TEAMS.md) §6.3). One entry per Team the caller could be notified about — active membership or an active forum grant — plus any Team they already hold a preference for; server-side defaults applied. An entry naming a Team the caller has no access to is **dropped, not refused**: a Team left between loading the screen and saving it is a race, not a client bug. The array is required even when empty (`../android/PLAN.md` §11) |
**Role-agnostic self-service (`/auth/me/*`).** The **only** self-service account surface, for every **Role-agnostic self-service (`/auth/me/*`).** The **only** self-service account surface, for every

View File

@@ -1,13 +1,13 @@
# The Engagement System — findings and plan # The Engagement System — findings and plan
**Status:** design of record. **Phases 1, 1a, 1b and 2 are built** (Phase 1: website#165 + docs#178, **Status:** design of record. **Phases 1, 1a, 1b, 2 and 3 are built** (Phase 1: website#165 +
with website#164 as its prerequisite; Phase 1a: website#166 + docs#179; Phase 1b: website#167 + docs#178, with website#164 as its prerequisite; Phase 1a: website#166 + docs#179; Phase 1b:
docs#180); everything from Phase 3 on is still design. The scope decisions below are settled; **six of website#167 + docs#180; Phase 2: website#168 + docs#181); everything from Phase 4 on is still design.
the eight questions in §7.1 are answered** — Q1, Q3, Q5 and Q7 on 2026-08-28, and Q6 on 2026-08-29 at The scope decisions below are settled; **six of the eight questions in §7.1 are answered** — Q1, Q3,
the start of Phase 2, which also settled §7.2's namespace question. Q1's answer added a whole phase Q5 and Q7 on 2026-08-28, and Q6 on 2026-08-29 at the start of Phase 2, which also settled §7.2's
(**Phase 1b**, unique email addresses). **Q2, Q4 and Q8** remain open and block Phases 4, 5b and 8 namespace question. Q1's answer added a whole phase (**Phase 1b**, unique email addresses). **Q2, Q4
respectively. Per CLAUDE.md § Conventions, no implementation starts without the org lead's approval of and Q8** remain open and block Phases 4, 5b and 8 respectively. Per CLAUDE.md § Conventions, no
the phase it belongs to. implementation starts without the org lead's approval of the phase it belongs to.
**Branching:** every phase lands on **`edge`** in its repo; `main` is touched once, by the cutover **Branching:** every phase lands on **`edge`** in its repo; `main` is touched once, by the cutover
(Phase 13). §6.0a records the blocking precondition — six `edge` branches are stale and two repos have (Phase 13). §6.0a records the blocking precondition — six `edge` branches are stale and two repos have
@@ -551,7 +551,8 @@ registerDeliveryChannel({
id: 'email', // 'email' | 'push' | 'inapp' | later 'discord.dm' id: 'email', // 'email' | 'push' | 'inapp' | later 'discord.dm'
label: 'Email', label: 'Email',
carriesContent: true, // false for push — enforces the tickle invariant structurally carriesContent: true, // false for push — enforces the tickle invariant structurally
defaultMode: 'off', // email opt-IN, push opt-OUT — G9, expressed here once defaultMode: 'off', // G9, expressed here once. All three are opt-IN as built —
// 'push opt-OUT' was wrong; see Phase 3's as-built
supportsDigest: true, // in-app and push are instant-only in v1 supportsDigest: true, // in-app and push are instant-only in v1
addressFor(userId), // → [{ address, meta }] ; email reads users.email, push reads push_devices addressFor(userId), // → [{ address, meta }] ; email reads users.email, push reads push_devices
render(template, vars, ctx), // → the channel's own payload shape render(template, vars, ctx), // → the channel's own payload shape
@@ -1650,7 +1651,7 @@ engine, and this is what it migrates onto.
--- ---
### Phase 3 — Channel preferences ### Phase 3 — Channel preferences
`notification_channel_prefs` + the idempotent backfill from `notification_subscriptions`. New `notification_channel_prefs` + the idempotent backfill from `notification_subscriptions`. New
`GET·PUT /auth/me/notifications/channels`. **`/auth/me/notifications/subscriptions` keeps its exact wire `GET·PUT /auth/me/notifications/channels`. **`/auth/me/notifications/subscriptions` keeps its exact wire
@@ -1658,11 +1659,87 @@ shape** and becomes the push projection — writes fan out to both.
**Acceptance:** the shipped Android app's flat `{streams:[…]}` PUT still round-trips, including the **Acceptance:** the shipped Android app's flat `{streams:[…]}` PUT still round-trips, including the
empty-array case the app's DTO comment warns about; a per-channel PUT sets `email` without touching empty-array case the app's DTO comment warns about; a per-channel PUT sets `email` without touching
`push`; a fresh user's email mode defaults `off` and push defaults `instant` 4.5's `defaultMode`). `push`; a fresh user's email mode defaults `off` and ~~push defaults `instant`~~ **push defaults
`off` too** 4.5's `defaultMode` the struck text was wrong; see the as-built below).
**Guardrails:** swagger + route manifest; a test pinning the legacy wire shape byte-for-byte. **Guardrails:** swagger + route manifest; a test pinning the legacy wire shape byte-for-byte.
--- ---
#### As built (2026-08-29)
**Three decisions were settled by the org lead before any code, and one of them corrects this
phase's own acceptance criterion.**
| | Question | Decision |
|---|---|---|
| | how much of §3.1's `registerDeliveryChannel` lands now | **the declarative half only** id, label, `carriesContent`, `defaultMode`, `supportsDigest`. `addressFor` / `render` / `deliver` wait for the phases that can exercise them |
| | push's `defaultMode` | **`off`.** The acceptance line below said `instant`; it could not be |
| | whole-set PUT or sparse | **sparse**, on the `(id, channel)` pair deliberately unlike the two whole-set PUTs either side of it |
**The acceptance line was wrong, and it is worth saying exactly how.** "A fresh user's email mode
defaults `off` and push defaults `instant`" reads naturally beside §3.1's "email opt-IN, push
opt-OUT", and §3.1 got that from `team_notification_prefs`, where no row genuinely does mean notified.
But push **stream subscriptions** have never worked that way: `notification_subscriptions` holds a row
only when a user opted in, so no row means not subscribed. A `defaultMode` of `instant` would have
projected the **entire catalog** into `GET /auth/me/notifications/subscriptions` for every existing
user, and the shipped Android client would have shown every toggle switched on after an upgrade
nobody asked for. It is a live behaviour change dressed as a default. All three channels declare
`off`, and a test asserts the legacy GET returns `{streams:[]}` for a fresh user so it cannot drift
back.
**Why the channel registry could not wait for Phase 6.** §3.1 says `defaultMode` is expressed once,
and reading a preference means knowing it a row exists only where a user has said something. The
alternative was a constant list beside the prefs model, i.e. that expression in a second place, two
phases before the registry replaced it. What did *not* land is the behavioural half: registering a
`deliver` nothing calls freezes a signature before anything has tried to use it, which is the reason
`transports/index.js` deferred the whole file in Phase 1. Core's three channels are declared, and
`inapp` is declared `off` for a reason particular to it the inbox does not exist until Phase 7, and
a default of `instant` would mean every user is opted into a surface with no rows, so the first thing
Phase 7 shipped would be a backlog.
**The sparse PUT is the one place this phase leaves the router's idiom, and it buys two things.** A
whole-set body forces a client that only manages email to send every push row back or wipe them. And
`off` becomes a mode rather than an omission which means this endpoint has **no empty-array case at
all**, so the kotlinx gotcha `putTeamPrefs` had to document (a defaulted array field is dropped from
the body, and "clear the last one" arrives as no array) simply cannot arise here. `prefs` is still
required, so a request DTO with no default is still the right shape on the app side.
**The projection, stated as an invariant.** `notification_subscriptions` stays exactly what
`utils/pushDispatch` reads, so this phase touches no delivery path at all. Both endpoints maintain:
**a `push` pref with `mode <> 'off'` ⟺ a `notification_subscriptions` row** the legacy PUT with a
whole-set sweep, the channels PUT one pair at a time. An explicit `off` is *stored* rather than
deleted, because folding "I turned this off" back into "I never said" is only harmless while the
default happens to be off.
**One thing landed that the phase did not name, and it is a G24 consequence rather than scope creep.**
A trigger whose ceiling is `staff` can never reach a non-staff user, so offering them a toggle is
offering a control that does nothing *and* disclosing that the event exists `uo.cheat.detected`
would otherwise appear by name in every player's preferences screen the moment Phase 11 declared it.
It is filtered from the catalog and gated on write, not merely hidden. `members` is deliberately not
filtered: membership is a runtime resolver's answer, and a preference set before joining a Team should
already be in place when you join. This gave `ceilings.js` its first consumer for the `staff` label's
long-standing claim of "admin / editor / moderator", now written down as `STAFF_CEILING_ROLES` and
deliberately **not** `teamGrants.STAFF_ROLES` (`['admin','moderator']`), which answers the different
question of who may act on a Team they are not in.
**What landed:**
- `server/src/engagement/channels.js` `registerDeliveryChannel`, `MODES`, `defaultMode`, `modesFor`,
`acceptsMode`; `coreChannels.js` declares push / email / inapp, registered through the subsystem's
one door (`require('./engagement')` from `app.js`, beside `registerCore()`)
- `notification_channel_prefs` + the replay-safe `INSERT IGNORE … SELECT` backfill, copying the
`announce_jobs → announce_job_legs` precedent
- `model/notificationChannelPrefs/` the catalog union, effective-mode resolution, the sparse apply,
and `mirrorPushSet` for the legacy path; two single-row helpers on `notificationSubs.db`
- `GET · PUT /auth/me/notifications/channels`, swagger schemas, route manifest
- `ceilings.STAFF_CEILING_ROLES` / `isStaffRole`
**Left for later, deliberately:** no web or app surface. The endpoint exists and is documented
(`../android/PLAN.md` §11); the screens are Phase 7 (web) and Phase 8 (app), which is where a user can
see something a preference actually governs.
---
### Phase 4 — The engine: rules, cooldowns, outbox ### Phase 4 — The engine: rules, cooldowns, outbox
`engagement_rules`, `engagement_cooldowns`, `engagement_outbox`, `engagement_sends`, the sweep worker `engagement_rules`, `engagement_cooldowns`, `engagement_outbox`, `engagement_sends`, the sweep worker