docs(website): §7 rewritten for SMTP, plus an upgrade-notes home

Engagement Phase 1's documentation (ENGAGEMENT.md §6.0b, row 1).

BACKEND_DESIGN.md §7 is REWRITTEN rather than amended, as the plan
requires: it documented Gmail OAuth2 as the mechanism. It now covers the
transport registry and why credentialFields is a single declaration
three consumers read, the credential blob and its fail-safe decrypt, the
no-phone-home rule and its CI guardrail, the three SMTP postures, why
Send test is the only verification left, the five failure contracts, and
the silent upgrade. The §2 tree, the PR-4 route-count note, the contact
row and the dashboard row are corrected alongside it — the dashboard now
returns warnings[].

UPGRADE_NOTES.md is new, and is the home for every later phase's
operator note as well (1b, 6, 10 and 13 each owe one). Newest first, and
scoped deliberately: an upgrade that needs nothing does not get an entry.
The entries worth writing are the ones that fail QUIETLY, which is
exactly what the Gmail removal does — mail stops, nothing errors, and
the only signal is a dashboard warning.

ENGAGEMENT.md records Phase 1 as built, with the four things that are
not what the plan says: only the transport half of §3.1 was built (a
channel nothing calls is a shape frozen too early), `enabled` now gates
every sender where it used to gate none of the direct ones, the G22
warning reads the deprecated column on purpose, and `npm run swagger`
had to be fixed first — the "per-file route limit" phases 8 and 9
recorded does not exist, and the real rule matters to every later phase
here, all of which add routes.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-08-28 21:01:19 -05:00
parent 7a2ad727cb
commit 6f6b54a509
4 changed files with 180 additions and 15 deletions

View File

@@ -40,7 +40,8 @@ Skeleton from the spec, with a small number of justified additions marked **(+)*
>
> `users`, `account`, `invites`, `auth/providers` (PR 1, 28 routes), `moderation`, `bot-activity`,
> `activity` (PR 2, 18 routes), `posts`, `uploads`, `wiki`, `pages` (PR 3, 31 routes) and `shard`,
> `uo-link`, `email`, `discord-bot`, `settings`, `dashboard`/`site-mode` (PR 4, 33 routes) each live
> `uo-link`, `email`, `discord-bot`, `settings`, `dashboard`/`site-mode` (PR 4, 33 routes at the time;
> `email` has since lost its two Gmail connect routes, §7) each live
> in their own router under `admin/`, behind `admin/index.js`. PR 5 did the same for `public/` (24),
> `player/` (20) and the residual `auth/` (10). **`admin.routes.js`, `public.routes.js`,
> `player.routes.js` and `auth.routes.js` are all deleted**; each group is now a directory whose
@@ -170,8 +171,11 @@ server/
from a manifest URL, enable,
disable, uninstall, purge, restart
and the source allowlist
email.router.js (6) /admin/email — Gmail OAuth2
delivery — adminOnly
email.router.js (4) /admin/email — outbound mail:
transport + credentials + send
test — adminOnly. The two
/connect/* routes went with Gmail
OAuth2 (§7)
discordBot.router.js (2) /admin/discord-bot — adminOnly
settings.router.js (4) /admin/settings — adminOnly. The
DELETE /:key is "reset to default"
@@ -207,7 +211,8 @@ server/
utils/
auth.js JWT sign/verify, isLoggedIn middleware
db.js MariaDB pool + ensureSchema()
mailer.js (+) nodemailer; mailto fallback if SMTP unset
mailer.js (+) nodemailer over a registered transport;
mailto fallback when unconfigured (§7)
client/ built in Phase 2/3 (React + Vite)
Dockerfile
docker-compose.yml
@@ -923,7 +928,7 @@ from the per-route **siteMode** middleware (§5), never from an auth gate.
| GET | `/posts/:category/:idOrSlug` | single published post |
| GET | `/wiki` | list of pages (slug + title) |
| GET | `/wiki/:slug` | single page |
| POST | `/contact` | (rate-limited) send mail via SMTP; if unconfigured, respond `{fallback:"mailto", email}` |
| POST | `/contact` | (rate-limited) send mail via the configured transport (§7); if unconfigured, respond `{fallback:"mailto", email}` |
| GET | `/teams/by-external/:moduleId/:externalId` | one Team named the way the OWNING MODULE names it. Exists so a module's page can find core's Team without holding core's identifiers, which are core-internal. The module id is matched rather than trusted: an external id is unique only within a module |
| GET | `/teams` | active, publicly visible Teams, paged. Every payload carries `{ configured, stale, lastSyncAt }` so a page can say how recently the projection was confirmed rather than presenting a stale roster as current, plus `enabled` — whether this deployment has Teams at all |
| GET | `/teams/:slug` | one Team. An **archived** Team still resolves, read-only, and names its successor when it was renamed — an old bookmark or Discord link lands somewhere that explains itself. A **hidden** Team returns 404, indistinguishable from one that does not exist: "absent from every public surface" includes not confirming it is there. Carries `id`/`externalId`/`moduleId` — this route only, since the index has no use for them |
@@ -973,7 +978,7 @@ never shown a control that would 403. The URLs below are unaffected by which
file a route sits in — that is the property the route manifest freezes.
| Method | Path | Purpose |
|---|---|---|
| GET | `/dashboard` | current mode, last change time + who, content counts, recent activity |
| GET | `/dashboard` | current mode, last change time + who, content counts, recent activity, and `warnings[]` — operator conditions that are quietly not working and would otherwise be discovered by somebody not receiving an email. Normally empty. Each entry is `{code, message, href}` and each one is computed defensively: a warning that can 500 the admin landing page is a worse bug than the one it reports. Today there is one, `EMAIL_TRANSPORT_MIGRATION` (§7) |
| PUT | `/site-mode` | `{mode}` → update settings, stamp who/when, log `site_mode.change` |
| GET | `/posts?category=` | all posts incl. unpublished |
| POST | `/posts` | create |
@@ -1144,13 +1149,62 @@ still at [`SHARD_VISIBILITY.md`](SHARD_VISIBILITY.md).
## 7. Email
`utils/mailer.js` (nodemailer) sends through **Gmail over OAuth2 (SMTP XOAUTH2)**, configured in
Admin → Settings → Email — not env. The mailbox is authorized by an in-app "Connect Gmail" consent
flow (`/admin/email/*`) that captures a refresh token, stored AES-GCM-encrypted in the `email_config`
singleton (never returned over the API). The OAuth client id/secret are reused from the `google`
auth-providers row. Recipient is the `contact_email` site setting. If email is unconfigured/disabled,
`POST /public/contact` returns `{fallback:"mailto", email}` so the client renders a `mailto:` link
instead. Errors never leak credentials.
`utils/mailer.js` (nodemailer) sends through a **registered mail transport**, configured in
Admin → Settings → Email — never env, and never a compiled-in provider. Gmail OAuth2 and its consent
flow were **removed** in engagement Phase 1 ([`ENGAGEMENT.md`](ENGAGEMENT.md) §1.2a); SMTP is the
baseline and the only transport core ships.
**Transport, not provider.** `server/src/engagement/transports/` holds the registry and its one
registration. A transport declares an id, a label and its own **`credentialFields`**, and that
declaration is the single thing the admin form renders, the request sanitizer filters against, and
the "is this value secret" answer comes from — so adding a relay is a registration, not four edits
across a form, a validator, a column set and a model. `registerDeliveryChannel`, the other half of
§3.1, arrives with the engine that consumes it.
**Configuration** lives in the `email_config` singleton: `transport` (default `smtp`),
`sender_email`, `sender_name`, `reply_to`, and `credential_enc` — the transport's whole credential
set as ONE AES-256-GCM JSON blob (`utils/secretBox.js`), because the field list belongs to the
transport and a column per union member would make each new transport a schema change. The blob is
**write-only over the API**: secret fields are never returned, only a per-field `secretsSet` flag,
and a blob that will not decrypt reads as *absent* rather than raising — a rotated `SECRET_ENC_KEY`
must land an admin on a screen that says "unconfigured", not a 500 that takes the contact form with
it. `provider` and `refresh_token_enc` remain as **deprecated, unread columns** under the
additive-only discipline.
**No phone-home.** No transport may ship a default host, port, endpoint or sender
([`ENGAGEMENT.md`](ENGAGEMENT.md) §3.2). A transport with no operator configuration is
`unconfigured` and its channel is off — it never falls back to a destination we chose.
`npm run check:hosts` is the CI guardrail; it reads code, not prose, so documentation naming a host
is fine and a literal in a transport is not.
**Three supported SMTP postures**, in the order an operator should consider them:
| Posture | Shape | When |
| --- | --- | --- |
| **A relay** (recommended) | Mailgun / SES / Postmark, host + port 587 + API-key-as-password | Anything with real volume. Reputation, bounce handling and DKIM are the relay's problem, not the operator's |
| **A mailbox provider over SMTP** | e.g. `smtp.gmail.com` port 587 with an **app password** (not the account password) | A small deployment, and the migration path off the removed OAuth2 flow. Subject to the provider's own daily send caps |
| **A self-hosted MTA** | An unauthenticated relay on port 25 on the same host | An operator who already runs mail. `user` and `password` are left blank; the transport treats a username with no password as incomplete, since that authenticates as nobody |
`secure` is the field operators get wrong: on for implicit TLS on 465, **off** for 587, which
nodemailer upgrades with STARTTLS. 587-with-secure-on hangs rather than erroring cleanly.
**"Send test" is the verification, and it has to be.** Under the removed consent flow the sending
address came back from Google's userinfo and was guaranteed to be a mailbox the credential owned.
Operator-typed, it can be refused by the relay — a silent SPF/DMARC deliverability failure, not an
error — so `POST /admin/email/test` is the only thing that proves the whole configuration, and its
failures name the sender and the likely cause rather than passing a bare `550` through.
**Failure contracts.** Recipient for the contact form is the `contact_email` site setting. If email
is unconfigured or disabled, `POST /public/contact` returns `{fallback:"mailto", email}` so the
client renders a `mailto:` link, invites return the accept link for an admin to share by hand,
password resets still answer a generic 200, and Team notifications are logged and swallowed. Only
the admin test send throws — it is the only one with someone waiting to be told. Errors never leak
credentials.
**The upgrade is silent by design and therefore announced.** An existing deployment backfills to
`transport='smtp'` with no credentials, so every sink above politely does nothing and **mail simply
stops**. The admin dashboard warns whenever the deprecated Gmail token is present and no replacement
credential is; see [`UPGRADE_NOTES.md`](UPGRADE_NOTES.md).
---