docs(website): §7 rewritten for SMTP, plus an upgrade-notes home
Engagement Phase 1's documentation (ENGAGEMENT.md §6.0b, row 1). BACKEND_DESIGN.md §7 is REWRITTEN rather than amended, as the plan requires: it documented Gmail OAuth2 as the mechanism. It now covers the transport registry and why credentialFields is a single declaration three consumers read, the credential blob and its fail-safe decrypt, the no-phone-home rule and its CI guardrail, the three SMTP postures, why Send test is the only verification left, the five failure contracts, and the silent upgrade. The §2 tree, the PR-4 route-count note, the contact row and the dashboard row are corrected alongside it — the dashboard now returns warnings[]. UPGRADE_NOTES.md is new, and is the home for every later phase's operator note as well (1b, 6, 10 and 13 each owe one). Newest first, and scoped deliberately: an upgrade that needs nothing does not get an entry. The entries worth writing are the ones that fail QUIETLY, which is exactly what the Gmail removal does — mail stops, nothing errors, and the only signal is a dashboard warning. ENGAGEMENT.md records Phase 1 as built, with the four things that are not what the plan says: only the transport half of §3.1 was built (a channel nothing calls is a shape frozen too early), `enabled` now gates every sender where it used to gate none of the direct ones, the G22 warning reads the deprecated column on purpose, and `npm run swagger` had to be fixed first — the "per-file route limit" phases 8 and 9 recorded does not exist, and the real rule matters to every later phase here, all of which add routes. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -40,7 +40,8 @@ Skeleton from the spec, with a small number of justified additions marked **(+)*
|
||||
>
|
||||
> `users`, `account`, `invites`, `auth/providers` (PR 1, 28 routes), `moderation`, `bot-activity`,
|
||||
> `activity` (PR 2, 18 routes), `posts`, `uploads`, `wiki`, `pages` (PR 3, 31 routes) and `shard`,
|
||||
> `uo-link`, `email`, `discord-bot`, `settings`, `dashboard`/`site-mode` (PR 4, 33 routes) each live
|
||||
> `uo-link`, `email`, `discord-bot`, `settings`, `dashboard`/`site-mode` (PR 4, 33 routes at the time;
|
||||
> `email` has since lost its two Gmail connect routes, §7) each live
|
||||
> in their own router under `admin/`, behind `admin/index.js`. PR 5 did the same for `public/` (24),
|
||||
> `player/` (20) and the residual `auth/` (10). **`admin.routes.js`, `public.routes.js`,
|
||||
> `player.routes.js` and `auth.routes.js` are all deleted**; each group is now a directory whose
|
||||
@@ -170,8 +171,11 @@ server/
|
||||
from a manifest URL, enable,
|
||||
disable, uninstall, purge, restart
|
||||
and the source allowlist
|
||||
email.router.js (6) /admin/email — Gmail OAuth2
|
||||
delivery — adminOnly
|
||||
email.router.js (4) /admin/email — outbound mail:
|
||||
transport + credentials + send
|
||||
test — adminOnly. The two
|
||||
/connect/* routes went with Gmail
|
||||
OAuth2 (§7)
|
||||
discordBot.router.js (2) /admin/discord-bot — adminOnly
|
||||
settings.router.js (4) /admin/settings — adminOnly. The
|
||||
DELETE /:key is "reset to default"
|
||||
@@ -207,7 +211,8 @@ server/
|
||||
utils/
|
||||
auth.js JWT sign/verify, isLoggedIn middleware
|
||||
db.js MariaDB pool + ensureSchema()
|
||||
mailer.js (+) nodemailer; mailto fallback if SMTP unset
|
||||
mailer.js (+) nodemailer over a registered transport;
|
||||
mailto fallback when unconfigured (§7)
|
||||
client/ built in Phase 2/3 (React + Vite)
|
||||
Dockerfile
|
||||
docker-compose.yml
|
||||
@@ -923,7 +928,7 @@ from the per-route **siteMode** middleware (§5), never from an auth gate.
|
||||
| GET | `/posts/:category/:idOrSlug` | single published post |
|
||||
| GET | `/wiki` | list of pages (slug + title) |
|
||||
| GET | `/wiki/:slug` | single page |
|
||||
| POST | `/contact` | (rate-limited) send mail via SMTP; if unconfigured, respond `{fallback:"mailto", email}` |
|
||||
| POST | `/contact` | (rate-limited) send mail via the configured transport (§7); if unconfigured, respond `{fallback:"mailto", email}` |
|
||||
| GET | `/teams/by-external/:moduleId/:externalId` | one Team named the way the OWNING MODULE names it. Exists so a module's page can find core's Team without holding core's identifiers, which are core-internal. The module id is matched rather than trusted: an external id is unique only within a module |
|
||||
| GET | `/teams` | active, publicly visible Teams, paged. Every payload carries `{ configured, stale, lastSyncAt }` so a page can say how recently the projection was confirmed rather than presenting a stale roster as current, plus `enabled` — whether this deployment has Teams at all |
|
||||
| GET | `/teams/:slug` | one Team. An **archived** Team still resolves, read-only, and names its successor when it was renamed — an old bookmark or Discord link lands somewhere that explains itself. A **hidden** Team returns 404, indistinguishable from one that does not exist: "absent from every public surface" includes not confirming it is there. Carries `id`/`externalId`/`moduleId` — this route only, since the index has no use for them |
|
||||
@@ -973,7 +978,7 @@ never shown a control that would 403. The URLs below are unaffected by which
|
||||
file a route sits in — that is the property the route manifest freezes.
|
||||
| Method | Path | Purpose |
|
||||
|---|---|---|
|
||||
| GET | `/dashboard` | current mode, last change time + who, content counts, recent activity |
|
||||
| GET | `/dashboard` | current mode, last change time + who, content counts, recent activity, and `warnings[]` — operator conditions that are quietly not working and would otherwise be discovered by somebody not receiving an email. Normally empty. Each entry is `{code, message, href}` and each one is computed defensively: a warning that can 500 the admin landing page is a worse bug than the one it reports. Today there is one, `EMAIL_TRANSPORT_MIGRATION` (§7) |
|
||||
| PUT | `/site-mode` | `{mode}` → update settings, stamp who/when, log `site_mode.change` |
|
||||
| GET | `/posts?category=` | all posts incl. unpublished |
|
||||
| POST | `/posts` | create |
|
||||
@@ -1144,13 +1149,62 @@ still at [`SHARD_VISIBILITY.md`](SHARD_VISIBILITY.md).
|
||||
|
||||
## 7. Email
|
||||
|
||||
`utils/mailer.js` (nodemailer) sends through **Gmail over OAuth2 (SMTP XOAUTH2)**, configured in
|
||||
Admin → Settings → Email — not env. The mailbox is authorized by an in-app "Connect Gmail" consent
|
||||
flow (`/admin/email/*`) that captures a refresh token, stored AES-GCM-encrypted in the `email_config`
|
||||
singleton (never returned over the API). The OAuth client id/secret are reused from the `google`
|
||||
auth-providers row. Recipient is the `contact_email` site setting. If email is unconfigured/disabled,
|
||||
`POST /public/contact` returns `{fallback:"mailto", email}` so the client renders a `mailto:` link
|
||||
instead. Errors never leak credentials.
|
||||
`utils/mailer.js` (nodemailer) sends through a **registered mail transport**, configured in
|
||||
Admin → Settings → Email — never env, and never a compiled-in provider. Gmail OAuth2 and its consent
|
||||
flow were **removed** in engagement Phase 1 ([`ENGAGEMENT.md`](ENGAGEMENT.md) §1.2a); SMTP is the
|
||||
baseline and the only transport core ships.
|
||||
|
||||
**Transport, not provider.** `server/src/engagement/transports/` holds the registry and its one
|
||||
registration. A transport declares an id, a label and its own **`credentialFields`**, and that
|
||||
declaration is the single thing the admin form renders, the request sanitizer filters against, and
|
||||
the "is this value secret" answer comes from — so adding a relay is a registration, not four edits
|
||||
across a form, a validator, a column set and a model. `registerDeliveryChannel`, the other half of
|
||||
§3.1, arrives with the engine that consumes it.
|
||||
|
||||
**Configuration** lives in the `email_config` singleton: `transport` (default `smtp`),
|
||||
`sender_email`, `sender_name`, `reply_to`, and `credential_enc` — the transport's whole credential
|
||||
set as ONE AES-256-GCM JSON blob (`utils/secretBox.js`), because the field list belongs to the
|
||||
transport and a column per union member would make each new transport a schema change. The blob is
|
||||
**write-only over the API**: secret fields are never returned, only a per-field `secretsSet` flag,
|
||||
and a blob that will not decrypt reads as *absent* rather than raising — a rotated `SECRET_ENC_KEY`
|
||||
must land an admin on a screen that says "unconfigured", not a 500 that takes the contact form with
|
||||
it. `provider` and `refresh_token_enc` remain as **deprecated, unread columns** under the
|
||||
additive-only discipline.
|
||||
|
||||
**No phone-home.** No transport may ship a default host, port, endpoint or sender
|
||||
([`ENGAGEMENT.md`](ENGAGEMENT.md) §3.2). A transport with no operator configuration is
|
||||
`unconfigured` and its channel is off — it never falls back to a destination we chose.
|
||||
`npm run check:hosts` is the CI guardrail; it reads code, not prose, so documentation naming a host
|
||||
is fine and a literal in a transport is not.
|
||||
|
||||
**Three supported SMTP postures**, in the order an operator should consider them:
|
||||
|
||||
| Posture | Shape | When |
|
||||
| --- | --- | --- |
|
||||
| **A relay** (recommended) | Mailgun / SES / Postmark, host + port 587 + API-key-as-password | Anything with real volume. Reputation, bounce handling and DKIM are the relay's problem, not the operator's |
|
||||
| **A mailbox provider over SMTP** | e.g. `smtp.gmail.com` port 587 with an **app password** (not the account password) | A small deployment, and the migration path off the removed OAuth2 flow. Subject to the provider's own daily send caps |
|
||||
| **A self-hosted MTA** | An unauthenticated relay on port 25 on the same host | An operator who already runs mail. `user` and `password` are left blank; the transport treats a username with no password as incomplete, since that authenticates as nobody |
|
||||
|
||||
`secure` is the field operators get wrong: on for implicit TLS on 465, **off** for 587, which
|
||||
nodemailer upgrades with STARTTLS. 587-with-secure-on hangs rather than erroring cleanly.
|
||||
|
||||
**"Send test" is the verification, and it has to be.** Under the removed consent flow the sending
|
||||
address came back from Google's userinfo and was guaranteed to be a mailbox the credential owned.
|
||||
Operator-typed, it can be refused by the relay — a silent SPF/DMARC deliverability failure, not an
|
||||
error — so `POST /admin/email/test` is the only thing that proves the whole configuration, and its
|
||||
failures name the sender and the likely cause rather than passing a bare `550` through.
|
||||
|
||||
**Failure contracts.** Recipient for the contact form is the `contact_email` site setting. If email
|
||||
is unconfigured or disabled, `POST /public/contact` returns `{fallback:"mailto", email}` so the
|
||||
client renders a `mailto:` link, invites return the accept link for an admin to share by hand,
|
||||
password resets still answer a generic 200, and Team notifications are logged and swallowed. Only
|
||||
the admin test send throws — it is the only one with someone waiting to be told. Errors never leak
|
||||
credentials.
|
||||
|
||||
**The upgrade is silent by design and therefore announced.** An existing deployment backfills to
|
||||
`transport='smtp'` with no credentials, so every sink above politely does nothing and **mail simply
|
||||
stops**. The admin dashboard warns whenever the deprecated Gmail token is present and no replacement
|
||||
credential is; see [`UPGRADE_NOTES.md`](UPGRADE_NOTES.md).
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user