`.env.example` — the file docker-compose.yml actually reads — never listed
SECRET_ENC_KEY. `utils/secretBox.js` resolves the key at require time and throws
`SECRET_ENC_KEY must be set in production`, so following README Option A exactly
produces a container that crash-loops before it ever listens.
It was easy to miss because the variable IS documented in two places that a
Compose operator never opens: `server/.env.example`, which is what local
development copies, and the README's environment-variable reference table. Only
the file the deployment reads was missing it.
Reproduced against the published image with a clean `cp .env.example .env`, then
verified the fix the same way: fill in the values the README names and
`docker compose up -d` reaches `listening on http://0.0.0.0:3000` and
`/api/health` → `{"status":"ok"}`.
- `.env.example` gains SECRET_ENC_KEY, beside JWT_SECRET, with what it encrypts,
that production refuses to start without it, and that changing it later
orphans every stored secret rather than re-encrypting them.
- README's Option A "set at least" list gains SECRET_ENC_KEY and
BOT_INTERNAL_KEY. Both are refused-at-boot in production, and BOT_INTERNAL_KEY
is required even on a deployment that runs no bot, which is exactly the case
the list omitted.
Found while writing the runicgateway.com installation docs, whose quickstart is
checked against this file on every build.
Co-Authored-By: Claude <noreply@anthropic.com>
MODULES declares the module set a deployment runs, one entry per module as
`<id>@<version>=<install manifest URL>`, and the container arrives at it by
itself (MODULE_SYSTEM.md §2.7.2 decision 4). A module already unpacked at the
declared version is a no-op that makes NO network call, so a restart with the
network down comes up unchanged; anything else goes through install.js — same
allowlist, same sha256, same inspect-then-extract — and install() now takes an
`expect: {id, version}` so a URL resolving to another module or version is
refused while it is still only a manifest.
Resolution runs inside start(), between the seed and the require of app.js: the
seed is where the host allowlist setting comes from, and the require is what
scans the volume. That buys it the database, so a compose-installed module gets
the same provenance columns an admin install writes.
A failure is logged and carried, never fatal — an unreachable release host must
not take the site down. The declaration owns what is on the volume; the row owns
whether a module runs, so uninstalling a declared module returns its files at
the next start and leaves it disabled. The admin list gains that as a fourth
source (declared / declaredVersion / declaredError), because a declared module
that failed to resolve has no row, no directory and nothing mounted.
Deferring the app require moved core's schema ahead of the volume scan, and the
module schema-fragment replay was wired to core's schema — so every installed
module silently got no tables. Invisible to the suite (each one stubs the loader
or the pool) and to a smoke on a database that already had the tables; found by
booting against an empty one. ensureSchema() now takes `replayModules: false`
for the one caller that scans later, server.js replays them itself after the
require, and a bootOrder test pins the five steps in the only order they work in.
741 server tests (+18), 187 client (+5); manifest unchanged at 166 public + 2
internal, OpenAPI byte-identical.
Co-Authored-By: Claude <noreply@anthropic.com>
Core's half of the slice that closes phase 3. Two things: the request-time
fragment merge core has owed since phase 1, and the last of core's UO copy.
**The merge (MODULE_API.md §6.1a).** `swagger-output.json` is core's own routes
and cannot be anything else — it is generated on a developer's machine and
committed, so it must come out the same regardless of what they had checked out,
and a module arrives on a volume long after the image was built. Module routes
therefore reach the document at request time, from the `swagger-fragment.json`
each module ships: `swagger/docsSpec.js` merges the fragments of STARTED modules
over the committed spec, cached on a new loader state version and rebuilt when a
module's state moves.
Until now neither half existed. `swagger/mergeSpec.js` named the request-time
caller in its header and that caller was never written, so the 72 routes
module-uo serves were in no OpenAPI spec at all — core's standing rule ("never
ship a route that isn't in the spec") broken by the extraction rather than by a
route.
Core wins every key collision, `swagger-output.json` is never mutated (it is a
require()d JSON module — one in-place merge would be permanent AND cumulative),
and a fragment that is missing or unreadable costs that module its paths and
nothing else. The Swagger UI is now built per request for the same reason the
JSON is: bound once at require time it would show core's routes for the life of
the process while /api/docs.json showed the merged set.
**The last of core's UO copy** (slice 4 deferred it; §5.2's check reads code, not
prose, so none of this was caught):
- 31 UO schemas and 4 UO tags in `swagger/swagger.js`, describing routes core has
not served since slice 1 — 578 lines. They moved to module-uo, namespaced
`Uo…`, and arrive back through the merge on an instance that installs it.
- `info.description` said "a private Ultima Online shard".
- README.md's 48 UO mentions, including the architecture diagram and the whole
`## Shard integration (uo-link)` section, now `## Modules`.
- `TOWNCRIER_DURATION_SEC` and `UOLINK_*` in the two `.env.example`s: read by the
module, not by core, and documented in the module's README instead.
**Two dropped annotations, and the reason nobody knew.** swagger-autogen reports
an annotation it cannot parse and then prints Success in green, having skipped
it. `npm run swagger` now captures its diagnostics and fails — which immediately
found `POST /api/v1/admin/invites` and `POST /api/v1/auth/invite/:token/accept`
documented with an EMPTY request body, both since the day they were written.
Fixing the tag list also cleared five tags used by routes but never declared
(`Admin · Email`, `Admin · Invites`, `Admin · Moderation`, `Admin · Pages`,
`Auth · Me`) — the same defect class, in the other direction.
- 646 server tests (+9), 157 client tests unchanged
- routes.manifest.json unchanged (158 public + 2 internal); check:modules clean
- swagger-output.json: 128 paths, 69 schemas, 0 orphan tags, 0 orphan schemas
- verified against a real boot with module-uo installed: 197 merged paths
(128 core + 69 module), all four module tags, 31 Uo schemas, no dangling $refs,
/api/docs renders the module's operations with zero console errors
Refs: docs/website/MODULE_API.md §2.8, §6.1a; MODULE_SYSTEM.md §2.7.1
Co-Authored-By: Claude <noreply@anthropic.com>
The site's declared version is the admin-set uo_link_config.protocol column, so
the sidecar's PROTOCOL_VERSION 2 -> 3 bump has to be matched here or every REST
call 409s and uoLinkSocket closes the WS on the ws.hello mismatch. Five places
carry the number and all five move together: the column default, the model's
DEFAULT_PROTOCOL (what a site with nothing saved yet declares), the two
`config.protocol || 1` fallbacks in uoLinkClient/uoLinkSocket -- unreachable
today, but an unset value quietly sending 1 is exactly the confusing 409 the
version check exists to prevent -- the admin form's initial value, and the
documented env default.
The boot migration is the only subtle part. schema.sql is re-run on EVERY boot,
and `protocol` is admin-editable, so a bare UPDATE would silently un-pin an
operator who had deliberately pinned an older sidecar in Admin -> Shard. It is
therefore gated on a marker row in `settings`, written after the UPDATE: the
first boot on this build migrates, every later boot is a no-op. `protocol < 3`
rather than `= 2` picks up an install still on the old default of 1, which could
not have been talking to a v2 sidecar anyway. A fresh install has no row to
update and just gets the marker plus the new column default.
Verified against the local MariaDB through ensureSchema (the production path):
2 -> 3 with the marker written and the column default now 3; pinned back to 2 by
hand, re-ran, and it STAYED 2 -- the one-shot property holds. 673 server tests,
47 client tests, client build green.
Co-Authored-By: Claude <noreply@anthropic.com>
The ntfy service was configured with no published host port, on the
assumption that the public reverse proxy shares the compose network and
can dial ntfy:80 directly. It does not — Pangolin runs outside the
compose network and reaches every service through a published host port
(exactly why `app` publishes 3000). With no published port there was
nothing for the notification subdomain to forward to, so push delivery
could never work in production.
Publish container :80 on a host port (NTFY_HOST_PORT, default 2586,
binds 0.0.0.0 like `app`) and correct the now-inaccurate comments in
docker-compose.yml and ntfy/server.yml. Document NTFY_HOST_PORT in
.env.example. No code change — deploy config only.
Co-Authored-By: Claude <noreply@anthropic.com>
Additive, v1-only backend contract for the Android app's opt-in push (Part 1 of
M7; docs/android/PLAN.md §11). The app is a pure consumer — this lands the
endpoints, fan-out, and relay it needs.
- Schema: push_devices (per-device endpoint) + notification_subscriptions
(per-user opted-in streams), FK→users ON DELETE CASCADE.
- Stream catalog + event→stream mapping (config/notificationStreams.js): public
streams (news.post, server.status, idoc.warning, champ.start, governor.election)
drawn ONLY from the SSE PUBLIC_KINDS allowlist; personal owner-keyed streams
(vendor.sale, house.idoc, account.login). Full-state upserts (champ/city) fire
only on a real transition via an injectable tracker.
- Fan-out (utils/pushDispatch.js): content-free tickles ({ stream, ref }) POSTed
to each subscribed device; never throws. Two producers — shardIngest.ingest
(beside the SSE broadcast) and the create/publish-post path (news.post).
Personal events resolve to the owner via shardLinks. SSRF guard: endpoints must
be HTTPS, non-private, and on the NTFY_BASE_URL/NTFY_ALLOWED_ORIGINS allow-set —
enforced at registration and every publish.
- Routes under the role-agnostic self surface (never /admin): POST|GET
/auth/me/devices, DELETE /auth/me/devices/:id, GET
/auth/me/notifications/streams, GET|PUT /auth/me/notifications/subscriptions.
Swagger regenerated (4 paths, PushDevice/NotificationStreams/etc. schemas).
- ntfy service in docker-compose.yml: pinned image, declarative ./ntfy/server.yml,
no published host port, anonymous unguessable topics (no accounts) — zero
interactive setup. No publish token required (content-free design); optional
NTFY_PUBLISH_TOKEN honored.
- Tests: pushDispatch (mapping, PUBLIC_KINDS gate, owner-keying, SSRF guard,
content-free payload) + notifications route auth gate. Full suite green (247).
Co-Authored-By: Claude <noreply@anthropic.com>
Replace baked-in UOM/MysticMoon/UOMysticmoon branding with a BRAND_* env
scheme so one prebuilt image runs as any shard; UOMysticmoon becomes the
first tenant that sets these vars rather than a special case in the code.
Architecture (chosen because the app ships as a prebuilt image):
- server/src/config/brand.js + bot/src/brand.js read BRAND_* once at boot,
with Runic Gateway defaults.
- Text/colors reach the SPA at RUNTIME through the existing public settings
API (settings.model.getPublic -> SiteContext), so no client rebuild. The
admin-editable site title + contact email still override BRAND_NAME/email.
- SiteContext applies BRAND_ACCENT_COLOR to the --accent CSS var at runtime.
- Express templates the built index.html <title>/description/OG/favicon at
serve time from BRAND_* (renderIndexHtml in app.js).
- Server-side consumers read brand directly: emails, TOTP issuer, API docs,
boot logs, HTML error page. Bot uses it for embed color + logs.
Assets: logo/hero/favicon delivered from a ./brand:/app/brand bind-mount
(BRAND_LOGO/HERO/FAVICON), with neutral defaults baked in; hero falls back
to a built-in image when unset.
Scope: also genericized package.json names (uomysticmoon-* -> runic-gateway-*)
and the DB_NAME/DB_USER/COOKIE_NAME code defaults (runic_gateway/runic/
rg_token). Production keeps its real values by pinning them in .env — see
.env.uomysticmoon.example, which reproduces the exact UOMysticmoon identity
(proof the substitution works). Changing a deployed COOKIE_NAME invalidates
existing sessions, so UOMysticmoon pins uomm_token.
Verified: 193 server tests pass, client builds, app.js loads + templates the
built index.html, brand transform injects title/description/OG/favicon.
Point the `app` and `bot` services at the images published to the Gitea
registry by the build-images workflow, so deploys pull instead of building:
image: gitea.whitlocktech.com/uom/website-app:${IMAGE_TAG:-latest}
image: gitea.whitlocktech.com/uom/website-bot:${IMAGE_TAG:-latest}
`build:` is kept, so `up --build` still works locally; the server runs
`docker compose pull && up -d`. IMAGE_TAG defaults to `latest` for routine
deploys and pins to an immutable `sha-<7>` build for reproducible deploys /
rollback — no per-deploy compose edits. Documented in .env.example + README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0114TpmrNW4wNXsHq5CR72jQ
Introduces the DB-backed connection config for the uo-link sidecar (the
HTTP + WebSocket bridge to the ServUO shard) and a never-throw REST client,
mirroring the existing Discord-bot integration:
- uo_link_config singleton table (base/ws URL, AES-256-GCM-encrypted shared
token, protocol pin, enabled, and last-known status/plugin_connected/
last_event_at/boot_id mirrors for the admin panel).
- model/uoLinkConfig: getSafe (never returns the token — only hasToken),
getWithToken (server-side decrypt), save (blank token = unchanged),
recordStatus (mirror the sidecar's reported state).
- utils/uoLinkClient: never-throw fetch client returning {ok,data,status,
error}; Bearer token + X-UOLink-Version on every call; brief config cache;
helpers for health/char/roster/vendors/history/economy/link/towncrier.
- .env.example: UOLINK_BASE_URL/WS_URL/PROTOCOL defaults (token stays
admin-managed in the DB, never an env var).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011qPmpmVH1xGCiZoz9m9vW3
Retire env-var SMTP basic-auth and send the contact form through Gmail over
OAuth2 (SMTP XOAUTH2), configured in Admin -> Settings -> Email via an in-app
"Connect Gmail" consent flow. Reuses the existing google SSO OAuth client; the
captured refresh token is stored AES-GCM-encrypted (write-only over the API,
never returned), mirroring the auth-provider and Discord-bot secret patterns.
- schema: new email_config singleton table (mirrors bot_config)
- model: emailConfig.{db,model} with encrypted refresh token + getSafe/getWithSecret
- mailer: nodemailer OAuth2 transport (client id/secret from the google provider
row), contact recipient = contact_email setting, mailto: fallback preserved,
plus sendTest()
- routes/controller: /admin/email config, connect start+callback (ssoState CSRF
+ PKCE), test, disconnect
- client: EmailDelivery section on the Settings page + api methods; Settings copy
now spells out that contact_email is the delivery recipient
- docs/env: drop SMTP_*/CONTACT_TO from env examples; update README/BACKEND_DESIGN
- tests: emailConfig.model + mailer suites (8 new; full suite 142 pass)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XKeCQEJZr1AFJN4Bgcmvh3
The GET /internal/bot-config route returns the DECRYPTED Discord bot
token and was mounted on the same Express app / port 3000 that Pangolin
proxies publicly. Its only guard was the BOT_INTERNAL_KEY shared secret,
and .env.example shipped a placeholder default — so a forwarded path or a
weak/unrotated key would expose the plaintext token to the internet.
Move server<->bot internal traffic onto its own listener and fail fast on
a weak key:
- Add server/src/internalApp.js: a standalone Express app mounting
requireInternalKey + /internal (and a no-secret /health), mirroring the
bot's unpublished port-4100 pattern.
- server.js starts a second listener on INTERNAL_PORT (default 3001),
closed on graceful shutdown.
- Remove the /internal mount from the public v1.router; the public app now
404s /api/v1/internal/bot-config even with a valid key.
- Fail fast: new utils/botInternalKey.js rejects an empty, placeholder, or
<16-char BOT_INTERNAL_KEY — fatal in production (exit 1), warning in dev.
- docker-compose: bot SITE_INTERNAL_URL -> app:3001/internal/bot-config;
document that INTERNAL_PORT stays unpublished.
- .env.example (root/server/bot): document INTERNAL_PORT, the fail-fast
behavior, and a defense-in-depth Pangolin deny rule for /api/v1/internal.
Tests: add requireInternalKey.test.js and botInternalKey.test.js
(node --test: 93 pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019rao86n5cXpwAyjdBFEshV
Standalone bot/ service (its own package.json/Dockerfile) managed entirely
through a new admin-only Discord Bot panel — token stored encrypted in the
DB and pushed to the bot process in-memory, never an env var. Built in
phases, each independently verified against a live Discord guild:
- Bot skeleton: gateway connection, internal shared-secret API, self-heals
on its own restart by pulling config from the site
- Moderation core: /ban /kick /mute /warn /warnings + mod-log channel
- Word/invite/spam filtering with leetspeak-resistant normalization and a
staff role/channel allowlist
- Scheduled messages: recurring (cron) and one-off channel posts
- Role assignment: button role menus, auto-role on join, temp roles,
bulk role ops
- Auto-rotating primary invite with an audit log
- Site integration: news-publish -> Discord announce webhook, manual
/announce, read-only /wiki search
Also fixes a pre-existing bug in both DB pools (server + bot): the mariadb
driver defaulted to timezone 'local', silently mis-serializing bound Date
params by the host's local offset instead of the DB's UTC session.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds a layered set of protections around the admin login and the app edge.
Trust proxy (server/src/utils/trustProxy.js)
- Configurable via TRUST_PROXY; pin to the newt agent ("ptero") LAN IP so
X-Forwarded-For is trusted ONLY from that peer. A blanket "true" is
rejected (coerced to 1) to prevent XFF spoofing that would dodge every
IP-based control. DEBUG_TRUST_PROXY logs peer/XFF/req.ip to re-verify the
proxy IP without a redeploy. Documents the Omada static-reservation
assumption.
Login throttling (server/src/middleware/loginProtection.js, rateLimit.js)
- express-slow-down progressive delay + the existing hard rate cap + a
separate per-IP exponential backoff that persists across the rate window.
All failures return one generic message (no user/pass disclosure).
Honeypot (login form + auth.controller)
- Hidden, plausibly-named field ("company"); a filled value fails
generically and is scored as an unambiguous bot.
Optional per-user TOTP 2FA (speakeasy/qrcode)
- totp_secret/totp_enabled columns (+ idempotent migration). Self-service
Account page: enroll via QR, confirm a code to enable, code-gated disable.
- Login is two-step for enrolled users: after the password, a short-lived
signed challenge (stage:'totp', not a session) is required before the
real session is issued.
Bot / scanner scoring + IP ban (server/src/middleware/botScore.js)
- Weighted CMS-scanner paths (this app uses none). Junk paths 404 FIRST,
unconditionally — independent of score/ban state, so a scanner rotating
through fresh Cloudflare IPs gets no free pass. /wp-admin/install.php is
the top-weighted near-1-hit ban (worst offender in prod logs). Per-IP
score with quiet-period decay temp-bans an IP from ALL routes once past a
(deliberately low) threshold, to protect /admin from credential stuffing.
Failed logins and honeypot hits feed the same score.
- Periodic sweep evicts stale, unbanned, quiet entries so the in-memory
store can't grow unbounded; the interval is unref'd and cleared on
graceful shutdown.
Tests: node --test suite (40) covering trust-proxy parsing + live req.ip
(incl. pinned-IP), rate limiter + exponential backoff, honeypot rejection,
TOTP verify (enabled/disabled) + challenge-isn't-a-session, bot-score
threshold/decay/ban + junk-404-independence + install.php + store sweep.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
