From a103e0ce10d88db140e75cabb63b0d9d8d3e98ae Mon Sep 17 00:00:00 2001 From: wtclaude Date: Tue, 11 Aug 2026 00:48:48 -0500 Subject: [PATCH] feat(modules): mount the modules directory as a volume (phase 2, PR 9) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes Phase 2. Modules live on a mount, never in the image — that is what lets an operator add one to a pull-only deployment without building anything. `./modules` is a bind mount rather than a named volume: placing a module directory by hand is a supported install (MODULE_SYSTEM.md §2.5), and that has to be doable from the host rather than through `docker cp`. Read-write, because the admin panel's install/uninstall unpacks and removes directories there. The directory is tracked via its README so it exists in the checkout with the operator's own ownership — Docker recreates a missing bind-mount source as root:root, which the container user could not then write. `.dockerignore` excludes it so a module in the builder's working tree can never ship inside an image. Also corrects the route-manifest generator's list of filesystem-conditional mounts, which never picked up `/modules` when PR 7 added it. Comment only; the generator filters on an allowlist, so its behaviour was already right. Verified against a real container, not just a parsed compose file: image carries an empty node-owned /app/modules despite a module in the build context; a module on the bind mount loads, mounts, replays and reaches `started`; `/api/v1/public/modules` lists it; the chunk serves from the entry's directory only (server source and module.json 404) with `no-cache`; the injected tag follows core's bundle; and in Chrome the page renders on first paint inside core's PublicLayout with its nav row interleaved into core's public nav, under enforced `script-src 'self'` with zero CSP reports and no console errors. Removing the directory by hand reconciles the row to `startup_failed`/`require` and leaves core healthy with no injection. 933 server + 160 client tests pass, manifest unchanged at 230 routes, swagger regenerates byte-identical. Co-Authored-By: Claude --- .dockerignore | 3 ++ .gitignore | 7 +++ Dockerfile | 6 +++ README.md | 12 ++++-- docker-compose.yml | 23 ++++++++++ modules/README.md | 75 +++++++++++++++++++++++++++++++++ server/scripts/routeManifest.js | 12 +++--- 7 files changed, 130 insertions(+), 8 deletions(-) create mode 100644 modules/README.md diff --git a/.dockerignore b/.dockerignore index 0431799..8b1c180 100644 --- a/.dockerignore +++ b/.dockerignore @@ -10,5 +10,8 @@ uploads server/logs logs *.log +# Installed modules are mounted at runtime, never baked into the image. Without +# this a module in the builder's working tree would ship inside every image. +modules .DS_Store Thumbs.db diff --git a/.gitignore b/.gitignore index 9d33164..a6a0669 100644 --- a/.gitignore +++ b/.gitignore @@ -21,6 +21,13 @@ uploads/ server/logs/ logs/ +# Installed modules (docs/website/MODULE_SYSTEM.md). Core ships no module, so +# anything here is an operator's install or a developer's scratch copy. The +# directory itself IS tracked, via its README: docker-compose.yml bind-mounts it, +# and a missing bind-mount source is recreated by Docker as root-owned. +modules/* +!modules/README.md + # Operator-supplied spawn atlas artwork. Creature art is never committed: sprites # are extracted from the operator's own UO client .mul/.uop files and are theirs, # not ours to redistribute. The images live under server/uploads/atlas/, already diff --git a/Dockerfile b/Dockerfile index b0395e3..0131633 100644 --- a/Dockerfile +++ b/Dockerfile @@ -21,6 +21,12 @@ RUN if [ -f client/package.json ]; then \ # Persistent uploads + logs live on mounted volumes. RUN mkdir -p /app/uploads /app/logs && chown -R node:node /app/uploads /app/logs +# Installed modules are mounted in too (docker-compose.yml), and .dockerignore +# keeps any local modules/ OUT of the image — a module must never be baked in. +# The directory is still created here so a container run without the mount finds +# an empty, writable modules dir rather than no directory at all. +RUN mkdir -p /app/modules && chown node:node /app/modules + USER node EXPOSE 3000 diff --git a/README.md b/README.md index f96a7d4..93b8a15 100644 --- a/README.md +++ b/README.md @@ -222,6 +222,10 @@ IMAGE_TAG=sha-042a151 docker compose pull && docker compose up -d - Health check: `GET http://localhost:3000/api/health` → `{ "status": "ok" }` - Logs: `docker compose logs -f app` (and `./logs/app.log` on the host) - Stop: `docker compose down` (add `-v` to also wipe the database + uploads volumes) +- Modules: installed into `./modules` on the host (bind-mounted to `/app/modules`), never baked into + the image — an operator adds one to a pull-only deployment without building anything. Adding or + removing one takes a `docker compose restart app`; the scan is synchronous at startup. See + [`modules/README.md`](modules/README.md). **Build the images locally instead of pulling** (offline, or to test an unmerged change) — overlay the dev file, which adds `build:` back: @@ -391,9 +395,10 @@ npm run routes:manifest -- --check # exit 1 if either file is stale (what CI ru The generator walks the live Express stack (runtime introspection, not source parsing — a route's path sits on the line *after* `router.get(`, which defeats greps) and keeps only -`/api/**` and `/.well-known/**` plus the internal listener. The SPA catch-all, `/uploads` and `/brand` -are filesystem-conditional static mounts, not API contract, so they are excluded and the output does -not depend on whether the client has been built. +`/api/**` and `/.well-known/**` plus the internal listener. The SPA catch-all, `/uploads`, `/brand` +and installed modules' `/modules/` chunks are filesystem-conditional static mounts, not API +contract, so they are excluded and the output depends neither on whether the client has been built +nor on which modules are mounted. Two generated files, two very different meanings: @@ -507,6 +512,7 @@ Copy `.env.example` (Compose) or `server/.env.example` (local) and fill in. **`. | `NODE_ENV` | `production` | | | `PORT` | `3000` | server listens on `0.0.0.0:PORT` | | `UPLOAD_DIR` | `/uploads` | where post images are written (`/app/uploads`, volume-mounted, in Compose) | +| `MODULES_DIR` | `/modules` | where installed modules are scanned from (`/app/modules`, bind-mounted, in Compose) | | `DB_HOST` / `DB_PORT` | `db` / `3306` | `db` in Compose; `127.0.0.1` for local dev | | `DB_NAME` / `DB_USER` / `DB_PASSWORD` | `runic_gateway` / `runic` / — | app database credentials | | `DB_ROOT_PASSWORD` | — | MariaDB root (Compose only) | diff --git a/docker-compose.yml b/docker-compose.yml index a7dcd45..43c6189 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -35,6 +35,10 @@ services: DB_HOST: db UPLOAD_DIR: /app/uploads LOG_DIR: /app/logs + # Where the loader scans for installed modules. Same path the code already + # defaults to (/modules, and the repo is /app in the image), set + # explicitly because the bind mount below is what makes it meaningful. + MODULES_DIR: /app/modules depends_on: db: condition: service_healthy @@ -47,6 +51,25 @@ services: # the image, so this mount only matters for custom brand images. Create # ./brand/ on the host and drop assets in; read-only in the container. - ./brand:/app/brand:ro + # Installed modules (docs/website/MODULE_SYSTEM.md). Modules live on a + # mount, NEVER in the image: that is what lets an operator add one to a + # pull-only deployment without building anything. A bind mount rather than + # a named volume because placing a module directory by hand is a supported + # install — `tar -xf uo-1.0.0.tgz -C ./modules` then restart — and that has + # to be doable from the host, not through `docker cp`. + # + # Read-WRITE: the admin panel's install/uninstall unpacks and removes + # directories here from inside the container. + # + # `modules/` is tracked (it ships a README) so the directory exists in the + # checkout with the operator's own ownership. Do not delete it — Docker + # would recreate a missing bind-mount source as root:root and the container + # user could no longer write it. If the app runs as a uid that does not own + # ./modules, `chown 1000:1000 modules` on the host. + # + # Adding or removing a module takes a RESTART: the scan is synchronous at + # require time (MODULE_API.md §4.1), so nothing here is picked up live. + - ./modules:/app/modules # Only the PUBLIC API port (3000) is published. The internal server<->bot # port (INTERNAL_PORT, default 3001) is deliberately NOT listed here, so it # stays reachable only over the private compose network — Pangolin/the public diff --git a/modules/README.md b/modules/README.md new file mode 100644 index 0000000..5afea07 --- /dev/null +++ b/modules/README.md @@ -0,0 +1,75 @@ +# Installed modules + +This directory is bind-mounted into the container at `/app/modules` (see +`docker-compose.yml`). It is where **installed modules** live — the game-specific +routes, tables, screens and nav that are not part of core. Design of record: +[`docs/website/MODULE_SYSTEM.md`](../../docs/website/MODULE_SYSTEM.md); the +normative contract module authors build against is +[`docs/website/MODULE_API.md`](../../docs/website/MODULE_API.md). + +**Core ships no module.** This directory is empty in a fresh checkout, and the +site runs cleanly that way — an empty `modules/` is the normal state for bare +core, not a misconfiguration. Everything below is ignored by git except this +README, which exists so the directory itself is tracked: `docker-compose.yml` +bind-mounts it, and Docker recreates a *missing* bind-mount source as a +root-owned directory the container user cannot write. + +## Layout + +One directory per module, named for its id, each holding a prebuilt bundle: + +``` +modules/ + uo/ + module.json # the manifest the loader reads + server/index.js # registers routes, streams, hooks + server/db/schema.sql # tables, replayed every boot + server/db/purge.sql # only ever run by an explicit purge + client/dist/entry.js # prebuilt ESM chunk, served at /modules/uo/ +``` + +**Nothing here is compiled by the operator.** A module arrives already built — +that is the whole point of the design. There is no install step that runs a +bundler, and none that needs one. + +## Installing a module + +Two supported paths, both writing the same `installed_modules` row: + +- **The admin panel** downloads the bundle from the module's release, verifies it + against its `sha256`, and unpacks it here. +- **By hand**, for a compose-managed host: unpack the bundle into a directory + named for the module id, e.g. `tar -xf uo-1.0.0.tgz -C ./modules`. + +Either way, **adding or removing a module takes a restart.** The loader scans +this directory synchronously at startup (`MODULE_API.md` §4.1); nothing placed +here is picked up by a running server. + +``` +docker compose restart app +``` + +On boot each module is validated, mounted, its schema fragment replayed and its +`onBoot` hook run — reaching `started`, or `startup_failed` with the stage and +reason recorded. A module that fails to start does not stop the site: core, and +every other module, carry on without it. + +## Uninstalling + +Removing a directory and restarting is enough to stop a module serving. Note that +this is *not* the same as an uninstall through the admin panel, which also marks +the row `disabled` — a directory that simply vanishes leaves a row claiming to be +enabled, which the loader records as `startup_failed`. + +A module's **tables and data are retained** in both cases. Dropping them is a +separate, explicit, destructive purge; it is never bundled into an uninstall. + +## Ownership + +The container runs as uid 1000 (`node`) and the admin panel writes here, so the +app must be able to write this directory. It is created by your checkout, with +your ownership. If they differ: + +``` +chown -R 1000:1000 modules +``` diff --git a/server/scripts/routeManifest.js b/server/scripts/routeManifest.js index cdc958a..33b85b0 100644 --- a/server/scripts/routeManifest.js +++ b/server/scripts/routeManifest.js @@ -16,10 +16,11 @@ * derived (only annotated routes appear) and documents intent; this records reality. * * Scope: only `/api/**` and `/.well-known/**` from the public app, plus everything - * on the internal app. Three mounts in app.js are *filesystem* conditional — the SPA - * catch-all `GET *`, the `/brand` static mount and swagger-ui's `/api/docs` static - * assets — so including them would make the output depend on whether CI had built - * the client. Static mounts are not API contract. + * on the internal app. Four mounts in app.js are *filesystem* conditional — the SPA + * catch-all `GET *`, the `/brand` static mount, installed modules' `/modules/` + * chunks and swagger-ui's `/api/docs` static assets — so including them would make + * the output depend on whether CI had built the client, or on which modules were + * mounted. Static mounts are not API contract. * * Usage: * npm run routes:manifest # write server/routes.manifest.json (+ guards) @@ -56,7 +57,8 @@ const GUARDS_COMMENT = '`npm run routes:manifest`.' // Only these prefixes are contract. Everything else the public app serves (SPA -// shell, /uploads, /brand, swagger-ui assets) is static delivery, not API surface. +// shell, /uploads, /brand, /modules, swagger-ui assets) is static delivery, not +// API surface. const PUBLIC_PREFIXES = ['/api/', '/.well-known/'] /**