From f402395fa500c6a39b3f8c8fbf5c5d56f2fe9aa6 Mon Sep 17 00:00:00 2001 From: wtclaude Date: Tue, 11 Aug 2026 01:50:37 -0500 Subject: [PATCH 1/2] docs(website): the server half does not slice, and why MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found before writing any of it. §2.7.1 split the server extraction five ways by feature; it does not divide, because two contract rules compose. A mount prefix is claimed whole -- ownedByCore probes the live tier router and registerRoutes validates single-segment prefixes -- and /admin/shard is one 386-line router carrying 25 routes across atlas, clilocs, shard-ops, visibility, market and account links. Meanwhile a model cannot be shared across the boundary (§5.1), so it moves with the last route that consumes it. Take the closure and every prefix is in it: /public/atlas holds shardAtlas with /admin/shard, which holds shardState/shardEvents/shardMarket with /public/shard, which holds uoLinkConfig with /admin/uo-link, and shardClilocs/shardLinks with /player/shard. Landing any one of the old slices alone would either strand core importing modules/uo/ -- what acceptance criterion 2 forbids -- or delete routes core is still serving. Giving the admin routes their own prefixes would divide it and is rejected: it changes API URLs, which §1.2 promises not to do, and not hypothetically. The shipped Android app calls POST /api/v1/admin/shard/kick, /ban, /unban, /broadcast and the three /pages routes. So the table is now six slices: 0 the skeleton (done), 1 the whole server half as ONE PR per repo structured as five commits along the old slice lines, 2-4 the client halves, 5 close the phase. The client half is unaffected and still slices cleanly -- the registry takes routes per area, with no prefix atomicity and no shared models, the same asymmetry that let the two halves be separated at all. Co-Authored-By: Claude --- website/MODULE_API.md | 2 +- website/MODULE_SYSTEM.md | 62 ++++++++++++++++++++++++++++++++-------- 2 files changed, 51 insertions(+), 13 deletions(-) diff --git a/website/MODULE_API.md b/website/MODULE_API.md index de6c820..4dd2c49 100644 --- a/website/MODULE_API.md +++ b/website/MODULE_API.md @@ -866,7 +866,7 @@ to defend is *structural*: core must not name a module's files, import them, rou declare their symbols. It can talk about them in English. Core's UO-flavoured default copy is dealt with directly instead, as `MODULE_SYSTEM.md` §2.7.1's -slice 8 — a rewrite with its own review, not an exemption. +slice 4 — a rewrite with its own review, not an exemption. ### 5.3 Zero-line route manifest diff (CI, both repos) diff --git a/website/MODULE_SYSTEM.md b/website/MODULE_SYSTEM.md index 029a12e..889f7e7 100644 --- a/website/MODULE_SYSTEM.md +++ b/website/MODULE_SYSTEM.md @@ -777,21 +777,59 @@ Each slice is one `module-uo` PR (adds), one `website` PR (deletes), and one `do | # | Slice | Moves | | --- | --- | --- | -| 0 | **The bundle skeleton** | `module.json`, both `package.json`s, `server/index.js` registering nothing, the Vite library build + the four shared-dep shims, an empty `schema.sql`/`purge.sql`, CI armed, the §5.1 zero-internal-imports check. `website` untouched. | -| 1 | **Atlas + clilocs** | `shardAtlas`/`shardClilocs` models, `clilocParse`/`clilocSource`/`spawnAtlasParse`/`spawnAtlasSource`, `public/atlas.*`, `admin/shardAtlas.controller`/`shardClilocs.controller`, `scripts/importSpawnAtlas.js`, the art JSON | -| 2 | **The live shard** | `shardState`/`shardEvents`/`shardVisibility`/`uoLinkConfig` models, `uoLinkClient`/`uoLinkSocket`/`shardIngest`/`shardBroadcast`/`shardPush`/`shardVisibility` utils, `config/shardStreams.js`, `public/shard.*`, `admin/shard.router`/`shardOps`/`shardVisibility`/`uoLink.*` | -| 3 | **Market** | `shardMarket` model, `shardSales` util and their routes | -| 4 | **Account links + the extension slot** | `shardLinks` model, `player/shard.*`, `admin/usersShard.*` minus `getUser` — the first real filling of `admin.users.detail` | -| 5 | **The news leg** | `newsGump.js`, `shardAnnounce.js`, the town-crier `registerAnnounceLeg` | -| 6 | **Public pages** | `Shard`, `ShardActivity`, `Rules`, `Atlas`, `AtlasCreature`, `ChampSpawns`, `Market`, `MarketVendor`, `Governors`, `Guilds`, `Houses`, `Leaderboards`, `PlayersOnline`, `VendorSales`, `data/cityCrests.js`, `lib/shardEvents.js`, `lib/useShardFeed.js`, and their public nav rows — under `/uo/*` per §2.8 | -| 7 | **Admin + player pages** | `ShardAdmin`, `ShardOps`, `ShardVisibility`, `SpawnAtlas`, `HousesAdmin`, `AdminCharacter(s)`, `PlayerCharacter(s)`, `GameAccounts`, `CharacterSheet`, `CharacterStats`, `ShardAccountActions`, `CreateGameAccountForm`, `useShardFeatures` and the `useShardFlags` feature provider — under `/admin/uo/*` and `/player/uo/*` | -| 8 | **De-UO core's copy** | `About`, `Screenshots`, `Website`, `SiteFooter`, `heroLayout`'s defaults, `api/client.js`'s `shard`/`atlas` namespaces, and the comments in `navOverrides.js` — plus the §5.2 CI grep that keeps them out | -| 9 | **Close the phase** | `module-uo`'s frozen route manifest and release workflow; `docs/modules/uo/` and `docs/modules/rust-dryrun.md` | +| 0 | **The bundle skeleton** | `module.json`, both `package.json`s, `server/index.js` registering nothing, the Vite library build + the four shared-dep shims, CI armed, the §5.1 zero-internal-imports check. `website` untouched. | +| 1 | **The whole server half** | 40 files / ~9,674 lines, 25 of core's 82 test files, 27 of its 68 tables — every UO model, util, router and controller, `config/shardStreams.js`, `scripts/importSpawnAtlas.js` and the art JSON. **One merge, five commits** (below). | +| 2 | **Public pages** | `Shard`, `ShardActivity`, `Rules`, `Atlas`, `AtlasCreature`, `ChampSpawns`, `Market`, `MarketVendor`, `Governors`, `Guilds`, `Houses`, `Leaderboards`, `PlayersOnline`, `VendorSales`, `data/cityCrests.js`, `lib/shardEvents.js`, `lib/useShardFeed.js`, and their public nav rows — under `/uo/*` per §2.8 | +| 3 | **Admin + player pages** | `ShardAdmin`, `ShardOps`, `ShardVisibility`, `SpawnAtlas`, `HousesAdmin`, `AdminCharacter(s)`, `PlayerCharacter(s)`, `GameAccounts`, `CharacterSheet`, `CharacterStats`, `ShardAccountActions`, `CreateGameAccountForm`, `useShardFeatures` and the `useShardFlags` feature provider — under `/admin/uo/*` and `/player/uo/*` | +| 4 | **De-UO core's copy** | `About`, `Screenshots`, `Website`, `SiteFooter`, `heroLayout`'s defaults, `api/client.js`'s `shard`/`atlas` namespaces, and the comments in `navOverrides.js` — plus the §5.2 CI grep that keeps them out | +| 5 | **Close the phase** | `module-uo`'s frozen route manifest and release workflow; `docs/modules/uo/` and `docs/modules/rust-dryrun.md` | + +##### Why the server half cannot be sliced — found 2026-08-11, before writing any of it + +The table above used to run to ten slices, with the server half split five ways by feature. It does +not divide, and the reason is that two contract rules compose: + +- **A mount prefix is claimed whole.** `ownedByCore` probes the live tier router and `registerRoutes` + validates single-segment prefixes, so `/admin/shard` moves as one unit — and it is a single + 386-line router carrying 25 routes that span atlas, clilocs, shard-ops, visibility, market *and* + account links. +- **A model cannot be shared across the boundary** (§5.1), so a model moves with the *last* route + that consumes it. + +Take the closure and every prefix is in it: + +``` +/public/atlas ──shardAtlas── /admin/shard ──shardState,shardEvents,shardMarket── /public/shard + │ │ + shardClilocs,shardLinks uoLinkConfig + │ │ + /player/shard /admin/uo-link +``` + +Landing any one of the old slices alone would either strand core importing `modules/uo/` — which is +precisely what acceptance criterion 2 forbids — or delete routes core is still serving. + +**Giving the admin routes their own prefixes would divide it, and is rejected.** `/admin/atlas` and +`/admin/clilocs` alongside a slimmer `/admin/shard` would make the closure fall apart. It also +changes API URLs, which §1.2 promises not to do — and not hypothetically: the shipped Android app +calls `POST /api/v1/admin/shard/kick`, `/ban`, `/unban`, `/broadcast` and the three `/pages` routes +(`data/api/AdminApi.kt`). A prefix rename is a client break, and the API surface is frozen for +exactly this reason. + +**So slice 1 is one PR per repo, structured as five commits** along the old slice lines, reviewable +one at a time while landing atomically: atlas + clilocs · the live shard · market · account links and +the `admin.users.detail` slot · the town-crier leg. The alternative considered was stacked PRs into a +per-repo integration branch; it buys PR-level granularity for ten extra PRs and two long-lived +branches, and commits give most of the same reading order for none of it. + +**The client half is unaffected** and still slices cleanly: the client registry takes routes per +*area*, with no prefix atomicity and no shared models — which is the same asymmetry that let the two +halves be separated in the first place. **Criterion 1 is a grep over code, not over prose** — see [API §5.2](MODULE_API.md#52-zero-uo-identifiers-in-core-ci-website-repo) for what that means precisely. Core's marketing copy says "shard" in a dozen places, and a literal word grep would have made every one of them a CI failure while proving nothing about the boundary. -Slice 8 rewrites that copy anyway, because a core that still reads as a UO site is not the +Slice 4 rewrites that copy anyway, because a core that still reads as a UO site is not the game-agnostic platform this workstream is for — but it is a deliberate piece of work with its own review, not an exemption hidden in a grep pattern. @@ -827,7 +865,7 @@ of what to catch, and the entry point's comment explaining why a module must nev `require('express')`. A check that cannot survive being described is one people stop writing comments around, so it strips comments and template literals with a character walk rather than a regexp (a URL in a string contains a comment opener; a comment contains quotes) and carries its own -test suite. The same applies to slice 8's §5.2 grep, which will be read by a codebase that discusses +test suite. The same applies to slice 4's §5.2 grep, which will be read by a codebase that discusses modules constantly. **One thing to know before running a module locally: the loader skips a *symlinked* module directory From f3a623108437bf0588a84551f27b1b17fd209540 Mon Sep 17 00:00:00 2001 From: wtclaude Date: Tue, 11 Aug 2026 12:11:09 -0500 Subject: [PATCH 2/2] docs(website): record slice 1, and MODULE_API 1.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The whole server half is out: 40 files, ~9,674 lines, 27 of 68 tables. The acceptance criterion held exactly -- core's manifest goes 228 to 158 public routes and the 70 that left reappear byte-identical once the module loads, with routes.guards identical across all 228. The contract grew to 1.1.0: ctx.activity.log, ctx.users.getById, ctx.site.baseUrl, ctx.middleware.rateLimit + accountChangeLimiter, and a fourth registry, registerPostHook. Each is documented with why it could not be vendored, because that reasoning is the useful part -- an admin action a module performs belongs in core's ONE audit log, a second rate-limit store is a limit enforced by two counters, and core's CMS was calling a UO file directly. §2.7.1 gains the slice record: the core.js port mechanism and its consequence (require order is load-bearing), the vendoring line (pure leaf helpers may be copied, security controls may not), the two core defects the extraction exposed, the one deliberate behaviour change, and the one test that looked like it should move and should not. Co-Authored-By: Claude --- website/MODULE_API.md | 37 ++++++++++++++++++++++++- website/MODULE_SYSTEM.md | 60 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 96 insertions(+), 1 deletion(-) diff --git a/website/MODULE_API.md b/website/MODULE_API.md index 4dd2c49..6d305bd 100644 --- a/website/MODULE_API.md +++ b/website/MODULE_API.md @@ -26,9 +26,20 @@ here extends the contract first, in this file, before the module is written agai Core exports a single integer-major semver string from `server/src/modules/version.js`: ```js -const MODULE_API_VERSION = '1.0.0' +const MODULE_API_VERSION = '1.1.0' ``` +The client half carries the same number (`client/src/modules/version.js`) and a test asserts the two +agree. Duplicated rather than fetched because the value has to be on `window.__rg` before the first +module chunk evaluates, which is earlier than any network round trip could answer. + +**1.1.0 — Phase 3 slice 1.** `ctx` gained `activity.log`, `users.getById`, `site.baseUrl`, and +`middleware.rateLimit` + `middleware.accountChangeLimiter`; `api` gained `registerPostHook`. +Additions only. Each exists because module-uo's extraction needed it and none could be vendored — an +admin action a module performs belongs in core's one audit log, the extension slot needs the user its +prefix names, §2.7 forbids a module reading core's `APP_BASE_URL`, a second rate-limit store is a +limit enforced by two counters, and core's CMS was calling a UO file directly. + Every `module.json` declares a `coreApi` semver **range**. The loader checks it at boot, before it requires a line of module code, and a mismatch fails that module loudly into `startup_failed` (§4.4) with the two versions in the reason. It never silently proceeds. @@ -128,6 +139,11 @@ module-uo does not need is on the list. | `ctx.uploads` | `{ upload, UPLOAD_DIR, MIME_EXT }` | `admin/imageUpload.js` | atlas art import | | `ctx.posts` | `{ listAll, getById, linkAnnounceJob, markAnnounced }` | `model/posts` | `newsGump.js:108`, `announceWorker.js:58` | | `ctx.paths.moduleRoot` | absolute path to `modules//` | loader | atlas art, cliloc files | +| `ctx.activity.log` | `({ req, action, detail }) => Promise` | `model/activity` | every admin UO controller (1.1.0) | +| `ctx.users.getById` | `(id) => Promise` | `model/users` | `usersShard.controller` (1.1.0) | +| `ctx.site.baseUrl` | getter, string with no trailing slash | `APP_BASE_URL` | `shardAnnounce` (1.1.0) | +| `ctx.middleware.rateLimit` | `(options) => middleware` | `middleware/rateLimit` | the market search (1.1.0) | +| `ctx.middleware.accountChangeLimiter` | middleware | `middleware/rateLimit` | `player/shard.router` (1.1.0) | | `ctx.moduleId` | the id from `module.json` | loader | log tags, table checks | Three narrowings from `MODULE_SYSTEM.md` §2.1, all deliberate: @@ -159,6 +175,7 @@ api.registerRoutes({ public: {...}, admin: {...}, player: {...} }) api.registerExtension(slot, router) api.registerNotificationStreams(streams) api.registerAnnounceLeg({ leg, label, dispatch, classify }) +api.registerPostHook({ onSaved, onDeleted }) api.onBoot(async (ctx) => {}) api.onShutdown(async () => {}) ``` @@ -240,6 +257,24 @@ next_attempt_at)` and `leg` is a stored value. The parent `status` rollup is ove legs — done when every leg delivered, failed when every leg gave up, partial in between; and `done` when a job has no legs at all, since nothing is left to deliver. +**`registerPostHook({ onSaved, onDeleted })`** — added in API 1.1.0. Core's CMS is the only writer of +posts, and a module may need to mirror one somewhere core knows nothing about. `onSaved` receives +`{ post, transition }` — the same transition `registerAnnounceLeg` fires on — and `onDeleted` +receives `{ post, id }`. Both are optional; a registration with neither is refused, since it is a +subscription that can never fire. One hook per registrant. + +Every hook is awaited and none may throw past core: a subscriber's failure is logged and costs +neither another subscriber nor the save itself. A sidecar hiccup breaking a post edit would be a +worse bug than a stale mirror. + +**It is deliberately not part of `registerAnnounceLeg`**, which fires on the same transition. A leg +is a one-shot *delivery* with retry and classification; a post hook maintains idempotent *state*, has +to run on delete as well as save, and refreshes silently on an edit. Overloading the leg would have +meant a `dispatch` that must not be retried and a `classify` that means nothing. + +Before it existed, core's post controller required `utils/newsGump` directly — core's publish path +naming a UO file, and the last thing binding core to the module. + **`onBoot(fn)` / `onShutdown(fn)`** — §2.5. ### 2.5 Lifecycle diff --git a/website/MODULE_SYSTEM.md b/website/MODULE_SYSTEM.md index 889f7e7..4d5563e 100644 --- a/website/MODULE_SYSTEM.md +++ b/website/MODULE_SYSTEM.md @@ -868,6 +868,66 @@ regexp (a URL in a string contains a comment opener; a comment contains quotes) test suite. The same applies to slice 4's §5.2 grep, which will be read by a codebase that discusses modules constantly. +#### Slice 1 — the whole server half (Module-uo#3 + website#137, 2026-08-11) + +40 files, ~9,674 lines, 27 of 68 tables, 25 of 82 test files. Core no longer contains anything that +knows what a shard is. Three commits per repo, readable in order. + +**The acceptance criterion held exactly.** Core's `routes.manifest.json` goes 228 → 158 public +routes, and the 70 that left reappear byte-identical once the module is loaded — proved by generating +the manifest against core+module and diffing it against the pre-extraction file: zero missing, zero +added, and `routes.guards.json` identical across all 228, so no auth gate moved either. + +**`server/core.js` is the port mechanism and the shape is the finding.** Ported code requires its +dependencies at file scope, which runs before `register()` and therefore before any `ctx` exists — so +every member of that file is a stable function resolving `ctx` when *called*, and nothing may be +destructured off `ctx` at init either, because core is free to hand over a getter. That kept the port +to a one-line import change per file instead of a signature change per function. Its consequence: +**require order is load-bearing.** A router does `const express = core.express` at its own file +scope, so `core.init(ctx)` must run before the first `require` under `router/`, and the module's +entry point requires its routers inside `register()` for exactly that reason. + +**The contract grew to 1.1.0**, four members, none of which could be avoided: +`ctx.activity.log` (an admin action a module performs belongs in core's *one* audit log — a module +with its own is a second place to look, which means a place nobody looks), `ctx.users.getById`, +`ctx.site.baseUrl`, and `ctx.middleware.rateLimit` + `accountChangeLimiter`. The rate-limit split is +worth restating: a module states its own window and cap because it knows what its endpoints cost, and +takes the plumbing from core so there is one `express-rate-limit` in the process and one place a +breach is logged. + +**`registerPostHook` is the fourth registry and the last coupling removed** — see +[API §2.4](MODULE_API.md#24-api--what-the-module-registers). + +**What was vendored, and what deliberately was not.** `deriveExcerpt` came across as nine lines of +pure text handling; core's **sanitiser** sitting beside it did not, because a second copy of a +security control diverges silently the moment either is fixed. That is the line: pure leaf helpers +may be copied, controls may not. + +**Two defects the extraction exposed, both in core.** The loader matched `CREATE TABLE` against the +**raw** fragment, so a schema file whose header says "every CREATE TABLE carries IF NOT EXISTS" was +rejected for a prefix violation on a table called `carries` — the same class as slice 0's boundary +check failing on its own documentation, and now fixed on both scans by reading split statements. And +the atlas art map resolved `../../../db/data`, correct in core and pointing outside `server/` in the +module: a path that happens to resolve is exactly what survives a green suite, because the +absent-file branch returns `{}` and looks like the normal case. It was caught by the integration run, +not by tests. + +**One deliberate behaviour change.** `uoLinkSocket.start()` and the sidecar health probe used to run +*after* the listener bound and now run before it, because `onBoot` does. `start()` returns as soon as +the reconnecting client is armed, but the probe is a real HTTP call, so it is fired and **not** +awaited — an unreachable sidecar must not hold the site closed. Reporting that the bridge is down is +diagnostics; being up is not a precondition for serving a page. + +**One test stayed that looked like it should move.** `playerRouteAccess.test.js` guards a real past +bug — an admin 403'd off their own characters — through a now-module-owned URL, but the *guarantee* +is core's: `/player/*` is role-agnostic self-service. It stays and asserts that through +`/player/appeals`. Moving it would have left core with no test of its own tier rule, which is +precisely what regressed once before. + +**A note for anyone running core's suite locally: remove `modules/uo` first.** With a module +installed the manifest tests fail correctly — core's committed manifest is core-only, and the live +stack has the module's routes on it. + **One thing to know before running a module locally: the loader skips a *symlinked* module directory silently.** `readdirSync(…, { withFileTypes: true }).filter(e => e.isDirectory())` reports a Windows junction as a symlink, so a module linked rather than copied into `modules/` is simply not there,