e59a68c152deaf391fad2395061183ed9fd65bb5
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| cfd1cb3c3c |
feat(engagement): let a module ship its own templates and rules (Phase 11b)
Phase 11a declared 24 triggers and stopped where the plan said it would. Standing
11b up found that the next sentence — "24 rules, all enabled = 0; bespoke template
bodies" — described work with no mechanism to land in: templateSeeds.js and
coreRules.js are core files with core arrays in them, and there was no
registerTemplates or registerRules anywhere in registries.js.
So a module could say what an event's payload was and could never say what the
mail should read like. That is tolerable for one trigger and not for a catalogue,
and it is decisive once the bodies carry domain prose core must not contain (§5.2).
Adds api.registerEngagementSeeds({ templates, ruleGroups }) — MODULE_API 1.9.0.
The module supplies data; core keeps seedOne's customized skip, its seed_version
comparison and the block registry's validation, which is the whole argument for a
registry over the ctx.query a module already holds: a copy of any of those living
outside engagement/ would drift the first time core improved the original, and the
drift would surface as a mail somebody already received.
The two halves behave differently, deliberately:
- Templates re-ensure on every boot, so a bumped seedVersion reaches every
deployment except the ones where an operator edited that row.
- Rule groups are ONE-SHOT, each under its own settings guard — re-ensuring
would resurrect a rule an operator deleted and reset one they enabled. This is
11a's seed-key finding stated as an API rather than as a warning: a rule
appended to an existing group reaches fresh installs only, and one that must
reach stamped deployments takes a new group key.
Three prohibitions, each a shipped mistake that would only surface as mail: a
seeded rule is always enabled = 0 (Q3's invariant, ignored rather than refused so
a typo cannot take a module offline at boot); a module may not mark a template
protected; and a rule may only name its own trigger ids and its own or core's
template keys, with template keys namespaced because the key column is UNIQUE.
Runs from modules/lifecycle.js boot() rather than seedDefaults(), and that is
forced rather than chosen: server.js seeds before it requires app.js, and
requiring app.js is what runs the loader — at the moment core seeds, no module has
registered anything. Placed after the installed_modules reconcile (so a disabled
or failed module is skipped) and before the onBoot dispatch (so a module warming a
cache may assume its rules exist).
16 new tests; 1549 core tests green; check:modules clean.
Refs docs#/ENGAGEMENT.md Phase 11b decision 7.
Co-Authored-By: Claude <noreply@anthropic.com>
|
|||
| cecd72915f |
feat(teams): the slash-command seam, and the first command through it
Phase 7 of TEAMS.md. `api.registerSlashCommands` stops throwing: a module registers a command's DEFINITION and its HANDLER together, the bot pulls the definitions over the internal listener and runs none of our code, and the handler executes here — forced by the bot container having no `modules` volume, and the right boundary anyway. Registration validates what Discord would reject as a batch (names, description lengths, the four option types, required-before-optional), because the bot registers the whole set in one PUT and a single bad entry costs every command including the bot's own. Commands are not namespaced under their owner — there is no dot in Discord's name grammar — so collisions are first-come with the holder named. The dispatcher is the access boundary: `linked` has no Discord equivalent, so the platform-side permission default can only ever be advertising. It resolves the actor by `auth_providers.kind` rather than the id slug, treats a banned account as unlinked, bounds a handler under the bot's own timeout, and keeps `ok` outside the envelope so a handler cannot forge it. Liveness is asked at both the pull and the dispatch. The registries have no removal path, so a module an operator disables at runtime would otherwise keep a live handler behind a command Discord still advertises. Co-Authored-By: Claude <noreply@anthropic.com> |
|||
| 92631347f9 |
feat(teams): the reconciler, its four refusal gates, and ctx.teams (API 1.6.0)
Core's projection of the module's Teams, kept in step (docs/website/TEAMS.md
§2.4), plus the two ctx members a module pushes through.
The four gates are the file, and each is invariant 1 in a different costume --
module unavailability is staleness, never emptiness:
1. getTeams() not ok -> record the failure, touch NOTHING, return.
2. ok but empty, core holds >=1 -> quarantine; apply only if the NEXT
authoritative answer, an interval later,
agrees.
3. getTeamMembers() not ok -> that Team's roster untouched and stale; the
other Teams sync normally.
4. ok but zero members, had some -> the same two-strikes quarantine, per Team.
Gates 2 and 4 exist because an authoritative-looking empty answer during a cold
start is the one failure indistinguishable from a real wipe. "Every Team on the
shard disbanded at once" costs one interval to confirm; getting it wrong empties
every roster on the site.
Events are an optimisation, never the source of truth. Member and leadership
deltas apply at once for a Team core already knows; team.created and
team.disbanded only ask for a run. §2.2 scopes archival to an authoritative full
list, so a repeated or spurious disband event costs a reconcile rather than a
Team -- and a Team invented from a delta would have no name, no roster and no
leaders anyway.
Two columns TEAMS.md did not contemplate, both on `teams`:
- roster_synced_at, because team_sync_state holds one row per MODULE and gate 3
leaves ONE Team behind while the others sync. Without a per-Team stamp that
Team's page would report the module's last success as its own -- exactly the
staleness the gate exists to surface.
- members_empty_since, gate 4's per-Team quarantine. The twin of
team_sync_state.pending_empty_since, which is per module and cannot express it.
One real bug found by its own test. The roster upsert was writing is_leader, so a
refused getTeamLeaders() left every member demoted -- the roster had already
written `leader: false` before the authoritative call was even made. §2.5 is
explicit that path 2 is answered by getTeamLeaders(), so is_leader is now set on
INSERT only (seeding a Team so it is not leaderless while that call fails) and
moved afterwards by setLeaders() alone. Two writers for one column was the whole
defect.
MODULE_API_VERSION 1.6.0 on both halves -- they state one contract and a module
declares one coreApi range. The number covers the whole Team surface per Part 11;
the members arrive by phase. registerTeamProvider, ctx.teams.publish and
ctx.teams.reconcile are live. ctx.teams.activity.push (§4, phase 3) and
api.registerSlashCommands (§7.1, phase 7) are present and THROW with a sentence
naming their phase, rather than being absent or silently accepting data into
tables that do not exist yet.
39 tests here, and the ctx surface guard in moduleLoader.test.js updated -- it
caught the addition, which is what it is for. Server 809 passed, client 192
passed, 0 failed.
Refs docs/website/TEAMS.md §2.2, §2.3, §2.4, Part 11, Part 12 phase 2
Co-Authored-By: Claude <noreply@anthropic.com>
|
|||
| 732927a6bb |
fix(modules): three defects a real install exposed (phase 4, slice 1)
Standing the slice-2 screen up against a live server and installing the
published module-uo v0.3.0 through it found three things, none of which any
unit test in this repo could have caught. Two of them are older than this
phase.
1. The boot refresh nulled every install's provenance
--------------------------------------------------------
`installed_modules.source` and `.sha256` exist so the admin panel can say
where a module came from. They never survived a restart.
`lifecycle.boot()` re-records every scanned module with no source and no
sha256 -- correctly, because a scan finds a directory and never where it came
from -- and `upsert` assigned both columns unconditionally. So an install's
provenance lasted exactly until the restart that install asked for, and the
screen then described a module installed from a URL as "placed on the volume
by hand". Verified live: install, restart, provenance gone.
Nothing could have caught it before now. Phase 4 wrote the first non-null
value these columns had ever had, so lifecycle.js's comment asserting that
"recordInstalled leaves what it is not given" described an intention rather
than the statement below it -- and modules.model.test.js's fake reproduced
the defect faithfully, assigning unconditionally just like the SQL.
Fixed with COALESCE(VALUES(col), col): a value overwrites, a NULL leaves what
is there. The fake now matches, and two tests pin both directions -- a boot
refresh must not wipe it, and a re-install from a new URL must still replace
it, or the column would become write-once and an upgrade would for ever show
where the first version came from.
2. The restart killed the server on Windows instead of stopping it
------------------------------------------------------------------
The route called `process.kill(process.pid, 'SIGTERM')` to reach server.js's
graceful-shutdown handler. That works on Linux. **Windows has no POSIX
signals, and Node documents SIGTERM there as unconditional termination of the
target process** -- so on a Windows host the restart killed the server
outright: no module onShutdown, no listener close, no pool close, no log
flush. Observed exactly that: the process was gone and the shutdown handler
had logged nothing at all.
`process.on('SIGTERM', ...)` is an ordinary EventEmitter listener, so
`process.emit('SIGTERM')` reaches the same handler on every platform without
involving the OS. One shutdown path, still; it just gets there by an event.
Deployment is Linux containers and would never have shown this. Development
is not, and neither is the smoke that found it.
The test was worse than useless: it stubbed `process.kill` and asserted it
had been called with SIGTERM, which is precisely the call whose MEANING
differs by platform. It now waits for the SIGTERM EVENT -- what server.js is
actually subscribed to -- so a pass here means the handler would run.
3. `present()` did not publish the running version
--------------------------------------------------
An upgrade writes new files and a new row while the old code stays loaded, so
the row's version is a promise about the next boot rather than a description
of this one. Adds `liveVersion` from the loader beside `liveState`, so the
screen can tell the two apart instead of reporting the new version as running.
723 server tests (+2), manifest and OpenAPI both unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
|
|||
| b30e82cde2 |
feat(modules): install, uninstall, purge and restart (phase 4, slice 1)
The consumer half of a release module-uo's CI has been publishing since
phase 3 closed. Before this, core had the installed_modules provenance
columns and no code that could ever fill them: nothing fetched, verified,
unpacked, removed or purged anything, and there was no admin route at all.
Adds modules/archive.js, modules/install.js, schema.runPurge(),
lifecycle.stop(), loader.stopHook(), and /api/v1/admin/modules with eight
routes. 797 server tests (+76), manifest 158 -> 166 + 2 internal, OpenAPI
gains 8 operations and loses nothing.
Reject, never sanitise
----------------------
The download is the easy part: an https-only allowlist re-checked on every
redirect hop, a declared sha256 compared against the bytes that arrived, and
a byte cap. Unpacking is where the archive chooses the filenames, and core
writes into a directory bind-mounted from the host, so an escape is not
confined to the container.
archive.js inspects the whole archive before a byte is unpacked and refuses
absolute and drive-absolute paths, `..` segments, NUL bytes, backslashes,
anything that is not a regular file or a directory, more than one top-level
entry, and anything over the entry or byte caps. Refusing symlinks and
hardlinks outright is what keeps this off the majority of node-tar's
published advisories rather than depending on the library to contain them.
That two-pass shape is load-bearing, and it was measured rather than assumed:
extracting an archive whose fourth member escapes upward throws under
node-tar 7.5.22 -- and leaves the first three members on disk. The loader
scans that directory at require time on the next boot, so a half-unpacked
module is a module. Everything therefore happens in a scratch directory that
is removed on any failure, and the move into place is the last step.
`tar` is pinned to ^7.5.22 rather than the ^6 that installs by default: 6.x
is flagged critical, and reading the advisory list is what the file's header
now says out loud -- almost all of it is hardlink or symlink traversal and
PAX header interpretation differentials, which is exactly this feature's
threat model.
Two things the plan had wrong
-----------------------------
The bundle's top-level directory is `module-uo-<version>`, not the module id
-- so "the top-level name must equal the id" was checked against nothing real.
The extractor strips that level instead, because its name belongs to whoever
published the bundle and the directory it lands in is core's. What is checked
instead is the unpacked module.json: a manifest promising `uo` and delivering
something else is refused rather than installed under the name it promised.
And purge cannot be a follow-up action (decision 5): purge.sql lives inside
the directory uninstall deletes. It is offered in the uninstall flow and as a
standalone action on a still-installed module, and the standalone one refuses
unless the module is already disabled -- dropping tables under something that
is still serving leaves it answering out of a world that no longer exists.
Disable now means stopped
-------------------------
lifecycle.stop() dispatches that one module's onShutdown before flipping the
guard, so a module an operator switches off actually releases its sockets and
closes its streams instead of merely becoming unreachable. The hook runs
first and the state moves after it, because while onShutdown runs the module
is still `started` and that is the only state in which its routes and the
world it is tearing down agree. A hook that throws does not stop the disable
-- the opposite of the boot path's rule, and deliberately.
Enable is not its mirror and there is no start(id) beside it. There is no
onBoot re-dispatch and the hooks were never promised re-entrant, so enable
moves the row and the restart route starts it. A test pins that enable does
not touch the loader, because "fixing" it is a one-line change that would put
a module with closed sockets back on the nav.
Restart raises SIGTERM against its own process rather than calling the
shutdown path directly, so server.js's handler stays the one graceful-shutdown
path and this route cannot drift from it.
The allowlist bootstraps from MODULE_SOURCE_HOSTS into a settings row and is
admin-managed after that (decision 6); seedDefault is INSERT IGNORE, so
changing the variable on an existing deployment is a no-op by design. An empty
list forbids every install rather than allowing every host -- the safe
direction for a value someone might blank by accident.
Verified against the real v0.3.0 release
----------------------------------------
Not a fixture: fetched the published install manifest over the real Gitea
host and its redirect chain, verified the sha256, inspected and unpacked the
252,517-byte artifact to 82 files, and then booted core against the result --
the module registered its five mounts, seven streams and eight capabilities
and resolved its client chunk, with no scratch directory left behind.
Two defects this slice's own tooling caught, both of which had already been
written down as classes:
- the controller destructured runPurge at require time, capturing the
function rather than the module, which made the one dependency whose
ORDER matters the one that could not be substituted;
- two swagger annotations carried an apostrophe inside a quoted string,
dropped silently by swagger-autogen before slice 5 taught it to fail loudly.
Co-Authored-By: Claude <noreply@anthropic.com>
|
|||
| 21196466ed |
feat(modules): boot/shutdown hook dispatch and the installed_modules reconcile
Phase 2, PR 5 of docs/website/MODULE_SYSTEM.md 2.7. api.onBoot/api.onShutdown stop throwing, server.js gains one call on each side, and the 2.4 state machine finally runs against real outcomes -- which is what makes 4.5's `disabled` 404 leg reachable for the first time. Dispatch and reconcile live in src/modules/lifecycle.js rather than in the loader, for the reason the schema replay does: routeManifest.js and swagger.js both require app.js against a dead pool, so the loader may not reach the database. The two halves meet at exactly one function, loader.setState(), so the in-memory record the dispatch guard reads and the row the admin panel reads are moved together and cannot disagree. Four decisions, all recorded in MODULE_API.md 2.5 and 4.4: - The loader classifies its failures by 4.3 step, so failure_stage says where a module broke instead of being a column nothing ever filled. The four steps readManifest covers in one pass label themselves; the rest are inferred from how far load() had got, and an unlabelled throw is recorded against the step that was running rather than guessed at. - A row whose directory is gone is marked startup_failed rather than left claiming `enabled` -- the boot reset has just moved it there, and a row claiming to be enabled for a module that is not on the volume is the one state that is simply untrue. An uninstall leaves `disabled`, which the reset never touches, so this catches only a hand-deleted directory. - Core's eight UO boot call sites stay in server.js until Phase 3. Unlike a registered announce leg, a boot call site already has somewhere to live, so moving it now would be extraction done early in a phase whose exit criterion is that nothing changes. - onBoot gets no timeout. Shutdown races a SIGKILL and boot does not, and a slow onBoot delaying the listener is the contract's promise to a module that must warm up before it serves. The operator's switch wins over everything: a disabled module is guarded, not booted, and does not have its failure re-recorded, or an outcome would silently switch it back on next boot. Every database write in the reconcile is individually caught -- a row that will not update is worse reporting, never a failed boot. 900 tests pass (17 new). routes.manifest.json is unchanged at 229 routes and the OpenAPI spec regenerates byte-identical. Co-Authored-By: Claude <noreply@anthropic.com> |
