docs(events): the Event System record — the cutover step 16b missed (edgemain) #232

Merged
whitlocktech merged 46 commits from edge into main 2026-09-10 02:55:11 +00:00
2 changed files with 134 additions and 4 deletions
Showing only changes of commit 261cce6423 - Show all commits

View File

@@ -473,6 +473,11 @@ surface would ever show them: `EVENT_STEP_MAX_ATTEMPTS` (3), `EVENT_STEP_RETRY_M
action's declared `budgetMs` plus a minute, because a registry that lets an action declare an hour
would otherwise have its steps reclaimed and re-dispatched fifty-nine minutes before they answered.
**A run's status is re-read between steps, not only at the top of a tick** (Phase 3). One tick drains
up to `EVENT_STEPS_PER_TICK` steps from one run, so without this a pause pressed mid-batch would be
answered by dispatching another two dozen steps — which is not a pause. It is one indexed read per
step, against a control whose entire value is that it takes effect at once.
**Serial within a phase.** The runner works the lowest-`seq` step of the current phase that is not
terminal, and does nothing with the one after it until that one finishes. This is the only reading
under which `core.wait` means anything, and the only one under which a cue can gate what follows it.
@@ -851,11 +856,40 @@ dimensions' labels. `check:modules` already fails core's build on a UO identifie
- **A GM cue step.** A core action whose implementation is "post the instruction, wait for a human to
confirm they did it in-client, then advance". No module, no protocol, no world write — and it makes
the entire system useful on day one, with a GM doing the target-driven parts in-client exactly as
`ADMIN_CONTROLS.md` §8 argued they should.
`ADMIN_CONTROLS.md` §8 argued they should. **Both halves exist as of Phase 3**: the action parks
the step in Phase 2, the run console's confirm ends it.
- **Live controls that are honest.** Pause, resume, skip a step, force a phase advance,
cancel-with-cleanup, cancel-without-cleanup — each logged with the actor. *Editing* a running event
is not one of them (see [Versioning](#versioning-and-editing-a-live-event)).
> **Six of those exist as of Phase 3, and two do not — for reasons, not for scope.** Pause, resume,
> cancel, and a step's confirm, skip and retry are built and gated to `admin` + `moderator`.
> **`advance` — force a phase forward — is not**, because it has no honest meaning yet: a phase today
> advances when its steps go terminal, and the per-step skip already does that one step at a time.
> Phase 5 gives a phase an advance *condition*, and that is the first moment "force it anyway" names
> something an operator could predict. **Cancel takes `{ reason }` and not `{ cleanup }`**, because
> the resource ledger a cleanup would work over arrives in Phase 8; a flag that changes nothing is
> the "control that answers 200 and does nothing" this plan has refused twice already.
> **Retry is one control, not two.** A step may be retried only while its run is `paused`, and a
> paused run is paused *at* that step — so re-queueing without resuming would leave the run in
> precisely the state it was already in, with a second button the operator now has to find.
> Splitting them would read as honesty and behave as a trap. The single action re-queues the step and
> resumes the run, and its log line says both.
>
> Two guards make it safe. The step must be `failed`, and it must be **the furthest step its phase has
> reached** — `MAX(seq) WHERE status <> 'pending'`. The near miss is worth recording because the
> obvious rule is the wrong one: "the lowest step that is not settled" looks equivalent and is not,
> because `nextOpenStep` selects `pending` and `running` only and the runner therefore steps *over* a
> failed step. Under that rule a phase whose second step failed-and-skipped and whose fifth then
> failed-and-paused would offer retry on the second, re-queueing a row behind the runner's own cursor
> where it would sit `pending` for ever.
>
> `attempts` returns to zero, and that is not the rule Engagement Phase 14 arrived at being broken.
> That rule is about *sweeps*: an automatic path that reset a counter made the ceiling unreachable and
> the row immortal. `EVENT_STEP_MAX_ATTEMPTS` bounds what the runner does **unattended**, and a named
> person deciding once is the thing it is unattended from.
**Public surface.** An upcoming-events calendar with series and arcs, a live-status page, and
published results. It must tolerate a run with `health: 'degraded'` without saying so — "the shard is
having trouble" is operator information, and the existing shard pages already model exactly this by
@@ -966,7 +1000,9 @@ controller stamps it from the session.
| **A GM changes a leased property in-client** | Restore is compare-and-set: current value ≠ what the event applied, so nothing is written. The resource becomes `drifted` and is surfaced beside the unreverted ones. |
| **A step would exceed its cap** | `refused`, with the dimension and the numbers, surfaced to the author. Not a retry and not a failure — it is an authoring error. |
| **An action fails** | Per-step `on_failure`, defaulted from the risk class: `retry(n) → skip` for `notify`, `retry(n) → pause` for `change`, `retry(n) → abort_run` for `irreversible`. `pause` stops the run advancing and waits for a human — the right default when the world is half-changed. `n` is `EVENT_STEP_MAX_ATTEMPTS`, 3 by default. **All three dispositions write the STEP `failed`**: `on_failure` says what happens to the run, and a step attempted three times that never worked is `failed` under every one of them. `skipped` is reserved for a step a human skipped from the run console — a status meaning both "nobody ran this" and "this failed and we moved on" would make the console's summary line unreadable. |
| **A run is cancelled** | Pending steps `cancelled`; a running one is left to finish or time out (nothing can recall a sent command); cleanup steps are generated from the ledger and run. Cancelling *without* cleanup is a separate, logged, admin-only action. |
| **A run is cancelled** | Pending steps `cancelled`; a running one is left to finish or time out (nothing can recall a sent command); cleanup steps are generated from the ledger and run. Cancelling *without* cleanup is a separate, logged, admin-only action. **A PARKED step is cancelled with the pending ones** (Phase 3): a cue is not a command already sent, it is an instruction nobody is holding, and leaving it `running` would have the console claim a cancelled event is still waiting for someone. The live lease is what tells the two apart, and it is in the `WHERE` clause. |
| **A human skips a step** | `skipped`, which is what that status is reserved for. Legal for a `pending` step and for a parked cue; refused for a step with a live lease, and unnecessary for a `failed` one — `nextOpenStep` already passes over it, so resuming the run carries the phase past it. |
| **A human retries a step** | Only while the run is `paused`, and only on the furthest step its phase has reached. `attempts` returns to zero and the run resumes in the same action; both facts are in the log line with the actor. |
| **Cleanup itself fails** | The run reaches `completed` with `cleanup_status = 'incomplete'`, the unreverted resources listed and a manual retry offered. It does **not** stay `running` — an event whose world changes are still up is a real state, and pretending the event is in progress hides it. |
### The ledger's two rules
@@ -1020,8 +1056,9 @@ no URL moved.
| `GET /admin/events/runs` | staff | run history across definitions |
| `GET /admin/events/runs/:runId` | staff | status, phase, steps, caps, resources, cleanup |
| `GET /admin/events/runs/:runId/log` | staff | the diagnostic log |
| `POST /admin/events/runs/:runId/pause\|resume\|advance\|cancel` | admin, moderator | live control of a run in flight; `cancel` takes `{ cleanup, reason }` |
| `POST /admin/events/runs/:runId/steps/:stepId/skip\|retry\|confirm` | admin, moderator | `confirm` resolves a GM cue step |
| `POST /admin/events/runs/:runId/pause\|resume\|cancel` | admin, moderator | live control of a run in flight (Phase 3); `cancel` takes `{ reason }`, and gains `cleanup` with the ledger in Phase 8 |
| `POST /admin/events/runs/:runId/advance` | admin, moderator | force a phase forward — **not built**; it has no honest meaning until Phase 5 gives a phase an advance condition |
| `POST /admin/events/runs/:runId/steps/:stepId/skip\|retry\|confirm` | admin, moderator | Phase 3. `confirm` resolves a GM cue step; `retry` re-queues the step a paused run is stopped at and resumes it |
| `POST /admin/events/runs/:runId/cleanup` | admin | re-run cleanup over unreverted resources |
| `GET /admin/events/catalog` | staff | registered actions, param schemas, risk classes, budget dimensions |
| `GET /admin/events/catalog/options/:sourceId` | staff | a module's option list for a param |
@@ -1059,6 +1096,26 @@ the shipped demo of Phase 2 is a run that announces, waits and completes without
which is exactly the thing that needs no control. `core.cue`'s confirm is the first of them that has
something to act on, and it arrives with the console that shows the cue.
**Phase 3 added six routes, and they are the live controls.** `pause`, `resume` and `cancel` on a
run; `confirm`, `skip` and `retry` on one of its steps. All six are `admin` + `moderator` — the one
gate in this feature wider than `admin`, and deliberately so (§N2). Every one of them is a
compare-and-set against the status it is allowed to act from rather than a read-then-write: the
runner ticks every fifteen seconds, so a console rendered thirty seconds ago describes a run that has
since moved, and a control that checked in JavaScript and then wrote would race the tick it exists to
interrupt. A refusal is a `409` naming the status the run is actually in.
Still absent, and still for reasons rather than for scope: `advance` (no advance condition until Phase
5), `cleanup` and cancel's `cleanup` flag (no resource ledger until Phase 8), `verify` and `GET/PUT
/admin/events/actions` (no caps to price against and no switchboard to serve until Phase 6).
Two response fields arrived with the console and are worth naming because both are **derived, not
columns**. A run carries `waitingSteps`, the number of its steps parked on a human, so the run LIST
can say so — a cue nobody notices is a run that never advances while looking perfectly healthy from
the outside. A step carries `parked`, a boolean, because the console has to tell a cue waiting on a
person from a step some process is mid-dispatch on, and it must not do so by being shown
`claimed_by` and `claim_expires_at`: those are the runner's business, and a UI that reasoned about
leases would be a second opinion about who owns a row.
A module registers actions server-side and adds **no routes** for them beyond its option endpoints,
which is what keeps the browser from being able to name a transport.

View File

@@ -289,6 +289,71 @@ mid-dispatch. Test both in-process. The decision is not licence to drop a CAS.
### Phase 3 — The minimal admin surface (`website`)
> **Complete.** `edge` in `website`. Three screens, a nav group and **six live run controls** — the
> routes Phase 1 left absent on purpose because nothing was in flight, and Phase 2 gave something to
> act on. **An admin can now author, publish, schedule, start and watch an event that announces
> things and cues a human, and a moderator can stop one that is going wrong.** This is the first
> phase with a demo.
>
> **Four decisions the org lead settled (2026-09-02), all as recommended:**
>
> - **Six controls, not four and not eight.** `pause`, `resume`, `cancel` on a run; `confirm`,
> `skip`, `retry` on a step. **`advance` is not built** — a phase today advances when its steps go
> terminal, and the per-step skip already does that one step at a time, so a force-advance now would
> silently change meaning under the operator when Phase 5 gives a phase an advance *condition*.
> `cleanup` needs Phase 8's ledger.
> - **Cancel takes `{ reason }`, not `{ cleanup, reason }`.** The flag arrives with the thing it would
> act on. A `cleanup: false` that changes nothing is the "control that answers 200 and does nothing"
> Phases 1 and 2 both refused.
> - **Its own top-level nav group, staff-wide** (`admin`, `editor`, `moderator`) — not admin-only like
> Engagement's. §K makes every read here `staff`, and the moderator's entire power over this feature
> is the run console; hiding it from them would leave the one role that exists for incident response
> unable to see the incident. The narrow gates are on the actions instead, and each button follows
> the route it calls.
> - **The params box is a raw JSON field with the action's declaration rendered beside it.** Both are
> already in the catalog — name, type, required, description, example — so the placeholder is usable
> without reading source, and it is captioned as a placeholder so it does not read as Phase 13's
> schema-driven form.
>
> **Three things the build settled, and the first is a defect in shipped code:**
>
> - **A pause pressed mid-tick did nothing for up to 24 more steps.** `advanceRun` drains up to
> `EVENT_STEPS_PER_TICK` steps from one run inside a single tick and only checked the run's status at
> the top of it — so the whole value of a pause, that it takes effect *now*, was absent. The loop
> re-reads the status between steps (`runsDb.statusOf`, one indexed column by primary key). Found by
> writing the test; the test was re-run against the unfixed code to confirm it fails, and it does.
> - **The retry guard was reading the wrong end of the phase.** The first draft asked for the lowest
> `seq` that is not *settled*, which looks equivalent to "the step the run is stopped at" and is not:
> `nextOpenStep` selects `pending` and `running` only, so the runner steps *over* a `failed` step.
> A phase whose second step failed-and-skipped and whose fifth then failed-and-paused would have
> offered retry on the second, re-queueing a row behind the runner's own cursor where it sits
> `pending` for ever. The rule is now `MAX(seq) WHERE status <> 'pending'` — the furthest the phase
> has reached — and the test that found it is the one that names the case.
> - **Retry and resume are one control, because there is no state in which you would want half of
> it.** Retry is legal only from `paused`, and a paused run is paused *at* that step; re-queueing
> without resuming leaves the run exactly where it was with a second button to find. `attempts`
> returns to zero: the ceiling bounds what the runner does **unattended**, and a named person
> deciding once is the thing it is unattended from. That is not Engagement Phase 14's rule being
> broken — that rule is about automatic *sweeps*.
>
> **Two smaller ones, taken as assumptions rather than asked:** the console polls every 5s while the
> run is non-terminal and stops the moment it is not (§N5, poll not SSE — a run changes on a
> fifteen-second tick and a console is a tab left open for two hours); and `confirm` takes an optional
> note saying what was actually done in-client, which is kept on the step and in the log.
>
> **A cue nobody notices is a run that never advances**, and it looks perfectly healthy from the
> outside — `running`, nothing failed. So `waitingSteps` is on the run LIST as well as the console,
> as a derived count rather than a column, and the list leads with a banner naming every run that is
> waiting on a person.
>
> **Verify, as run.** `npm test` — the pre-existing `engagementManifest.test.js` CRLF failure is the
> only red, exactly as in Phase 2. **42 new tests**: `eventRunControls.test.js` (22, almost all of them
> *refusals* — a control that works from a status it should not have is a staff member changing a live
> world from a stale screen), 8 more in `eventRunnerSql.test.js` proving the four new statements
> against a real MariaDB, 2 more in `eventRunner.test.js`, and `eventAuthoring.test.js` (20) on the
> client. `routes:manifest` and `swagger` regenerated — **six routes added, none moved**. The client
> builds.
`client/src/routes/admin/views/EventsAdmin.jsx`, `EventEditor.jsx`, `EventRun.jsx`, plus the nav rows.
A list with state and next occurrence; a create/edit form; publish; start now; cancel; and a run
console showing the phase, the step list with status and attempts, and the log.
@@ -298,6 +363,14 @@ console showing the phase, the step list with status and attempts, and the log.
consult does not exist yet — a button that is admin-only later and open now is a gate nobody
notices was missing.
**As built, the control set is six**: `pause`, `resume`, `cancel` on a run and `confirm`, `skip`,
`retry` on a step, all `admin` + `moderator`. Every one of them is a compare-and-set on the status it
may act from, never a read-then-write — the runner ticks every fifteen seconds, so a console rendered
thirty seconds ago describes a run that has moved, and a control that checked in JavaScript and then
wrote would race the tick it exists to interrupt. A refusal is a `409` naming the status the run is
actually in, and the client models the same guards so a button the server will refuse is not offered
in the first place.
The spec is edited as **structured fields for the parts that exist** (name, description, schedule,
phases with their steps) and the step's params as a raw JSON field — a deliberate placeholder that P13
replaces with the schema-driven editor. Say so in the UI, so it does not read as the finished thing.