feat(events): the resource ledger, leases and cleanup (Phase 8)
Some checks failed
PR Checks / client-build (pull_request) Successful in 3m15s
PR Checks / server-tests (pull_request) Failing after 8m21s
PR Checks / bot-tests (pull_request) Successful in 11m12s

Event System Phase 8 (EVENTS_PLAN.md). Docs half: RunicGateway/docs#NNN.

One table, one core action, one route, one body field, and two members added to
MODULE_API 1.10.0 in place. The safety property the whole world-write half
depends on: core now remembers what a run changed in the world, and gives it
back on every terminal path.

Four decisions settled by the org lead on 2026-09-03, all as recommended:

- A lease is acquired by a new CORE action, `core.lease`. Section F puts the
  duration bound and the two-events-one-target conflict check on core's side of
  the seam, and a lease verb per module would be both re-implemented once per
  module, advisory everywhere.
- Record-before-confirm is a PLACEHOLDER keyed by the step's idempotency key. A
  spawn's ref does not exist until the module answers, so what core writes
  before the dispatch is `kind: '@step'`, `ref` = that key. If the answer never
  comes it stands, and cleanup calls revert() with the key and no resources --
  which is why section F's revert takes the key at all.
- Cleanup is one sweep over the ledger, not synthetic step rows. The
  step-shaped version costs a second retry counter beside `revert_attempts`.
- `reconcile` is declared here and TRIGGERED BY THE MODULE, through
  `ctx.events.reconcile()`. Core has no concept of the game being up, so it
  cannot decide when to ask; it asks once at its own boot.

MODULE_API stays 1.10.0. A protocol owes a bump once it has landed on `main`;
while it is on `edge` it is amended in place, so the whole module contract
reaches an author as one version they read once.

Verify

- `npm test` -- 2025 tests, 1935 pass, 89 skipped, 1 fail. That one is the
  pre-existing engagementManifest CRLF failure, in a file this branch does not
  touch (`edge` before: 1950/1876/73/1). +75 tests.
- The unique key was proved against a REAL MariaDB, because nothing else can
  prove it: whether multiple NULLs collide in a unique index, whether a STORED
  generated column is recomputed on UPDATE, and whether the SET NULL foreign key
  survives beside it are properties of the server. eventRunnerSql.test.js gained
  16 tests; 65 pass against the container. The real schema.sql was applied to a
  fresh database and to an existing one.
- Client: 362 pass, and it builds. routes:manifest and swagger -- one route
  added, none moved.

The live walk found three defects, and two of them are the phase's real finding

Driven by a throwaway `rig` module in website/modules/, deleted before commit.

1. A lease was never given back at all. `core.lease` reserves its own ledger
   row, so it never went through the ledger's dirty-marking, so a run holding
   only a lease kept `cleanup_status = 'not_required'` and the cleanup leg --
   which selected on `pending` -- never looked at it.
2. EVENT_REVERT_MAX_ATTEMPTS meant one attempt, not three. The first failing
   sweep moved the run to `incomplete`, which took it out of the leg's own scan
   for ever. The test covering the bound asserted `<= 3` and was satisfied by 1:
   a bound has two halves, and a test that only asserts the ceiling passes
   against a floor.
3. The first fix for (2) made the console lie. Spending every row's
   `revert_attempts` was a tidy way to take a `cleanup: false` run out of a
   counter-bounded scan, and the run page then rendered "3 attempts" beside
   resources nothing had ever tried. Found by opening the page.

Both (1) and (2) are the same mistake: deriving "is there anything to do" from a
summary column instead of from the rows. Neither was visible to a unit test,
because a test that calls the sweep directly never asks what would have selected
the run.

The two properties that need the process to die were walked as the plan asks.
With the module's perform() hanging, the placeholder existed while the dispatch
was in flight and nothing was named; after taskkill and a restart the reclaim
re-dispatched the same idempotency key, the retry re-used its own placeholder,
and everything was given back. Then, with the module reporting one of two
resources as no longer in force, the boot-time reconcile marked the other
`orphaned` -- never `reverted`.

This branch does NOT bump MODULE_API_VERSION, so the integration kit stays as
Phase 7 left it: red until the Phase 16 cutover re-pins ci/core-ref.json.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-09-03 21:19:27 -05:00
parent 57d183e921
commit fdc118166c
29 changed files with 3928 additions and 72 deletions

View File

@@ -538,8 +538,12 @@ export const api = {
pauseEventRun: (runId, reason) =>
req(`/admin/events/runs/${runId}/pause`, { method: 'POST', body: { reason } }),
resumeEventRun: (runId) => req(`/admin/events/runs/${runId}/resume`, { method: 'POST' }),
cancelEventRun: (runId, reason) =>
req(`/admin/events/runs/${runId}/cancel`, { method: 'POST', body: { reason } }),
// `cleanup` defaults to true server-side and has to be asked out of: EVENTS.md
// §L makes cancelling WITHOUT cleanup the separate, admin-only, logged action,
// so an absent flag means "give back what this run took".
cancelEventRun: (runId, reason, cleanup = true) =>
req(`/admin/events/runs/${runId}/cancel`, { method: 'POST', body: { reason, cleanup } }),
cleanupEventRun: (runId) => req(`/admin/events/runs/${runId}/cleanup`, { method: 'POST' }),
advanceEventRun: (runId, reason) =>
req(`/admin/events/runs/${runId}/advance`, { method: 'POST', body: { reason } }),
confirmEventStep: (runId, stepId, note) =>

View File

@@ -38,6 +38,15 @@ import {
// SERVER'S. `gates[].where` arrives already rendered, because those labels are
// defined in the condition grammar and a second renderer in the browser would
// be a second opinion about what `gte` reads as.
//
// **Phase 8 gave it a third, and it is the one that outlives the event.** The
// resource ledger is what this run changed in the world and what became of it,
// and its unresolved rows are the reason a `completed` run can still need a
// person — EVENTS.md §L: a run reaches `completed` with `cleanup_status =
// 'incomplete'` rather than being held open, because a tidy `completed` row over
// a shard full of orphaned monsters is the failure that would end this feature's
// credibility on its first bad night. The panel is shown on finished runs for
// exactly that reason, and it is the only panel here whose empty state matters.
const POLL_MS = 5000
@@ -50,6 +59,29 @@ const STATUS_COLOR = {
completed: '#8fc79a',
}
// The six ledger statuses, in the two groups that matter to a reader: green is
// resolved, amber wants a person. `orphaned` and `drifted` are amber rather than
// red because neither is a fault — one thing vanished, the other was taken by
// somebody with every right to take it — and red is reserved for "this did not
// come back and core kept asking".
const RESOURCE_COLOR = {
reverted: '#8fc79a',
confirmed: '#d9c184',
pending: '#d9c184',
reverting: '#d9c184',
drifted: '#d9c184',
orphaned: '#d9c184',
}
const RESOURCE_WORD = {
pending: 'recorded, unconfirmed',
confirmed: 'still out there',
reverting: 'being given back',
reverted: 'given back',
orphaned: 'gone',
drifted: 'someone else moved it',
}
const STEP_COLOR = {
done: '#8fc79a',
failed: '#d98b84',
@@ -148,6 +180,9 @@ export default function EventRun() {
// into the run when it was created, so this is what THIS run is allowed rather
// than what the switchboard says today.
const [budget, setBudget] = useState([])
// What this run created or borrowed, and what became of each (Phase 8).
const [resources, setResources] = useState([])
const [unresolved, setUnresolved] = useState(0)
const [lines, setLines] = useState([])
const [loading, setLoading] = useState(true)
const [error, setError] = useState(null)
@@ -168,6 +203,8 @@ export default function EventRun() {
setCounts(detail.counts || {})
setGates(detail.gates || [])
setBudget(detail.budget || [])
setResources(detail.resources || [])
setUnresolved(detail.unresolvedResources || 0)
setLines(log.log || [])
}, [runId])
@@ -308,6 +345,18 @@ export default function EventRun() {
onClick={() => act(() => api.admin.cancelEventRun(run.id, reason))}>
Cancel run
</button>
{/* The separate, admin-only decision (§L). It is a second button rather
than a checkbox on the first because the two are not variants of one
action: one gives the world back, the other deliberately leaves it
changed. A checkbox next to Cancel is a thing an operator unticks by
accident at two in the morning. The server refuses this to a
moderator, and the refusal arrives as a sentence in `problem`. */}
{controls.cancel && (
<button type="button" className="pill" style={{ fontSize: '0.74rem' }} disabled={busy}
onClick={() => act(() => api.admin.cancelEventRun(run.id, reason, false))}>
Cancel, leave changes up
</button>
)}
</div>
{isTerminalRun(run.status) && (
@@ -400,6 +449,79 @@ export default function EventRun() {
</div>
)}
{/* ── What this run changed in the world (Phase 8) ──
The WHOLE ledger, reverted rows included: "how much did last night's
invasion actually spawn, and did all of it come back" is one question
with two halves, and a list of only the failures answers neither.
Shown on finished runs for the same reason the caps meter is. */}
{(resources.length > 0 || run.cleanupStatus === 'incomplete') && (
<div
className="panel-flat"
style={{
padding: 14,
marginBottom: 14,
borderLeft: `3px solid ${unresolved > 0 ? '#d9c184' : 'var(--rule)'}`,
}}
>
<div style={{ display: 'flex', justifyContent: 'space-between', alignItems: 'baseline', gap: 12, flexWrap: 'wrap' }}>
<h3 className="sans" style={{ margin: '0 0 4px', fontSize: '0.92rem' }}>
What this run changed
</h3>
{/* The manual retry. Offered only on a terminal run, because a run
still in flight has a ledger that is still growing and reverting a
resource the next step is about to use would be undoing an event
while it is happening. */}
{isTerminalRun(run.status) && unresolved > 0 && (
<button type="button" className="pill" style={{ fontSize: '0.74rem' }} disabled={busy}
onClick={() => act(() => api.admin.cleanupEventRun(run.id))}>
Try cleanup again
</button>
)}
</div>
<p className="sans dim" style={{ margin: '0 0 10px', fontSize: '0.8rem' }}>
{unresolved > 0 ? (
<>
{unresolved} of these {unresolved === 1 ? 'is' : 'are'} still unresolved. The runner
gives them back on its own and stops asking after a few tries;{' '}
<em>Try cleanup again</em> clears that count and asks once more.
</>
) : (
'Everything this run created or borrowed has been given back.'
)}
</p>
{resources.length === 0 ? (
<p className="sans dim" style={{ margin: 0, fontSize: '0.8rem' }}>
Nothing named a step changed the world and its answer never arrived, so core kept the
record it wrote beforehand and will ask the module to undo it by key.
</p>
) : (
<table className="sans" style={{ fontSize: '0.82rem', borderCollapse: 'collapse', width: '100%' }}>
<tbody>
{resources.map((r) => (
<tr key={r.id}>
<td style={{ padding: '3px 12px 3px 0', whiteSpace: 'nowrap' }}>
<code style={{ fontSize: '0.78rem' }}>{r.kind}</code>{' '}
<code className="dim" style={{ fontSize: '0.78rem' }}>{r.ref}</code>
</td>
<td style={{ padding: '3px 12px 3px 0', whiteSpace: 'nowrap', color: RESOURCE_COLOR[r.status] }}>
{RESOURCE_WORD[r.status] || r.status}
</td>
<td className="dim" style={{ padding: '3px 12px 3px 0', whiteSpace: 'nowrap', fontSize: '0.78rem' }}>
{r.module}
{r.leaseUntil ? ` · until ${clock(r.leaseUntil)}` : ''}
{r.revertAttempts > 0 ? ` · ${r.revertAttempts} attempt${r.revertAttempts === 1 ? '' : 's'}` : ''}
</td>
<td className="dim" style={{ width: '100%', padding: '3px 0', fontSize: '0.78rem' }}>
{r.lastError || ''}
</td>
</tr>
))}
</tbody>
</table>
)}
</div>
)}
{/* ── Waiting on a person ── */}
{parked.length > 0 && (
<div className="panel-flat" style={{ padding: 14, marginBottom: 14, borderLeft: '3px solid #d9c184' }}>