Only the Gitea release workflow carries the publishing placeholders. The report said both flavours; the GitHub one uses GITHUB_REPOSITORY and friends and has nothing to rename. Found while fixing it in the kit (Integration-kit#4), which is where the single checklist row landed. Also records what that implies about a run like this: every finding was re-derived before being acted on, and it was not ceremony — this one was wrong in a detail and F5 understated its problem in a way that changed the fix. A cold agent's report is evidence, not a verdict, which is the standard the rest of the plan already applies to a passing suite. Separate PR because #147 was already merged; a push to a merged branch is silent. Co-Authored-By: Claude <noreply@anthropic.com>
145 lines
11 KiB
Markdown
145 lines
11 KiB
Markdown
# Integration Kit — the acceptance run
|
|
|
|
**Phase 5's fifth decision** ([`../website/MODULE_SYSTEM.md`](../website/MODULE_SYSTEM.md) §2.11.1 d5).
|
|
The [Integration Kit](https://gitea.whitlocktech.com/RunicGateway/Integration-kit) is finished when
|
|
someone outside this project builds a working module for a new game by following it alone. That
|
|
cannot be manufactured, so this is the **proxy**: a cold agent given the kit and the documents it
|
|
links to — never core's source, never `module-uo` — asked to build a module for a second game, whose
|
|
result was then installed into a real core and loaded in a browser.
|
|
|
|
Run **2026-08-12**, against `Integration-kit` at slice 2 (PR #3) and core at website `edge` 1b692bf.
|
|
|
|
> The verdict below does **not** clear the draft banner. Decision 32 makes that a human's to remove,
|
|
> and this exercise is aimed at the same target with a machine. It is worth what it found, not more.
|
|
|
|
---
|
|
|
|
## 1. Verdict
|
|
|
|
> **Yes, with caveats.** A competent developer who had never seen this platform can build a working
|
|
> module from this kit alone.
|
|
|
|
It did, in one pass, without reading core's source and **without opening three of the four normative
|
|
documents** — `MODULE_SYSTEM.md` and both `link/` specs were never touched, correctly, since the
|
|
deliverable was a website module with no sidecar. `MODULE_API.md` was opened only at the end, to
|
|
audit the kit rather than to build against it.
|
|
|
|
Its own summary of what carried the run:
|
|
|
|
> What raises this to "yes" rather than "yes but barely" is the template. The four suites plus the
|
|
> two guard scripts constitute a machine-checkable specification of a valid module, and they are
|
|
> better documentation than any chapter could be — every rule they enforce comes with a comment
|
|
> explaining the bug that motivated it and how it looked when it went wrong.
|
|
|
|
And the caveat that mattered most, which is a process one rather than a comprehension one: **the
|
|
first command the kit tells a reader to run fails on a pristine copy** (F1).
|
|
|
|
## 2. Method, and the one thing it could not enforce
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| **Given** | `Integration-kit` at slice 2, tracked files only (so the template's chunk had to be built, as a reader's would); local mirrors of the five documents the kit links out to, at the same relative paths, because the kit's links point at a Gitea host a credential-less agent cannot reach |
|
|
| **Denied** | core's source, `module-uo`, the web beyond the npm registry |
|
|
| **Asked for** | a trivial but real module for **Rust** — valid `module.json`, a public route backed by its own table, a prebuilt chunk rendering it, a purge fragment, and whatever guards the kit says a module needs, passing |
|
|
| **Told to log** | findings *as they happened*, into a running file, on the grounds that being stuck is worth more than remembering being stuck |
|
|
| **Model** | Claude Opus 5 |
|
|
|
|
Rust was chosen deliberately over an unfamiliar game: it is the subject of
|
|
[`rust-dryrun.md`](rust-dryrun.md), which the kit's front page names as the first thing to read, so
|
|
the run also measures whether that document is followable.
|
|
|
|
**The honest limitation.** A subagent on this machine inherits the workspace as its shell's default
|
|
directory, so isolation is a clean tree plus explicit instructions plus a self-reported list of every
|
|
path read — not a sandbox. The agent reported no read outside its root and disclosed one unrelated
|
|
process slip unprompted, which is some evidence of good faith but is not proof. A future run should
|
|
put the agent somewhere it *cannot* reach core.
|
|
|
|
## 3. What was built, and what it proved
|
|
|
|
`module-rust` — id `rust`, `coreApi ^1.4.0`, two public routes over two tables (`rust_servers` +
|
|
`rust_wipes`, FK, child-first purge), a 8.0 kB chunk with two pages and a nav row. **44 server + 18
|
|
client tests, none skipped**; `check:imports`, `check:swagger`, `check:externals` all green.
|
|
|
|
Installed into a real core on `edge` and booted **against an empty database** — the condition Phase 4
|
|
slice 3 established as the only one that can see a schema-ordering defect:
|
|
|
|
- loader scanned it, schema fragment replayed (4 statements), `onBoot` ran, state `started`
|
|
- `/api/v1/public/modules` published it; both routes served correct JSON
|
|
- the chunk registered under enforced `script-src 'self'` with **zero CSP reports and zero console
|
|
errors**, and a **deep-linked param route rendered on a cold load** — the `readyState` ordering
|
|
path [`../website/MODULE_API.md`](../website/MODULE_API.md) §7.7 exists for
|
|
|
|
Two things it did *not* do are worth recording as evidence the contract held for a stranger: it never
|
|
reached into core (`check:imports` green), and it wrote no connection to a game server — in fact it
|
|
wrote a test asserting that about itself, having read in chapter 2 that §2.7 is the one rule with no
|
|
CI behind it.
|
|
|
|
## 4. Findings
|
|
|
|
Eight from the agent, one from the browser. **B1 could not have been found by the agent** — it had no
|
|
core to render against, which is the structural gap this two-stage method exists to cover, and it is
|
|
the most valuable single result of the run.
|
|
|
|
**Every finding was re-derived before it was acted on**, and that was not ceremony: one of them (F3)
|
|
was wrong in a detail, and another (F5) turned out to understate the problem in a way that changed
|
|
the fix. A report from a run like this is evidence, not a verdict — the same standard the rest of
|
|
this plan applies to a passing test suite.
|
|
|
|
| # | Class | Where | What | Disposition |
|
|
| --- | --- | --- | --- | --- |
|
|
| **B1** | **WRONG** | `MODULE_API.md` §3.4 | A module page built exactly as the kit teaches **renders outside the site**. `PublicLayout` gives the chrome, not the body; core's nine public pages each wrap content in `shell-… page-body`, whose class names appear in no contract. Content at x=0, no padding, footer riding up under it | **Fixed** — `PublicLayout` takes an opt-in `shell` prop, `MODULE_API_VERSION` **1.5.0** (website#148); §3.4 rewritten; template and chapter 2 updated |
|
|
| F1 | WRONG | `template/README.md`, `book/02` | `npm run check:swagger` **fails on a pristine, unedited copy** on Windows: the fragment compares byte-for-byte and a default Windows clone is CRLF. The message asserts a false cause — "the routes or their annotations changed" | **Fixed** — `template/.gitattributes` (`eol=lf`) *and* the comparison normalises line endings |
|
|
| F2 | WRONG | `rust-dryrun.md` §1 | `coreApi: "^1.3.0"` while everything else said 1.4.0 — in the only complete `module.json` in the kit's reading path | **Fixed** — `^1.5.0`, as a dated correction |
|
|
| F3 | MISSING | `template/README.md` | The rename checklist claims to name every file carrying a placeholder. `.gitea/workflows/release.yml` carries `gitea.example.com` and `your-org/your-module` under a literal `# CHANGE THESE` and is absent — and `checkRenameSites.js`'s pattern cannot match them, so CI is silent **by construction** | **Fixed** — row added, pattern widened. *Corrected: the report said both workflow flavours; only the Gitea one is affected, since GitHub supplies `GITHUB_REPOSITORY` and friends* |
|
|
| F4 | FRICTION | `template/.gitea/workflows/release.yml` | The bundle's include list is hardcoded (`for d in boot.js core.js index.js db model router`). Add any top-level directory under `server/` and it is **silently dropped from every release**; the post-check only resolves the five paths in `module.json` | **Fixed** — inverted to an exclusion list |
|
|
| F5 | AMBIGUOUS | `world.router.js`, `book/02` | The comment warns that a backtick in a single-quoted `#swagger` description ends the string early — but the same file uses two backtick spans and they survive verbatim | **Fixed**, and sharpened: see below |
|
|
| F6 | AMBIGUOUS | `template/` | No `.gitignore`. The kit's own covers the template's paths, but a reader who copies the directory and runs `git init` inherits nothing — `node_modules/` included | **Fixed** — `template/.gitignore` ships |
|
|
| F7 | WRONG | `MODULE_API.md` §3.4 | "The kit is those **seven** members" over a table publishing **eight** exports (five rows; `PageState` contributes three). Propagated faithfully into the template's comment and core's `shared.js` — the kit's never-re-specify discipline carrying a miscount *out* of the contract | **Fixed** in all three |
|
|
| F8 | note | — | Three of the four normative documents were never opened, and never needed to be | Recorded above |
|
|
|
|
### F5 is worse than reported, and the correction is the useful part
|
|
|
|
Measured directly rather than reasoned about. A **backtick** inside a single-quoted `#swagger`
|
|
description is harmless — it survives verbatim into the fragment. A **double quote** is not, and it
|
|
does not fail the way the comment predicts:
|
|
|
|
```
|
|
// #swagger.summary = 'A "quoted" world status'
|
|
→ "summary": "A \"" ← silently truncated
|
|
→ swagger-autogen prints Success, and swaggerFragment.js's error capture sees nothing
|
|
```
|
|
|
|
The template told a reader that the tool would tell them. For the one case that actually bites, it
|
|
does not: the annotation is truncated, the generator reports success, and the only signal is a
|
|
`check:swagger` diff whose message blames the routes. **A chapter that predicts the wrong debugging
|
|
heuristic is worse than one that predicts none** (§2.11.1) — restated here for a comment inside the
|
|
template, which no check in that repo can see either.
|
|
|
|
### Two things the run recommends that are not defects
|
|
|
|
- **"Run every check on the untouched copy before you edit anything."** The agent did this off its
|
|
own bat, and it is what found F1; without a baseline it would have blamed its own edits. Chapter 1
|
|
now says so.
|
|
- **The template should ship the §2.7 self-check.** Chapter 2 correctly says the no-game-connection
|
|
rule has no CI behind it, because an outbound socket is not statically detectable *in general*. But
|
|
a module can make a decidable claim about **itself**, and the agent wrote that check, with a
|
|
positive control (it catches a real RCON dial) and a negative one (prose about RCON does not trip
|
|
it). It called it "the check I would most want the template to ship", noting that for Rust
|
|
`new WebSocket(rconUrl)` in `boot.js` is ten lines away. Adopted into the template, from its code.
|
|
|
|
## 5. What this does and does not settle
|
|
|
|
**Settles:** the kit is sufficient for the website-module half of the job. Someone can start from
|
|
nothing and finish with a module that loads into a real core, serves, renders in the site's chrome,
|
|
and passes every rule the contract has a check for — without reading core.
|
|
|
|
**Does not settle:** the sidecar and the game-plugin halves. This module has neither, so chapters 3
|
|
and 4 were never exercised and remain proved only by the fact that they describe systems that exist
|
|
(`link/`, `servuo-plugins/`). Nor does it settle anything about a *person's* experience — an agent
|
|
does not get frustrated, does not skim, and does not give up, which are three of the failure modes
|
|
the banner is really about.
|
|
|
|
**So the banner stays.** The next run of this exercise should either be a person, or an agent placed
|
|
somewhere it genuinely cannot reach core's source, and should be asked for a module **with** a
|
|
sidecar.
|