diff --git a/installer/INSTALL.md b/installer/INSTALL.md index 1de87a9..1ef089b 100644 --- a/installer/INSTALL.md +++ b/installer/INSTALL.md @@ -5,12 +5,13 @@ installation and connects it to a Runic Gateway website. > **Status: the installer binary is not released yet.** > -> Phases 1 and 2 are built and live on the installer repo's `edge` branch: the installer core -> (bundle resolution, ServUO detection, the overlay sync, `install.json`) and the sidecar half (the -> binary, its config, its service, and the token handoff). What is still missing is the **patch -> tier** ([§4](#4-the-patch-tier-optional)), which `install` reports as not applied rather than -> silently skipping. The first release follows Phase 3, so that a released binary answers every -> question this guide says it answers. +> Phases 1 to 3 are built and live on the installer repo's `edge` branch: the installer core +> (bundle resolution, ServUO detection, the overlay sync, `install.json`), the sidecar half (the +> binary, its config, its service, and the token handoff), and the +> [patch tier](#4-the-patch-tier-optional). `install` is therefore complete; what is still missing +> is [`doctor`, `update` and `uninstall`](#7-day-two), each of which reports which phase it arrives +> in rather than failing as though you had mistyped it. The first release is being cut from that +> branch now. > > Everything it installs *is* released and published — the sidecar, the plugin overlay, and the > [bundle manifest](https://gitea.whitlocktech.com/RunicGateway/installer/src/branch/main/bundles/current.json) @@ -34,7 +35,7 @@ Three things, on the machine that runs your shard: |---|---|---| | 1 | **The plugin overlay** — C# source that ServUO compiles at boot, copied into your server tree | [`RunicGateway/servuo-plugins`](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins) release tarball | | 2 | **The uo-link sidecar** — a small Rust service that the shard dials out to, and that your website reads from | [`RunicGateway/link`](https://gitea.whitlocktech.com/RunicGateway/link) release binary | -| 3 | **A record of what it did** — `install.json`, plus a cached copy of any patches it applied | Written by the installer | +| 3 | **A record of what it did** — `install.json`, plus cached copies of the patches and of every file the patch tier edited | Written by the installer | ``` ServUO shard ──loopback TCP 127.0.0.1:7788──► uo-link sidecar ──HTTP + WebSocket──► website @@ -173,9 +174,9 @@ Overlay sync ADD Config/Bridge.cfg ADD Scripts/Custom/Bridge/*.cs (22 files) CHANGE Scripts/Scripts.csproj - deployed. add=23 change=1 unchanged=0 + deployed. add=23 change=1 unchanged=0 kept=0 -Patch tier skipped (not selected) + Patch tier not selected Without it: no vendor.sale events, no in-game moderation audit forwarding. uo-link sidecar @@ -230,7 +231,8 @@ reports "unchanged" and writes nothing. | `/usr/bin/runicgateway-link` | The sidecar binary | | `/etc/runicgateway/sidecar.toml` | Sidecar config, including the auth token | | `/etc/runicgateway/install.json` | What the installer deployed: versions, commit, per-file hashes, applied patches, timestamps | -| `/etc/runicgateway/patches/` | Copies of any patches applied, so `uninstall` can print the exact hunks long after the release tarball is gone | +| `/etc/runicgateway/patches/` | Copies of the patches the tier evaluated, so `uninstall` can print the exact hunks long after the release tarball is gone, and a refused one is still on hand to apply yourself | +| `/etc/runicgateway/patches/originals/` | Each file the patch tier edited, exactly as it was beforehand — a revert you can verify rather than reconstruct | | `/var/lib/runicgateway/uo-link.db` | The sidecar's SQLite store (event history, cached profiles, link map) | | `/etc/systemd/system/runicgateway-link.service` | The service unit, running as a dedicated user | @@ -242,6 +244,7 @@ reports "unchanged" and writes nothing. | `%ProgramData%\RunicGateway\sidecar.toml` | Sidecar config, including the auth token | | `%ProgramData%\RunicGateway\install.json` | As above | | `%ProgramData%\RunicGateway\patches\` | As above | +| `%ProgramData%\RunicGateway\patches\originals\` | As above | | `%ProgramData%\RunicGateway\uo-link.db` | The sidecar's SQLite store | | Service `RunicGatewayLink` | Automatic start, restart on failure, running as `NT SERVICE\RunicGatewayLink` | @@ -312,12 +315,27 @@ How the installer handles it: appear exactly once; anything less and it stops and hands you the hunk to apply by hand. It never force-fits a patch by loosening the match. - **All or nothing per feature.** The two vendor-sale patches are one unit and are applied together - or not at all — and within a patch, if one hunk cannot be placed safely, none are. + or not at all — and within a patch, if one hunk cannot be placed safely, none are. A patch that + could have been placed but was held back by its sibling says exactly that; it is never reported as + applied. +- **It does not need `git`, and does not use it.** The matching and the writing are the installer's + own, which is why it can place a patch on a shard where `git apply` refuses — the shipped patches + and their target files do not all use the same line endings, and that alone defeats `git apply`. + Nothing outside a patched region is touched, down to the byte, and inserted lines take your file's + own line ending. - **Your ServUO version is reported, not decisive** — but see the warning below before running this on anything other than 57.4. - **Recorded, and the `.patch` files cached**, so re-runs stay idempotent and `uninstall` can print the exact hunks to revert — along with how each was applied, since a patch placed into a file you - had already modified is one to look at more carefully when reverting. + had already modified is one to look at more carefully when reverting. Patches that were *not* + applied are cached too, because that is the copy the run tells you to apply by hand. +- **A copy of every file it edits is kept, exactly as it was beforehand**, under + `patches/originals/` in the installer's own directory — not in your ServUO tree. It is written + before the first edit and never overwritten, so however many times you re-run `install`, it stays + the version from before the tier ever touched the file. That is what lets you verify a revert + rather than reconstruct one. +- **Re-running is safe.** A patch already in place is recognised and left alone, and the record + keeps the way it originally landed rather than relabelling it. ### ⚠ On any ServUO that is not 57.4: unsupported, untested, no guarantees @@ -347,19 +365,29 @@ Because of that, on a non-57.4 tree the tier is off by default and takes a delib A run where the tier is selected on a shard that has been worked on looks like this: ``` -Patch tier 2 of 3 applied - ✓ playervendor-sale-eventsink Server/EventSink.cs stock file - ✓ playervendor-sale-gump Scripts/Gumps/PlayerVendorGumps.cs - file modified, patched region stock — applied at line 1180 - ✗ commandlogging-event Scripts/Commands/Logging.cs - patched region has been modified — not applied - apply this hunk by hand, then re-run install: - /etc/runicgateway/patches/commandlogging-event.patch + Patch tier 2 of 3 applied + ✓ playervendor-sale-eventsink Server/EventSink.cs + stock file — applied at line 171, 1521, 1771, 2416 + ✓ playervendor-sale-gump Scripts/Gumps/PlayerVendorGumps.cs + file modified, patched region stock — applied at line 95 + ✗ commandlogging-event Scripts/Commands/Logging.cs + patched region has been modified (hunk 1) — not applied + apply this by hand, then re-run install to record it: + /etc/runicgateway/patches/commandlogging-event.patch - ⚠ Server/EventSink.cs changed — rebuild the core: dotnet build ServUO.sln - Without commandlogging-event: no in-game moderation audit forwarding. +⚠ Server/EventSink.cs — a CORE ServUO file was patched. Rebuild the solution: + dotnet build ServUO.sln + A shard restart is not enough; ServUO's dynamic script build does not rebuild the core, and it + will not tell you so. + + Not applied, so you do not get: no in-game moderation audit forwarding. + Everything else works. Apply the hunks by hand if you want them, then re-run install to record it. ``` +The line numbers are where each hunk was actually found in *your* file, not where it sits in stock +ServUO — they differ as soon as anything above the region has been edited, and yours is the one to +go to. + If it is skipped or fails, you lose exactly two things — **`vendor.sale` events** and **in-game moderation audit forwarding**. Everything else works. You can apply the patches later by hand (see `patches/README.md` in the tarball) and re-run `install` to record it. @@ -515,7 +543,7 @@ your work. | **Removed** | The sidecar binary, its service entry, `install.json`, the cached patch set | | **Kept** | `sidecar.toml` and `uo-link.db` — config and history survive (`--purge` drops them) | | **Printed, not done** | Every overlay file deployed into your ServUO tree, by path, for you to delete | -| **Printed, not done** | The exact hunks each applied patch added to `EventSink.cs`, `PlayerVendorGumps.cs` and `Logging.cs`, for you to revert | +| **Printed, not done** | The exact hunks each applied patch added to `EventSink.cs`, `PlayerVendorGumps.cs` and `Logging.cs`, for you to revert — with how each landed, since one placed into a file you had already modified is worth a closer look. The pre-patch copy kept under `patches/originals/` is there to diff against. | The report is also written to a file, so it survives the scrollback. @@ -594,7 +622,13 @@ cp patches/BridgeModerationAudit.cs Scripts/Custom/Bridge/ ``` `git apply` works in a plain directory — the shard does not need to be a git repo. If you use -`patch` instead, note the core files are CRLF: use `patch --binary`. +`patch` instead, note that some core files are CRLF while others are LF: use `patch --binary`. + +**If `git apply` refuses a patch whose target region is visibly untouched, line endings are the +usual cause** — the `.patch` files and their targets do not all use the same ones, and `git apply` +compares them literally. The installer's own tier normalizes line endings and trailing whitespace +for the *comparison* while writing back your file's own endings, which is why it can place patches +`git apply` rejects. Running the installer is the easier route here. ### A3. Install the sidecar diff --git a/installer/PLAN.md b/installer/PLAN.md index 7081b59..0f19880 100644 --- a/installer/PLAN.md +++ b/installer/PLAN.md @@ -1,15 +1,15 @@ # Runic Gateway Installer — plan -Status: **Phases 1 and 2 built, on `edge`.** Phase 0's prerequisites all landed, the installer repo -publishes the bundle manifest, and [`INSTALL.md`](INSTALL.md) specified the operator-facing run +Status: **Phases 1, 2 and 3 built, on `edge`.** Phase 0's prerequisites all landed, the installer +repo publishes the bundle manifest, and [`INSTALL.md`](INSTALL.md) specified the operator-facing run before the binary existed. The crate now implements the installer core (bundle resolution, ServUO detection and validation, the overlay sync, `install.json` — [Phase 1 as -built](#phase-1--installer-core)) and the sidecar half (binary, config, service, token handoff — -[Phase 2 as built](#phase-2--uo-link-install-and-service)). Both are on the `edge` branch, not -`main`, so no half-capable binary is released. **Phase 3 (the patch tier) is next, and the -`edge → main` cutover follows it** rather than Phase 2: `INSTALL.md` §4 describes the tier as part -of the run, and a first release whose every patch-tier answer is "not implemented" is the same -half-capable binary that kept Phase 1 off `main`. +built](#phase-1--installer-core)), the sidecar half (binary, config, service, token handoff — +[Phase 2 as built](#phase-2--uo-link-install-and-service)), and the patch tier (the rung ladder, the +unsupported-version path, the cached patch set — [Phase 3 as built](#phase-3--patch-tier-opt-in)). +All three are on the `edge` branch, not `main`, so no half-capable binary is released. **The +`edge → main` cutover is next**, and it now cuts a binary that does everything `INSTALL.md` +describes except `doctor`/`update`/`uninstall`, each of which says which phase it arrives in. This document is the design of record; it supersedes the informal overview it grew out of, which described a ServUO integration that does not match how `servuo-plugins` actually ships (see [Corrections](#corrections-to-the-original-overview)). @@ -102,10 +102,19 @@ most real shards are hand-modified. Therefore: audit forwarding**. - The `EventSink.cs` patch must warn loudly that a **core solution rebuild** is required, not just a shard restart. -- Record applied patches in `install.json`, **and cache the applied `.patch` files** next to it +- Record applied patches in `install.json`, **and cache the `.patch` files** next to it (`/etc/runicgateway/patches/`, `%ProgramData%\RunicGateway\patches\`). Re-runs stay idempotent, and uninstall can print the exact hunks offline long after the release tarball is gone (§5, - Phase 4). + Phase 4). As built this caches every patch the tier *evaluated*, not only those that applied, + because the refusal message names that path as the file to apply by hand. +- **Cache the pre-image of every file the tier edits**, under `patches/originals/`, mirroring its + path in the ServUO tree. It is written before the first edit and never overwritten, so a revert + can be verified byte-for-byte rather than reconstructed from a printed diff — which matters most + after a `region-match` apply, where the surrounding file was already the operator's. It stays out + of the ServUO tree, since uninstall has promised never to clean up in there. +- **A `.patch` does not carry everything the tier needs.** Which patches form one unit, which + companion `.cs` follows which, whether a core rebuild is required and what declining costs are + declared by the overlay release and read from its manifest — see §7.0. #### 2.2.1 A whole-file hash mismatch is not a verdict — check the region @@ -137,9 +146,13 @@ rules: - **Line numbers are advisory.** The hunk header's offsets are used only to prefer the nearest candidate when reporting; the match itself is by content, since insertions above the region shift every number below it. -- **All-or-nothing per patch file.** If one hunk of a patch reaches rung 3, none of that patch's - hunks are applied. A half-patched `EventSink.cs` compiles against a companion `.cs` that expects - the whole thing, and a partial apply is harder for an operator to unpick than an untouched file. +- **All-or-nothing per patch file, and again per feature.** If one hunk of a patch reaches rung 3, + none of that patch's hunks are applied — a half-patched `EventSink.cs` compiles against a + companion `.cs` that expects the whole thing, and a partial apply is harder for an operator to + unpick than an untouched file. The same rule then applies across the patches of one feature: the + two vendor-sale patches are a unit (the event, the call site, and the subscriber that needs both), + so a patch that *could* have been placed is held back when a sibling cannot be — and the run says + that rather than reporting it as applied. - **Rung 0 is checked first and is also all-or-nothing.** A file where some hunks are already present and others are not is a hand-merge in progress, not an idempotent re-run — that is rung 3. @@ -631,6 +644,74 @@ pick the first). Fixtures are cheap: the three stock 57.4 files, each with a han patched region (must reach rung 2), an edit inside it (must reach rung 3), and an already-patched copy (must reach rung 0). +**As built** ([installer#6](https://gitea.whitlocktech.com/RunicGateway/installer/pulls/6), with the +metadata half in [servuo-plugins#10](https://gitea.whitlocktech.com/RunicGateway/servuo-plugins/pulls/10)) +— `src/diff.rs` (the parser), `src/patch.rs` (the ladder and the applier) and `src/tier.rs` (consent, +writing, reporting, recording), wired into the same `install` run between the overlay sync and the +sidecar. The decisions that were not already settled above: + +- **The engine is fully native; `git` is never invoked.** §2.2.1 wrote rung 1 as "apply verbatim + with `git apply`", but §1 chose the release tarball precisely so there would be **no git on the + shard host**, and rung 2 needs a native applier regardless. One engine now serves both: rung 1 + keeps its distinct, stronger verdict — the whole file reproduced the diff's `index` pre-image, + computed as a git blob SHA1 in process — while the write goes through rung 2's code path. That + leaves one set of CRLF and whitespace behaviours to reason about instead of two, and a bug report + never has to say which engine ran. It is also not academic: the shipped `.patch` files are CRLF in + a Windows checkout while two of their three targets are LF, so `git apply` **refuses** patches + this places correctly. +- **What a `.patch` cannot say is declared by the release, with a built-in fallback.** Which patches + form one all-or-nothing unit, which companion `.cs` follows which, whether a **core** rebuild is + needed, and what declining costs are all things a diff does not carry. `servuo-plugins/patches/tier.json` + declares them and the release workflow folds them into `manifest.json` as `patch_tier` (§7.0), so + adding a patch regenerates release metadata rather than requiring an installer release — the same + rule §7.1 applies to the bundle. Overlay `v0.1.1` is in the current bundle and declares nothing, + so the installer carries a built-in description of exactly that release; a declared tier always + wins. A checked-in fixture of the release workflow's **own jq output** asserts the two descriptions + are identical, so the two repos cannot drift apart quietly — the failure mode otherwise is a tier + that is silently never offered. +- **All-or-nothing gained a second level.** §2.2.1 makes it per *patch file*; the tier is also + all-or-nothing per **feature**, because the two vendor-sale patches are one unit — `EventSink.cs` + grows the event, `PlayerVendorGumps.cs` raises it, and the companion subscribes to it. Applying + either alone yields a tree that does not compile or silently never emits. A patch that could have + been placed but was held back by a sibling says so in as many words; reporting it as applied is + the exact misreading this tier exists to prevent. +- **The pre-image of every patched file is cached**, under `/patches/originals/`, mirroring + its path in the ServUO tree. The tier is the only part of the installer that edits a file the + operator owns, and this is what turns "here are the hunks we added" into a revert anyone can + verify — which matters most for a `region-match` apply, where the surrounding file was already + theirs. It lives in the state directory rather than beside the file it copies, because an + installer-owned file inside the ServUO tree is one `uninstall` has promised never to clean up. It + is written before the first edit and never overwritten, so it stays pre-tier however many times + `install` runs. +- **Every patch the tier *evaluated* is cached, not only the ones that applied** — a refinement of + §2.2's "cache the applied `.patch` files". The refusal message names that path as the file to + apply by hand (as §4 of `INSTALL.md` already illustrated), so caching only successes would point + an operator at a file the run had decided not to write. +- **Rung 0 reuses the previous record whole rather than re-deriving it.** The rung is the + support-relevant fact — how did this land? — and a later run re-deriving it answers + `already-present` for something that first landed as `region-match`. That flip rewrites + `install.json` on the second run of an identical install, which is the same class of bug as the + `Bridge.cfg` comparison in Phase 1: a record describing the run instead of the state. A tree + patched by hand per `INSTALL.md` Appendix A2 has no prior record, so there `already-present` is + correctly what gets minted. +- **Declining never erases what an earlier run applied**, and no longer claims a loss that is not + real. `--no-patches` and an unselected prompt both carry the previous `patches` section through + untouched, as `--verify` does — and the "Without it:" line now names only the features the record + does not already show as applied. +- **Withholding `--patches-unsupported-servuo` skips the tier loudly rather than failing the run.** + By that point the overlay is deployed and the sidecar is about to be installed; turning a completed + base install into exit 1 over a tier documented as optional would cost the operator more than the + tier is worth. Saying nothing would be the real failure, so it is reported where it happens. + +Verified on this machine against the ServUO 57.4 tree at `C:\Users\colby\Desktop\ServUO`, across +four scratch roots built from its real files: a hand-patched tree (rung 0 on both vendor-sale +patches), a reverse-applied stock one (**rung 1 on the real `EventSink.cs`**, whose blob hash +reproduces the patch's declared `index d30788f` pre-image), a feature resolving at mixed rungs, a +tree with edits inside two patched regions (rung 3 — nothing written, the placeable sibling held +back, no companions copied, and all three patches cached anyway), and a non-57.4 tree both with and +without the extra consent flag. Three consecutive runs left `install.json` byte-identical, the +patched files unchanged, and the cached pre-image still pre-patch. + ### Phase 4 — diagnostics and updates `runicgateway doctor` — the command that makes the whole thing supportable: @@ -745,14 +826,30 @@ Shipped inside every `runicgateway-overlay-.tar.gz`, generated by that repo "repo": "RunicGateway/servuo-plugins", "protocol": 3, "servuo": { "min_version": "57.4", "patches_verified_against": "57.4" }, + "patch_tier": { + "features": [{ + "name": "vendor-sale", + "summary": "vendor.sale events — player-vendor purchases with buyer, owner, item, price and commission", + "lost": "no vendor.sale events", + "rebuild": "core", + "patches": [ + { "name": "playervendor-sale-eventsink", "file": "patches/playervendor-sale-eventsink.patch", "target": "Server/EventSink.cs" }, + { "name": "playervendor-sale-gump", "file": "patches/playervendor-sale-gump.patch", "target": "Scripts/Gumps/PlayerVendorGumps.cs" } + ], + "companions": [ + { "file": "patches/BridgeVendorSale.cs", "install_to": "Scripts/Custom/Bridge/BridgeVendorSale.cs" } + ] + }] + }, "files": { "overlay/Config/Bridge.cfg": "32718424…", "patches/…": "…" } } ``` `version` and `commit` come from the release engine; `protocol` and the `servuo` block are read from -`servuo-plugins/overlay.toml`; `files` is a SHA256 per shipped file. +`servuo-plugins/overlay.toml`; `patch_tier` is folded in from `servuo-plugins/patches/tier.json`; +`files` is a SHA256 per shipped file. -Two of these carry weight beyond documentation: +Three of these carry weight beyond documentation: - **`protocol` is a hand-maintained declaration, and has to be.** The plugin announces no version on the wire and none is queryable before ServUO boots, so nothing in CI can derive it — which makes @@ -763,6 +860,18 @@ Two of these carry weight beyond documentation: overlay moved on"** (§5, Phase 4). The installer copies these hashes into `install.json` at deploy time; a later mismatch against *both* the manifest and `install.json` means upstream changed, a mismatch against `install.json` alone means local edits. +- **`patch_tier` is everything a `.patch` cannot say about itself**, and is the reason the tier is + data rather than code. Which patches form one all-or-nothing unit, which companion `.cs` may only + be copied once that unit lands, whether the change needs a **core** solution rebuild or just the + dynamic script build, and what the operator loses by declining are none of them derivable from a + diff. Declaring them here means adding a patch regenerates release metadata rather than requiring + an installer release — the rule §7.1 already applies to the bundle. The maintainer-facing source + is `servuo-plugins/patches/tier.json`; the release workflow folds it in and removes the staged + copy, so the tarball carries exactly one statement of the table, and gates that every `.patch` is + described by exactly one feature, that every named patch and companion exists, and that each + declared `target` is the file its diff actually edits. Installers older than this key ignore it; + an installer newer than the overlay it is deploying falls back to a built-in description of the + release that predates it (see [Phase 3 as built](#phase-3--patch-tier-opt-in)). `min_version` and `patches_verified_against` are separate on purpose. The base overlay only *adds* files and is expected to work broadly; the patch tier diffs stock ServUO files and is verified