Some checks failed
PR Checks / rust-gates (pull_request) Failing after 46s
Two features need edits to stock ServUO sources, because the events they depend on do not exist. This adds the rung ladder of PLAN.md §2.2.1, the unsupported-version path of §2.2.2, and the record and cache Phase 4 will read. Three decisions were not settled by the plan: * The engine is fully native, with no `git`. §2.2.1 wrote rung 1 as "apply verbatim with git apply", but §1 chose the release tarball specifically so there would be no git on the shard host, and rung 2 needs a native applier regardless. Rung 1 keeps its distinct, stronger verdict — the whole file reproduced the diff's `index` pre-image, computed as a git blob SHA1 in process — while the write goes through the same code path as rung 2. On the real trees here that is not academic: the shipped .patch files are CRLF in a Windows checkout and two of their three targets are LF, so `git apply` refuses patches this applies correctly. * Per-patch metadata is declared by the release, with a built-in fallback. Which patches form one all-or-nothing unit, which companion .cs follows which, whether a CORE rebuild is needed and what declining costs are not derivable from a diff. servuo-plugins now declares them; overlay v0.1.1 is in the current bundle and declares nothing, so a built-in copy stands in for it. A checked-in fixture of the release workflow's own jq output asserts the two descriptions are identical, so the repos cannot drift quietly. * Pre-images are cached in the state directory. The tier edits files the operator owns, and `/etc/runicgateway/patches/originals/` is what turns "here are the hunks we added" into a revert anyone can verify — kept out of the ServUO tree, which uninstall has promised never to clean up. Everything else follows §2.2.1: exact matching with only line-ending and trailing-whitespace normalization, exactly one occurrence or it fails, all-or-nothing per patch file and again per feature, and a byte-preserving splice so nothing outside a hunk can be reformatted. Verified against the ServUO 57.4 tree on this machine across four scratch roots: a hand-patched tree (rung 0), a reverse-applied stock one (rung 1 on the real EventSink.cs, its blob matching the patch's declared pre-image), a mixed-rung feature, a tree with edits inside two patched regions (rung 3 — nothing written, nothing held back applied, no companions copied), and a non-57.4 tree both with and without the extra consent flag. Three consecutive runs left install.json byte-identical and the cached pre-image still pre-patch. Three reporting defects the live runs caught are fixed with tests: a dry run and a held-back patch both claimed to be "applied", the core-rebuild warning fired when nothing had been written and named a Scripts file as core, and a declined tier announced the loss of features install.json showed as applied. Refused patches are now cached too, since the refusal message names that path. Refs: docs/installer/PLAN.md §2.2, §5 Phase 3 Co-Authored-By: Claude <noreply@anthropic.com>
366 lines
13 KiB
Rust
366 lines
13 KiB
Rust
//! Hashing, scratch directories, and running other programs.
|
|
|
|
use std::ffi::OsStr;
|
|
use std::fs::{self, File};
|
|
use std::io::{self, Read, Write};
|
|
use std::path::{Path, PathBuf};
|
|
use std::process::{Command, Output};
|
|
use std::time::{SystemTime, UNIX_EPOCH};
|
|
|
|
use anyhow::{bail, Context, Result};
|
|
use sha2::{Digest, Sha256};
|
|
|
|
/// Lower-case hex, written out rather than taken from a crate.
|
|
///
|
|
/// Every hash this tool handles is compared against one produced by `sha256sum` or by `jq` in CI,
|
|
/// both of which emit lower-case hex — so the formatting is part of the contract, not a display
|
|
/// choice.
|
|
pub fn hex(bytes: &[u8]) -> String {
|
|
let mut out = String::with_capacity(bytes.len() * 2);
|
|
for b in bytes {
|
|
out.push_str(&format!("{b:02x}"));
|
|
}
|
|
out
|
|
}
|
|
|
|
/// Hashes a buffer.
|
|
///
|
|
/// The run hashes files and streams almost everywhere, since they are large. The exception is the
|
|
/// patch tier, which already holds each `.patch` in memory to parse it and would otherwise re-read
|
|
/// from disk purely to hash a few kilobytes it is looking at.
|
|
pub fn sha256_bytes(bytes: &[u8]) -> String {
|
|
let mut hasher = Sha256::new();
|
|
hasher.update(bytes);
|
|
hex(&hasher.finalize())
|
|
}
|
|
|
|
/// Streams a file through SHA256 rather than reading it whole: the overlay tarball and ServUO's
|
|
/// `Scripts.dll` are both large enough that slurping them is a waste, and this same function runs
|
|
/// once per deployed file.
|
|
pub fn sha256_file(path: &Path) -> Result<String> {
|
|
let mut file =
|
|
File::open(path).with_context(|| format!("cannot read {} to hash it", path.display()))?;
|
|
let mut hasher = Sha256::new();
|
|
let mut buf = vec![0u8; 64 * 1024];
|
|
loop {
|
|
let n = file
|
|
.read(&mut buf)
|
|
.with_context(|| format!("cannot read {}", path.display()))?;
|
|
if n == 0 {
|
|
break;
|
|
}
|
|
hasher.update(&buf[..n]);
|
|
}
|
|
Ok(hex(&hasher.finalize()))
|
|
}
|
|
|
|
/// The git object name of a buffer treated as a blob: `sha1("blob " + len + "\0" + content)`.
|
|
///
|
|
/// This is what `git hash-object` prints and what a patch's `index <old>..<new>` line records, so
|
|
/// reproducing it is how the patch tier answers rung 1 — "is this whole file still the one the
|
|
/// patch was written against?" (PLAN.md §2.2.1). Computed here rather than by shelling out, because
|
|
/// the entire reason the plugin ships as a release tarball is that a shard host has no git on it
|
|
/// (§1).
|
|
///
|
|
/// The bytes are hashed exactly as they sit on disk. That matters: the three files this tier edits
|
|
/// are CRLF, and the recorded hashes were taken from those CRLF bytes, so any normalization here
|
|
/// would make every rung-1 check miss.
|
|
pub fn git_blob_hash(content: &[u8]) -> String {
|
|
use sha1::{Digest as _, Sha1};
|
|
let mut hasher = Sha1::new();
|
|
hasher.update(format!("blob {}\0", content.len()).as_bytes());
|
|
hasher.update(content);
|
|
hex(&hasher.finalize())
|
|
}
|
|
|
|
/// A [`Write`] that hashes everything passing through it.
|
|
///
|
|
/// Downloads are verified *while* being written rather than by re-reading the finished file: it
|
|
/// halves the I/O and, more importantly, means the bytes that were hashed are provably the bytes
|
|
/// that were written.
|
|
pub struct HashingWriter<W: Write> {
|
|
inner: W,
|
|
hasher: Sha256,
|
|
}
|
|
|
|
impl<W: Write> HashingWriter<W> {
|
|
pub fn new(inner: W) -> Self {
|
|
Self {
|
|
inner,
|
|
hasher: Sha256::new(),
|
|
}
|
|
}
|
|
|
|
pub fn finish(self) -> String {
|
|
hex(&self.hasher.finalize())
|
|
}
|
|
}
|
|
|
|
impl<W: Write> Write for HashingWriter<W> {
|
|
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
|
|
let n = self.inner.write(buf)?;
|
|
self.hasher.update(&buf[..n]);
|
|
Ok(n)
|
|
}
|
|
|
|
fn flush(&mut self) -> io::Result<()> {
|
|
self.inner.flush()
|
|
}
|
|
}
|
|
|
|
/// A scratch directory that deletes itself.
|
|
///
|
|
/// Downloads and the extracted overlay land here. Hand-rolled rather than pulled from a crate
|
|
/// because the requirement is one directory with a unique name and a `Drop` — and because a failed
|
|
/// cleanup must never fail the run: by the time it matters the install has already succeeded or
|
|
/// failed on its own merits.
|
|
pub struct TempDir {
|
|
path: PathBuf,
|
|
}
|
|
|
|
impl TempDir {
|
|
pub fn new(prefix: &str) -> Result<Self> {
|
|
let nanos = SystemTime::now()
|
|
.duration_since(UNIX_EPOCH)
|
|
.map(|d| d.as_nanos())
|
|
.unwrap_or(0);
|
|
let path = std::env::temp_dir().join(format!("{prefix}-{}-{nanos}", std::process::id()));
|
|
fs::create_dir_all(&path)
|
|
.with_context(|| format!("cannot create scratch directory {}", path.display()))?;
|
|
Ok(Self { path })
|
|
}
|
|
|
|
pub fn path(&self) -> &Path {
|
|
&self.path
|
|
}
|
|
}
|
|
|
|
impl Drop for TempDir {
|
|
fn drop(&mut self) {
|
|
let _ = fs::remove_dir_all(&self.path);
|
|
}
|
|
}
|
|
|
|
/// Writes a file by writing a sibling `.tmp` and renaming over the target.
|
|
///
|
|
/// `install.json` is the record every later command reasons from; a half-written one after a
|
|
/// crash or a full disk would be worse than none at all, because `doctor` and `update` would
|
|
/// believe it.
|
|
pub fn write_atomic(path: &Path, contents: &[u8]) -> Result<()> {
|
|
if let Some(parent) = path.parent() {
|
|
fs::create_dir_all(parent)
|
|
.with_context(|| format!("cannot create {}", parent.display()))?;
|
|
}
|
|
let tmp = path.with_extension("tmp");
|
|
{
|
|
let mut file =
|
|
File::create(&tmp).with_context(|| format!("cannot create {}", tmp.display()))?;
|
|
file.write_all(contents)
|
|
.with_context(|| format!("cannot write {}", tmp.display()))?;
|
|
file.sync_all()
|
|
.with_context(|| format!("cannot flush {}", tmp.display()))?;
|
|
}
|
|
// Windows will not rename onto an existing file, so the old one goes first. The window this
|
|
// opens is the reason for the .tmp file existing at all: its content is already durable.
|
|
if path.exists() {
|
|
fs::remove_file(path).with_context(|| format!("cannot replace {}", path.display()))?;
|
|
}
|
|
fs::rename(&tmp, path).with_context(|| format!("cannot move {} into place", tmp.display()))?;
|
|
Ok(())
|
|
}
|
|
|
|
/// How a command is written back to the operator when it fails.
|
|
///
|
|
/// Reproducible by hand is the whole point: every external command this tool runs — `systemctl`,
|
|
/// `useradd`, `sc.exe` — is one an operator can run themselves, and a failure they can retype is a
|
|
/// failure they can diagnose.
|
|
pub fn command_line<S: AsRef<OsStr>>(program: &str, args: &[S]) -> String {
|
|
let mut line = String::from(program);
|
|
for arg in args {
|
|
let text = arg.as_ref().to_string_lossy().into_owned();
|
|
line.push(' ');
|
|
if text.contains(' ') && !text.starts_with('"') {
|
|
line.push('"');
|
|
line.push_str(&text);
|
|
line.push('"');
|
|
} else {
|
|
line.push_str(&text);
|
|
}
|
|
}
|
|
line
|
|
}
|
|
|
|
/// Runs a program to completion, capturing its output. A non-zero exit is **not** an error here —
|
|
/// several callers ask questions whose answer *is* the exit code (`id -u`, `sc query`).
|
|
pub fn run<S: AsRef<OsStr>>(program: &str, args: &[S]) -> Result<Output> {
|
|
Command::new(program).args(args).output().with_context(|| {
|
|
format!(
|
|
"cannot run `{}` — is it installed and on PATH?",
|
|
command_line(program, args)
|
|
)
|
|
})
|
|
}
|
|
|
|
/// Runs a program and treats a non-zero exit as a failure, quoting what it printed.
|
|
///
|
|
/// **Never call this on anything that emits a secret.** The sidecar's `--print-config` writes the
|
|
/// auth token to stdout, so it is run through [`run`] and handled where the token can be kept out
|
|
/// of the error path (PLAN.md §6).
|
|
pub fn run_ok<S: AsRef<OsStr>>(program: &str, args: &[S]) -> Result<Output> {
|
|
let output = run(program, args)?;
|
|
if !output.status.success() {
|
|
bail!(failure_message(
|
|
&command_line(program, args),
|
|
output.status.code(),
|
|
&output.stderr,
|
|
&output.stdout,
|
|
));
|
|
}
|
|
Ok(output)
|
|
}
|
|
|
|
/// The message a failed command produces. Split out from [`run_ok`] because it is the part worth
|
|
/// testing — spawning a process that fails identically on Linux and Windows is not.
|
|
fn failure_message(line: &str, code: Option<i32>, stderr: &[u8], stdout: &[u8]) -> String {
|
|
let detail = first_useful_line(stderr)
|
|
.or_else(|| first_useful_line(stdout))
|
|
.unwrap_or_else(|| "(no output)".to_string());
|
|
let status = match code {
|
|
Some(code) => format!("exit code {code}"),
|
|
None => "no exit code (killed by a signal)".to_string(),
|
|
};
|
|
format!("`{line}` failed with {status}: {detail}")
|
|
}
|
|
|
|
/// The first non-blank line of a captured stream, for a one-line error message.
|
|
fn first_useful_line(bytes: &[u8]) -> Option<String> {
|
|
String::from_utf8_lossy(bytes)
|
|
.lines()
|
|
.map(str::trim)
|
|
.find(|l| !l.is_empty())
|
|
.map(str::to_string)
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
// The canonical empty-input SHA256. If this ever changes, everything else in the trust chain
|
|
// is meaningless, so it is worth one line.
|
|
const EMPTY: &str = "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855";
|
|
|
|
#[test]
|
|
fn hashing_matches_sha256sum() {
|
|
assert_eq!(sha256_bytes(b""), EMPTY);
|
|
assert_eq!(
|
|
sha256_bytes(b"abc"),
|
|
"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn file_and_byte_hashing_agree() {
|
|
let dir = TempDir::new("rg-test-hash").unwrap();
|
|
let path = dir.path().join("f.bin");
|
|
// Larger than the 64 KiB read buffer, so the streaming path is actually exercised.
|
|
let blob: Vec<u8> = (0..200_000u32).map(|i| (i % 251) as u8).collect();
|
|
fs::write(&path, &blob).unwrap();
|
|
assert_eq!(sha256_file(&path).unwrap(), sha256_bytes(&blob));
|
|
}
|
|
|
|
#[test]
|
|
fn blob_hashing_matches_git_hash_object() {
|
|
// These are the values `git hash-object` prints, and the same ones a patch's `index` line
|
|
// carries. If this drifts, rung 1 silently stops recognising a stock file and every patch
|
|
// falls through to the region match — which still works, and would hide the bug for a long
|
|
// time.
|
|
assert_eq!(
|
|
git_blob_hash(b""),
|
|
"e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"
|
|
);
|
|
assert_eq!(
|
|
git_blob_hash(b"hello\n"),
|
|
"ce013625030ba8dba906f756967f9e9ca394464a"
|
|
);
|
|
// CRLF is hashed as it sits on disk — the ServUO files this is used on are all CRLF, and
|
|
// normalizing here would make every rung-1 check miss.
|
|
assert_ne!(git_blob_hash(b"a\r\n"), git_blob_hash(b"a\n"));
|
|
}
|
|
|
|
#[test]
|
|
fn the_hashing_writer_sees_what_was_written() {
|
|
let mut w = HashingWriter::new(Vec::new());
|
|
w.write_all(b"abc").unwrap();
|
|
assert_eq!(
|
|
w.finish(),
|
|
"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn a_temp_dir_removes_itself() {
|
|
let path = {
|
|
let dir = TempDir::new("rg-test-drop").unwrap();
|
|
fs::write(dir.path().join("x"), b"x").unwrap();
|
|
dir.path().to_path_buf()
|
|
};
|
|
assert!(!path.exists());
|
|
}
|
|
|
|
#[test]
|
|
fn a_failed_command_is_reported_with_what_it_printed() {
|
|
// Both halves matter: the command to retype, and the reason it failed. stderr wins over
|
|
// stdout because that is where systemctl and sc.exe put the reason.
|
|
let message = failure_message(
|
|
"systemctl enable --now runicgateway-link.service",
|
|
Some(1),
|
|
b"Failed to enable unit: Unit file does not exist.\n",
|
|
b"noise\n",
|
|
);
|
|
assert!(message.contains("systemctl enable"), "{message}");
|
|
assert!(message.contains("exit code 1"), "{message}");
|
|
assert!(message.contains("Unit file does not exist."), "{message}");
|
|
|
|
// A command that fails silently must still say something usable.
|
|
let quiet = failure_message("sc.exe start RunicGatewayLink", Some(1053), b"", b"");
|
|
assert!(
|
|
quiet.contains("1053") && quiet.contains("(no output)"),
|
|
"{quiet}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn a_missing_program_says_so_rather_than_panicking() {
|
|
let err = run("rg-no-such-program-exists", &["x"])
|
|
.unwrap_err()
|
|
.to_string();
|
|
assert!(err.contains("rg-no-such-program-exists"), "{err}");
|
|
}
|
|
|
|
#[test]
|
|
fn command_lines_quote_arguments_containing_spaces() {
|
|
// These strings are printed for an operator to paste back; an unquoted Windows path with
|
|
// spaces in it would be a command that does not work when they do.
|
|
let line = command_line(
|
|
"sc.exe",
|
|
&[
|
|
"create",
|
|
"RunicGatewayLink",
|
|
"binPath=",
|
|
"C:\\Program Files\\x.exe",
|
|
],
|
|
);
|
|
assert!(line.contains("\"C:\\Program Files\\x.exe\""), "{line}");
|
|
}
|
|
|
|
#[test]
|
|
fn atomic_write_replaces_an_existing_file() {
|
|
let dir = TempDir::new("rg-test-atomic").unwrap();
|
|
let path = dir.path().join("nested").join("install.json");
|
|
write_atomic(&path, b"first").unwrap();
|
|
write_atomic(&path, b"second").unwrap();
|
|
assert_eq!(fs::read(&path).unwrap(), b"second");
|
|
assert!(!path.with_extension("tmp").exists());
|
|
}
|
|
}
|