feat(sidecar): protocol 1 — the transport
The rust-link sidecar: it owns the loopback listener the Oxide bridge plugin dials into, and serves the website a WebSocket feed plus store-backed reads. Protocol 1 is deliberately three frames — server.hello, ping/pong, and one correlated server.status — because phase 1's job is to get every seam working at once with almost nothing in them. What is load-bearing rather than incidental: * The plugin is the TCP client and this process owns the listener, so a Rust server opens no extra port. Loopback is the trust boundary on that link and there is no token on it; the website-facing surface is the opposite, with auth always on and a token generated and persisted on first start. * Inbound lines are capped at 1 MiB from the start rather than after the first large frame arrives. An over-long line is discarded and the connection stays up: one malformed frame is not a reason to drop a link live events flow over. * Store-backed reads answer while the game is off, which is what lets a website render a server list during a wipe. /status is the one route that fails when the game is down, and /server answers 204 rather than a null when the game has never connected -- those are different answers and a client that cannot tell them apart renders a server that does not exist. * The two RPC failures get distinct codes. 503 means the game is down; 504 means it is up and did not answer. Different fixes. * rpc::REPLY_TIMEOUT is a ceiling every later command budget sits under: core classifies a budget overrun as retryable unconditionally, so an action whose budgetMs does not exceed it can never report retry:false. One defect found while building, which no unit test would have caught: a four-connection SQLite pool over :memory: hands out four separate empty databases, because an in-memory database is per connection. It presents as 'no such table' from a random subset of queries. The pool is now capped at one connection for an in-memory path, which is the only coherent reading of :memory: and is what makes it usable at all. Exercised end to end against a live Rust server: a server.hello travelled game -> sidecar -> module -> the public website API, and killing this process left the game untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
422
sidecar/src/web.rs
Normal file
422
sidecar/src/web.rs
Normal file
@@ -0,0 +1,422 @@
|
||||
//! The website-facing HTTP surface: a WebSocket live feed, and REST reads backed by the store.
|
||||
//!
|
||||
//! Two rules shape everything here, and both are inherited rather than invented:
|
||||
//!
|
||||
//! 1. **Authentication is always on.** `/health` is the only unauthenticated route, because
|
||||
//! monitoring must be able to reach it. Everything else is behind a shared token that the config
|
||||
//! generates on first run, so there is no state in which this process is listening without one.
|
||||
//! 2. **Every response advertises the protocol version**, and a client that declares a different
|
||||
//! one is refused `409` rather than served something it will mis-parse. A version mismatch is a
|
||||
//! deployment fault, and it should look like one.
|
||||
//!
|
||||
//! The reads are store-backed on purpose. `/server` answers the last thing the game said about
|
||||
//! itself even while the game is off, which is what lets the website render a server list during a
|
||||
//! wipe or a restart. `/status` is the one route that round-trips the plugin, and it is the one
|
||||
//! route that fails when the game is down — which is the honest answer to "what is it doing *right
|
||||
//! now*".
|
||||
|
||||
use std::sync::atomic::{AtomicI64, Ordering};
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use axum::{
|
||||
extract::ws::{Message, WebSocket, WebSocketUpgrade},
|
||||
extract::{Query, Request, State},
|
||||
http::{HeaderValue, StatusCode},
|
||||
middleware::{self, Next},
|
||||
response::{IntoResponse, Response},
|
||||
routing::get,
|
||||
Json, Router,
|
||||
};
|
||||
use serde::Deserialize;
|
||||
use serde_json::{json, Value};
|
||||
use tokio::sync::broadcast;
|
||||
use tracing::{debug, info, warn};
|
||||
|
||||
use crate::game::GameHandle;
|
||||
use crate::rpc::{Rpc, RpcError};
|
||||
use crate::store::Store;
|
||||
use crate::PROTOCOL_VERSION;
|
||||
|
||||
/// The header every response carries, and the one a client may send to declare its own version.
|
||||
pub const VERSION_HEADER: &str = "X-RustLink-Version";
|
||||
|
||||
/// Where the live feed lives. Named rather than spelled inline because `--print-config` reports it
|
||||
/// to the installer, and the two must not drift.
|
||||
pub const WS_PATH: &str = "/ws";
|
||||
|
||||
/// Shared state handed to each request handler.
|
||||
#[derive(Clone)]
|
||||
pub struct AppState {
|
||||
pub events: broadcast::Sender<String>,
|
||||
pub game: GameHandle,
|
||||
pub rpc: Rpc,
|
||||
pub store: Store,
|
||||
/// Shared secret the website must present. Always set (config guarantees non-empty).
|
||||
pub token: Arc<String>,
|
||||
pub started: Instant,
|
||||
/// Epoch ms of the last line received from the plugin, 0 if none yet.
|
||||
pub last_event: Arc<AtomicI64>,
|
||||
}
|
||||
|
||||
pub async fn serve(addr: &str, state: AppState) -> anyhow::Result<()> {
|
||||
let protected = Router::new()
|
||||
.route(WS_PATH, get(ws_upgrade))
|
||||
// The board: the last `server.hello`. Store-backed, so it answers while the game is off.
|
||||
.route("/server", get(server_board))
|
||||
// Event history, newest first, optionally filtered by kind.
|
||||
.route("/events", get(events))
|
||||
// Live: a correlated round trip to the plugin. Fails when the game is down, by design.
|
||||
.route("/status", get(status))
|
||||
.route_layer(middleware::from_fn_with_state(state.clone(), gate));
|
||||
|
||||
let app = Router::new()
|
||||
.route("/health", get(health))
|
||||
.merge(protected)
|
||||
// Every response advertises the protocol version, so a client can notice a mismatch even
|
||||
// on /health or on an error response.
|
||||
.layer(middleware::from_fn(version_header))
|
||||
.with_state(state);
|
||||
|
||||
let listener = tokio::net::TcpListener::bind(addr).await?;
|
||||
info!(addr = %listener.local_addr()?, "web server listening");
|
||||
axum::serve(listener, app).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ---- health ----
|
||||
|
||||
/// Protocol version, whether the plugin is connected, database reachability, uptime, and when the
|
||||
/// plugin last sent anything. Unauthenticated, so monitoring can reach it.
|
||||
async fn health(State(st): State<AppState>) -> impl IntoResponse {
|
||||
let plugin = st.game.is_connected().await;
|
||||
let db_ok = st.store.ping().await.is_ok();
|
||||
let last_ms = st.last_event.load(Ordering::Relaxed);
|
||||
|
||||
let status = if plugin && db_ok { "ok" } else { "degraded" };
|
||||
|
||||
Json(json!({
|
||||
"status": status,
|
||||
"protocol": PROTOCOL_VERSION,
|
||||
"plugin_connected": plugin,
|
||||
"database": if db_ok { "ok" } else { "error" },
|
||||
"uptime": format_uptime(st.started.elapsed()),
|
||||
"last_event": iso_ms(last_ms),
|
||||
}))
|
||||
}
|
||||
|
||||
fn format_uptime(d: Duration) -> String {
|
||||
let secs = d.as_secs();
|
||||
let (days, hours, mins) = (secs / 86400, (secs % 86400) / 3600, (secs % 3600) / 60);
|
||||
if days > 0 {
|
||||
format!("{days}d {hours}h")
|
||||
} else if hours > 0 {
|
||||
format!("{hours}h {mins}m")
|
||||
} else {
|
||||
format!("{mins}m")
|
||||
}
|
||||
}
|
||||
|
||||
fn iso_ms(ms: i64) -> Option<String> {
|
||||
if ms <= 0 {
|
||||
return None;
|
||||
}
|
||||
chrono::DateTime::from_timestamp_millis(ms)
|
||||
.map(|dt| dt.format("%Y-%m-%dT%H:%M:%SZ").to_string())
|
||||
}
|
||||
|
||||
// ---- reads ----
|
||||
|
||||
/// The last `server.hello`, or `204` if the game has never connected.
|
||||
///
|
||||
/// `204` rather than `200` with a null: "we have never heard from this server" and "this server
|
||||
/// reports nothing" are different answers, and a client that cannot tell them apart renders a
|
||||
/// server that does not exist.
|
||||
async fn server_board(State(st): State<AppState>) -> Response {
|
||||
match st.store.server_state().await {
|
||||
Ok(Some(v)) => Json(v).into_response(),
|
||||
Ok(None) => StatusCode::NO_CONTENT.into_response(),
|
||||
Err(e) => {
|
||||
warn!(error = %e, "server board read failed");
|
||||
store_error()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
struct EventsQuery {
|
||||
kind: Option<String>,
|
||||
limit: Option<i64>,
|
||||
}
|
||||
|
||||
async fn events(State(st): State<AppState>, Query(q): Query<EventsQuery>) -> Response {
|
||||
match st
|
||||
.store
|
||||
.recent(q.kind.as_deref(), q.limit.unwrap_or(50))
|
||||
.await
|
||||
{
|
||||
Ok(rows) => Json(json!({ "events": rows })).into_response(),
|
||||
Err(e) => {
|
||||
warn!(error = %e, "event read failed");
|
||||
store_error()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn store_error() -> Response {
|
||||
(
|
||||
StatusCode::INTERNAL_SERVER_ERROR,
|
||||
Json(json!({"error": "store read failed"})),
|
||||
)
|
||||
.into_response()
|
||||
}
|
||||
|
||||
/// A correlated round trip to the plugin: what the server is doing right now.
|
||||
async fn status(State(st): State<AppState>) -> Response {
|
||||
let req_id = st.rpc.next_req_id();
|
||||
let command = json!({ "cmd": "server.status", "reqId": req_id });
|
||||
respond(st.rpc.call(&st.game, command, &req_id).await)
|
||||
}
|
||||
|
||||
/// Maps an RPC outcome onto a status code.
|
||||
///
|
||||
/// The two failures are deliberately different codes. `NoPlugin` is `503`: the game is down and the
|
||||
/// caller should stop asking for a moment. `Timeout` is `504`: the game is up and did not answer in
|
||||
/// time, which is a different operational problem with a different fix.
|
||||
fn respond(result: Result<Value, RpcError>) -> Response {
|
||||
match result {
|
||||
Ok(v) => Json(v).into_response(),
|
||||
Err(RpcError::NoPlugin) => (
|
||||
StatusCode::SERVICE_UNAVAILABLE,
|
||||
Json(json!({"error": "no plugin connected"})),
|
||||
)
|
||||
.into_response(),
|
||||
Err(RpcError::Timeout) => (
|
||||
StatusCode::GATEWAY_TIMEOUT,
|
||||
Json(json!({"error": "the plugin did not reply in time"})),
|
||||
)
|
||||
.into_response(),
|
||||
}
|
||||
}
|
||||
|
||||
// ---- gate: protocol check + auth ----
|
||||
|
||||
/// Adds the version header to every response.
|
||||
async fn version_header(req: Request, next: Next) -> Response {
|
||||
let mut resp = next.run(req).await;
|
||||
if let Ok(v) = HeaderValue::from_str(&PROTOCOL_VERSION.to_string()) {
|
||||
resp.headers_mut().insert(VERSION_HEADER, v);
|
||||
}
|
||||
resp
|
||||
}
|
||||
|
||||
/// Guards every non-health route: first a protocol-version check (if the client declares one), then
|
||||
/// authentication. The token may arrive as `Authorization: Bearer <t>`, `X-Api-Key: <t>`, or
|
||||
/// `?token=<t>` (the last so browser WebSocket clients, which cannot set handshake headers, still
|
||||
/// authenticate). The token compare is constant-time.
|
||||
async fn gate(State(st): State<AppState>, req: Request, next: Next) -> Response {
|
||||
// Protocol version: if the client states one and it disagrees, fail loudly and specifically.
|
||||
if let Some(v) = req
|
||||
.headers()
|
||||
.get(VERSION_HEADER.to_ascii_lowercase().as_str())
|
||||
.and_then(|h| h.to_str().ok())
|
||||
{
|
||||
if v.trim() != PROTOCOL_VERSION.to_string() {
|
||||
return (
|
||||
StatusCode::CONFLICT,
|
||||
Json(json!({
|
||||
"error": "protocol version mismatch",
|
||||
"sidecar_protocol": PROTOCOL_VERSION,
|
||||
"client_protocol": v.trim(),
|
||||
})),
|
||||
)
|
||||
.into_response();
|
||||
}
|
||||
}
|
||||
|
||||
match extract_token(&req) {
|
||||
Some(t) if constant_time_eq(t.as_bytes(), st.token.as_bytes()) => next.run(req).await,
|
||||
_ => (
|
||||
StatusCode::UNAUTHORIZED,
|
||||
Json(json!({"error": "missing or invalid auth token"})),
|
||||
)
|
||||
.into_response(),
|
||||
}
|
||||
}
|
||||
|
||||
fn extract_token(req: &Request) -> Option<String> {
|
||||
// Authorization: Bearer <token>
|
||||
if let Some(v) = req
|
||||
.headers()
|
||||
.get("authorization")
|
||||
.and_then(|h| h.to_str().ok())
|
||||
{
|
||||
if let Some(rest) = v
|
||||
.strip_prefix("Bearer ")
|
||||
.or_else(|| v.strip_prefix("bearer "))
|
||||
{
|
||||
return Some(rest.trim().to_string());
|
||||
}
|
||||
}
|
||||
// X-Api-Key: <token>
|
||||
if let Some(v) = req.headers().get("x-api-key").and_then(|h| h.to_str().ok()) {
|
||||
return Some(v.trim().to_string());
|
||||
}
|
||||
// ?token=<token>
|
||||
if let Some(q) = req.uri().query() {
|
||||
for pair in q.split('&') {
|
||||
if let Some(v) = pair.strip_prefix("token=") {
|
||||
return Some(v.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Length-independent, early-return-free comparison, so a wrong token leaks no timing signal.
|
||||
fn constant_time_eq(a: &[u8], b: &[u8]) -> bool {
|
||||
if a.len() != b.len() {
|
||||
return false;
|
||||
}
|
||||
let mut diff = 0u8;
|
||||
for (x, y) in a.iter().zip(b.iter()) {
|
||||
diff |= x ^ y;
|
||||
}
|
||||
diff == 0
|
||||
}
|
||||
|
||||
// ---- live feed ----
|
||||
|
||||
async fn ws_upgrade(ws: WebSocketUpgrade, State(state): State<AppState>) -> impl IntoResponse {
|
||||
ws.on_upgrade(move |socket| ws_client(socket, state))
|
||||
}
|
||||
|
||||
async fn ws_client(mut socket: WebSocket, state: AppState) {
|
||||
let mut rx = state.events.subscribe();
|
||||
info!("ws client connected");
|
||||
|
||||
let hello = json!({"kind": "ws.hello", "protocol": PROTOCOL_VERSION}).to_string();
|
||||
if socket.send(Message::Text(hello)).await.is_err() {
|
||||
return;
|
||||
}
|
||||
|
||||
loop {
|
||||
tokio::select! {
|
||||
recv = rx.recv() => {
|
||||
match recv {
|
||||
Ok(line) => {
|
||||
if socket.send(Message::Text(line)).await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Err(broadcast::error::RecvError::Lagged(n)) => {
|
||||
warn!(skipped = n, "ws client lagged; dropping missed events");
|
||||
}
|
||||
Err(broadcast::error::RecvError::Closed) => break,
|
||||
}
|
||||
}
|
||||
msg = socket.recv() => {
|
||||
match msg {
|
||||
Some(Ok(Message::Close(_))) | None => break,
|
||||
Some(Ok(Message::Ping(p))) => { let _ = socket.send(Message::Pong(p)).await; }
|
||||
Some(Ok(other)) => debug!(?other, "ws client message ignored"),
|
||||
Some(Err(e)) => { debug!(error = %e, "ws client error"); break; }
|
||||
}
|
||||
}
|
||||
_ = tokio::time::sleep(Duration::from_secs(30)) => {
|
||||
if socket.send(Message::Ping(Vec::new())).await.is_err() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
info!("ws client disconnected");
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use axum::body::Body;
|
||||
|
||||
fn req_with(headers: &[(&str, &str)], uri: &str) -> Request {
|
||||
let mut b = Request::builder().uri(uri);
|
||||
for (k, v) in headers {
|
||||
b = b.header(*k, *v);
|
||||
}
|
||||
b.body(Body::empty()).unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_token_is_read_from_all_three_places() {
|
||||
assert_eq!(
|
||||
extract_token(&req_with(&[("authorization", "Bearer abc")], "/events")).as_deref(),
|
||||
Some("abc")
|
||||
);
|
||||
assert_eq!(
|
||||
extract_token(&req_with(&[("authorization", "bearer abc")], "/events")).as_deref(),
|
||||
Some("abc")
|
||||
);
|
||||
assert_eq!(
|
||||
extract_token(&req_with(&[("x-api-key", "abc")], "/events")).as_deref(),
|
||||
Some("abc")
|
||||
);
|
||||
// The query form exists for browser WebSocket clients, which cannot set a handshake header.
|
||||
assert_eq!(
|
||||
extract_token(&req_with(&[], "/ws?token=abc")).as_deref(),
|
||||
Some("abc")
|
||||
);
|
||||
assert_eq!(extract_token(&req_with(&[], "/events")), None);
|
||||
}
|
||||
|
||||
/// `Authorization: abc` with no scheme is not a token. Accepting it would make the header's
|
||||
/// grammar optional, and a caller that got it wrong would work here and nowhere else.
|
||||
#[test]
|
||||
fn a_bare_authorization_value_is_not_a_token() {
|
||||
assert_eq!(
|
||||
extract_token(&req_with(&[("authorization", "abc")], "/x")),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn constant_time_eq_still_compares_correctly() {
|
||||
assert!(constant_time_eq(b"abc", b"abc"));
|
||||
assert!(!constant_time_eq(b"abc", b"abd"));
|
||||
assert!(!constant_time_eq(b"abc", b"abcd"));
|
||||
assert!(constant_time_eq(b"", b""));
|
||||
}
|
||||
|
||||
/// The two RPC failures must not collapse into one code: "the game is down" and "the game is up
|
||||
/// and slow" have different fixes, and the website's client branches on the status.
|
||||
#[test]
|
||||
fn rpc_failures_map_to_distinct_codes() {
|
||||
assert_eq!(
|
||||
respond(Err(RpcError::NoPlugin)).status(),
|
||||
StatusCode::SERVICE_UNAVAILABLE
|
||||
);
|
||||
assert_eq!(
|
||||
respond(Err(RpcError::Timeout)).status(),
|
||||
StatusCode::GATEWAY_TIMEOUT
|
||||
);
|
||||
assert_eq!(respond(Ok(json!({"ok": true}))).status(), StatusCode::OK);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn uptime_reads_as_a_human_would_write_it() {
|
||||
assert_eq!(format_uptime(Duration::from_secs(90)), "1m");
|
||||
assert_eq!(format_uptime(Duration::from_secs(3 * 3600 + 120)), "3h 2m");
|
||||
assert_eq!(
|
||||
format_uptime(Duration::from_secs(2 * 86400 + 3600)),
|
||||
"2d 1h"
|
||||
);
|
||||
}
|
||||
|
||||
/// A zero timestamp means "never", and must not render as 1970.
|
||||
#[test]
|
||||
fn a_never_timestamp_is_none_rather_than_the_epoch() {
|
||||
assert_eq!(iso_ms(0), None);
|
||||
assert_eq!(iso_ms(-1), None);
|
||||
assert!(iso_ms(1_757_000_000_000).unwrap().starts_with("2025-"));
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user