feat(sidecar): protocol 8 — the asset plane, and a bound on what the shard can send
All checks were successful
PR Checks / rust-gates (pull_request) Successful in 2m39s
All checks were successful
PR Checks / rust-gates (pull_request) Successful in 2m39s
Asset Bridge phase 1, sidecar half (docs/link/v8.md §3.3, §14). Shard half: RunicGateway/servuo-plugins#28. Docs half: RunicGateway/docs#236. Three things, one of which is not additive. ## The inbound line cap (§3.3) — the one that matters `read_line` had **no bound at all**. That was survivable only because the shard had never had a reason to send a large line. Protocol 8 gives it one deliberately, and an unbounded read facing a component that now sends megabytes is a memory-exhaustion shape we would be inventing ourselves. `MAX_INBOUND_LINE_BYTES` is **1 MiB** — symmetric with the cap `BridgeLink.cs` has always applied to its own inbound lines, so both directions of this link now read the same. The shard's batch budget is 512 KiB, and the factor of two is load-bearing: a page always admits its first item even when that item alone exceeds the budget (the alternative is an oversized item skipped for the budget on every page forever), so the wire needs room for one overshoot. An over-long line is **discarded and the connection kept** — `BridgeLink.cs`'s own disposition in the other direction. Tearing the link down would take the live event feed with it over one malformed frame, and the lost reply just times out and is re-requested; everything on this plane is idempotent. **`LineReader` holds its state in a struct rather than in locals, and that is the subtle part.** This is polled inside a `tokio::select!`, so the future is dropped whenever a command wins the race. A `discarding` flag in a local would be lost with it — and losing it turns the tail of an over-long line into a line of its own, silently. There is a test for exactly that, and another for an over-long line whose terminator lands in the very chunk that crosses the cap. ## `GET /assets/sources` Stage 1 of the import gate, forwarded verbatim like everything else. `respond_assets` maps `bridge.busy` → **425** and a disabled plane → **403**. 425 deserves a note: on this plane it is not an idempotency collision, it is flow control, and it is the **ordinary** answer mid-import rather than a rare one. The shard serves one asset request at a time because its outbound queue is bounded in lines, not bytes. A caller treating it as an error would abandon a healthy transfer. 403 for the same reason the event plane's gate is a 403: `Bridge.AssetsEnabled` off is an operator declining to let the website read their client files, not a malformed request, and 400 would send an administrator hunting a bug in a correct call. ## `PROTOCOL_VERSION` 7 → 8 Paired with `servuo-plugins/overlay.toml` in the linked PR — the installer refuses to compose a bundle whose halves disagree, so a split bump fails silently at the next release. ## Also `docs/link/INTEGRATION.md` still advertised `X-UOLink-Version: 6`; it was already two versions stale before this change. Fixed in the docs PR. 61 tests pass, `cargo fmt --check` and `cargo clippy -- -D warnings` clean. Verified against the real shard: `/health` reports protocol 8, `/assets/sources` returns 200 with `X-UOLink-Version: 8`, and live events kept flowing through the new reader with no warnings logged. - [x] AI-assisted — Claude Code (Opus 5) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
This commit is contained in:
@@ -135,6 +135,12 @@ pub async fn serve(addr: &str, state: AppState) -> anyhow::Result<()> {
|
||||
// readability trap nobody wins. The only PAGED read the sidecar serves — a whole-world
|
||||
// market does not fit in one response.
|
||||
.route("/market", get(market))
|
||||
// The Asset Bridge (Protocol 8). Stage 1 of the two-stage import gate: what the shard's
|
||||
// UO client files currently are. RPC, never store-backed — unlike the boards above there
|
||||
// is nothing here worth serving stale, because the only question this answers is "have
|
||||
// the files on that host changed since the last import", and a cached answer to that is
|
||||
// worse than no answer.
|
||||
.route("/assets/sources", get(assets_sources))
|
||||
.route_layer(middleware::from_fn_with_state(state.clone(), gate));
|
||||
|
||||
let app = Router::new()
|
||||
@@ -1296,6 +1302,76 @@ async fn market(State(st): State<AppState>, Query(q): Query<PageQuery>) -> impl
|
||||
}
|
||||
}
|
||||
|
||||
// ---- the Asset Bridge (Protocol 8) ----
|
||||
|
||||
/// Stage 1 of the import gate: the shard's UO client files as they are right now — size, mtime and
|
||||
/// content hash — plus the version of the extractor that would read them, and whether this host
|
||||
/// can render an image at all.
|
||||
///
|
||||
/// The website diffs this against what it last imported and, in the overwhelmingly common case
|
||||
/// that nothing changed, stops. That is the whole reason stage 1 exists separately from the asset
|
||||
/// manifest: the normal case is a restart that changed nothing, and it has to cost nothing.
|
||||
///
|
||||
/// Forwarded verbatim, like everything else on this link. The sidecar does not know what a cliloc
|
||||
/// or an anim file is, does not cache this, and has no opinion about what the website does with
|
||||
/// the answer — the same dumb-forwarder property that keeps access control on the website where it
|
||||
/// belongs.
|
||||
async fn assets_sources(State(st): State<AppState>) -> impl IntoResponse {
|
||||
let req_id = st.rpc.next_req_id();
|
||||
let cmd = json!({"kind": "assets.sources", "reqId": req_id});
|
||||
|
||||
respond_assets(st.rpc.call(&st.shard, cmd, &req_id).await)
|
||||
}
|
||||
|
||||
/// Maps an asset-plane reply to a status.
|
||||
///
|
||||
/// Two of these matter more than the rest and neither is the generic responder's answer:
|
||||
///
|
||||
/// **`bridge.busy` is a 425**, as everywhere else. On this plane it is not an idempotency
|
||||
/// collision, it is flow control: the shard serves one asset request at a time on purpose, because
|
||||
/// its outbound queue is bounded in *lines* and a queue of large replies is how the shard runs out
|
||||
/// of memory. So it means "come back", it is entirely expected during an import, and a caller that
|
||||
/// treated it as an error would abandon a perfectly healthy transfer.
|
||||
///
|
||||
/// **The plane being switched off is a 403.** `Bridge.AssetsEnabled` is an operator declining to
|
||||
/// let the website read their client files off this host — a deliberate refusal, not a malformed
|
||||
/// request — and answering 400 would send an administrator hunting a bug in a call that is written
|
||||
/// correctly. Same argument the event plane's gate made in protocol 7.
|
||||
fn respond_assets(result: Result<Value, RpcError>) -> (StatusCode, Json<Value>) {
|
||||
match result {
|
||||
Ok(value) => {
|
||||
let kind = value.get("kind").and_then(|k| k.as_str()).unwrap_or("");
|
||||
|
||||
if kind == BUSY_KIND {
|
||||
(BUSY_STATUS, Json(value))
|
||||
} else if kind == "assets.error" {
|
||||
let reason = value
|
||||
.get("reason")
|
||||
.and_then(|r| r.as_str())
|
||||
.unwrap_or("request rejected");
|
||||
|
||||
let code = if reason.contains("disabled") {
|
||||
StatusCode::FORBIDDEN
|
||||
} else {
|
||||
StatusCode::BAD_REQUEST
|
||||
};
|
||||
|
||||
(code, Json(value))
|
||||
} else {
|
||||
(StatusCode::OK, Json(value))
|
||||
}
|
||||
}
|
||||
Err(RpcError::NoShard) => (
|
||||
StatusCode::SERVICE_UNAVAILABLE,
|
||||
Json(json!({"error": "shard not connected"})),
|
||||
),
|
||||
Err(RpcError::Timeout) => (
|
||||
StatusCode::GATEWAY_TIMEOUT,
|
||||
Json(json!({"error": "shard did not reply in time"})),
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
// ---- websocket ----
|
||||
|
||||
async fn ws_upgrade(ws: WebSocketUpgrade, State(state): State<AppState>) -> impl IntoResponse {
|
||||
@@ -1366,6 +1442,84 @@ mod tests {
|
||||
StatusCode::TOO_EARLY
|
||||
);
|
||||
assert_eq!(respond_event(reply("bridge.busy")).0, StatusCode::TOO_EARLY);
|
||||
|
||||
// Protocol 8. On the asset plane `bridge.busy` is not a keyed retry colliding with itself
|
||||
// -- it is flow control, and it is the ORDINARY answer during an import rather than a rare
|
||||
// one. The shard serves one asset request at a time because its outbound queue is bounded
|
||||
// in lines, not bytes, so a queue of large replies is how it runs out of memory. A
|
||||
// responder that answered 200 here would tell the website an import step succeeded and
|
||||
// returned nothing.
|
||||
assert_eq!(
|
||||
respond_assets(reply("bridge.busy")).0,
|
||||
StatusCode::TOO_EARLY
|
||||
);
|
||||
}
|
||||
|
||||
/// The asset plane's own gate, and it is a refusal rather than a mistake: an operator who has
|
||||
/// not enabled `Bridge.AssetsEnabled` has declined to let the website read their UO client
|
||||
/// files off the shard host. 403, for the same reason the event plane's switch is a 403.
|
||||
#[test]
|
||||
fn assets_disabled_is_a_403() {
|
||||
let value = json!({
|
||||
"kind": "assets.error",
|
||||
"reqId": "r-9",
|
||||
"reason": "asset extraction is disabled on this shard"
|
||||
});
|
||||
|
||||
assert_eq!(respond_assets(Ok(value)).0, StatusCode::FORBIDDEN);
|
||||
}
|
||||
|
||||
/// Anything else the shard refuses on this plane is the caller's mistake.
|
||||
#[test]
|
||||
fn other_asset_errors_are_400() {
|
||||
let value = json!({
|
||||
"kind": "assets.error",
|
||||
"reason": "assets.sources requires a reqId"
|
||||
});
|
||||
|
||||
assert_eq!(respond_assets(Ok(value)).0, StatusCode::BAD_REQUEST);
|
||||
}
|
||||
|
||||
/// A source manifest comes back whole. Worth asserting because `respond_assets` sniffs `kind`
|
||||
/// and a family whose success kind ends in `.ok` sits one character away from the `.error`
|
||||
/// suffix the generic responder matches on -- which is exactly why this plane has its own
|
||||
/// responder and matches `assets.error` exactly rather than by suffix.
|
||||
#[test]
|
||||
fn a_source_manifest_is_a_200() {
|
||||
let value = json!({
|
||||
"kind": "assets.sources.ok",
|
||||
"reqId": "r-9",
|
||||
"extractorVersion": 1,
|
||||
"imaging": {"ok": true},
|
||||
"files": [],
|
||||
"more": false,
|
||||
"cut": "end"
|
||||
});
|
||||
|
||||
let (status, body) = respond_assets(Ok(value));
|
||||
|
||||
assert_eq!(status, StatusCode::OK);
|
||||
assert_eq!(
|
||||
body.0.get("extractorVersion").and_then(|v| v.as_i64()),
|
||||
Some(1)
|
||||
);
|
||||
}
|
||||
|
||||
/// A shard that is not connected is a 503 and a shard that did not answer in time is a 504,
|
||||
/// and the asset plane needs the second one to stay distinct more than any other plane does:
|
||||
/// hashing a 195 MB anim.mul is the one thing on this link that can genuinely outlast the
|
||||
/// 10 s reply timeout, and the website's response to that is to poll again rather than to
|
||||
/// declare the shard down.
|
||||
#[test]
|
||||
fn asset_transport_failures_keep_their_own_statuses() {
|
||||
assert_eq!(
|
||||
respond_assets(Err(RpcError::NoShard)).0,
|
||||
StatusCode::SERVICE_UNAVAILABLE
|
||||
);
|
||||
assert_eq!(
|
||||
respond_assets(Err(RpcError::Timeout)).0,
|
||||
StatusCode::GATEWAY_TIMEOUT
|
||||
);
|
||||
}
|
||||
|
||||
/// The event plane is the FIRST place `bridge.busy` is reachable on a live shard rather than
|
||||
|
||||
Reference in New Issue
Block a user