9 Commits

Author SHA1 Message Date
9ecc469a5b Merge pull request 'docs(readme): the nine files the Asset Bridge added (Phase 9a)' (#35) from docs/asset-bridge-p9 into edge
Reviewed-on: #35
2026-09-14 22:25:17 +00:00
5050425b0b docs(readme): the nine files the Asset Bridge added (Phase 9a)
The file table in this README stopped at Phase 6's town crier -- ten rows for a
directory that now holds 38 files, stale across four workstreams. Filling all of
it is not this phase's job; documenting the nine files this workstream added is,
and the table now says plainly what it covers so a reader does not take it for an
inventory.

The rows carry the reasoning worth having at a glance: the validator is the
boundary that turned 22,102 confident wrong pictures into honest absences, the
catalogue's action ceiling is what stops the fallback walk serving the next
body's art, `BridgeBodies` is the one question no code outside ServUO can answer,
and `BridgeUop`/`BridgePng` are written without `System.Drawing` on purpose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 13:14:26 -05:00
2539764cf7 Merge pull request 'feat(asset-bridge): the shard's own files stop needing a shared filesystem (Phase 7)' (#34) from feat/asset-bridge-p7 into edge
Reviewed-on: #34
2026-09-14 07:36:54 +00:00
936a922487 fix(asset-bridge): an empty catalog is an absent one on every family, not just the tree
Phase 7 found this on the tree family and fixed it there. It was inline in THREE
places: the body catalogue (phase 3), statics and land (phase 5), and the tree.
`expected != null` treats "" as a real fingerprint, so a caller that serialises a
missing value as an empty string has EVERY fetch refused -- with a sentence that
names no catalog at all ("catalog  is now 8159778b"), which reads as a shard
fault rather than a caller one.

All three now go through one BridgeAssets.CatalogMismatch. Three copies of a
comparison are three chances for the next family to get it wrong in a way only a
differently-written client would ever reveal.

BridgeLeases keeps its own `expected != null` and is deliberately untouched:
there the value is a world property, where an empty string is a legitimate thing
to expect.

Verified against a live shard on a stock ServUO install, every family asked three
ways -- with a real catalog, with the field absent, and with an empty string:

  cliloc.table walk                     67,496 rows, 12 pages
  body manifest / fetch                 1,095 rows; ok all three ways
  static + land fetch                   ok all three ways
  static/land carry their OWN catalog   art 66a112c1 vs body 323f284f
  a cross-family catalog                refused 422
  tree manifest / fetch                 141 files incl. BOTH empty ones, all three ways
  empty files carry a VALID gzip member 2 rows gunzip to 0 bytes
  a STALE catalog                       still refused on body, static and tree

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 02:33:15 -05:00
13b6fc02a4 feat(asset-bridge): the shard's own files stop needing a shared filesystem (Phase 7)
The spawn atlas was the one place the platform's rule -- only the sidecar
bridges the shard -- was broken, and it was broken by the component that faces
the internet: SPAWN_ATLAS.md required the website to read the ServUO tree off a
bind mount or a shared volume. This serves those files over the loopback link
instead (docs/link/v8.md 10).

The measurement came first and changed the shape. 10 said the shard would serve
`tree/<label>` -> bytes; against a stock 57.4 tree it cannot. Spawns/trammel.xml
is 4.03 MB, the sidecar discards any inbound line over 1 MiB, and that file as
one base64 row is 5.4 MiB -- it would be dropped, time out, and be re-requested
forever with no error anywhere. Two files on a STOCK tree are in that state.

So a file crosses as 512 KiB chunks, each gzipped: tree/Spawns/trammel.xml/c0
and so on, which is 5's depth scheme doing the same job it does for
body/400/a0/f0 and needing no protocol change to do it. The chunk is the bound
and the compression is only the saving -- nothing guarantees an operator's files
compress, so the ceiling has to hold when they do not, and a 512 KiB chunk that
refuses to compress is still ~683 KiB of base64, inside the wire cap that
AssetBatchBytes' deliberate factor of two leaves room for.

It is a `tree` FAMILY on assets.fetch rather than 14's separate tree.* commands:
phase 5 had already learned that the command is the transport and the family is
a property of the key, and assets.manifest is generalised here the same way.
That reuses the single slot, the paging envelope, the key ceiling and the
mid-import guard -- and leaves `link` with nothing to do for the third phase
running.

But it gets its OWN consent, Bridge.TreeEnabled. AssetsEnabled is an operator
agreeing the website may read their EA-licensed UO client; this is the shard's
own configuration, which they wrote, and which the public bestiary is built
from. One switch could not express both, and the thing that would silently
disappear for an operator who declined the first is their spawn atlas. So the
consent check moved into the family lookup, and assets.sources answers whenever
either plane is on, reporting `families` filtered to what is actually enabled --
which is how a tree-only shard's website discovers there is anything to ask for.

Two defects found, and which harness found which is the part worth keeping:

  - An empty `catalog` is not an absent one. `expected != null` refused every
    fetch from a caller that sent "", with a sentence naming no catalog at all.
    Found by an offline probe that passed one by accident.
  - GZipStream writes NOTHING for zero bytes of input -- the header is emitted
    lazily, so a stream opened and closed without a write yields a zero-length
    buffer rather than the 20-byte empty member. Stock ServUO ships two empty
    decoration files, so this broke every import off an untouched tree. The
    offline probe reassembled all 141 files and reported success, because .NET's
    own decompressor reads an empty stream as empty data and the chunk's
    declared length (0) and hash (of nothing) both agreed. Only the live walk,
    through a reader on another runtime, disagreed.

Measured end to end against a live shard, the real sidecar and the website's own
reader: 141 files, 11,895,427 bytes, 158 chunks, 3 pages, 1.33 MB on the wire,
512 ms; every file byte-identical to disk; the atlas built over the bridge
identical to the one built off it. A drift check is the manifest alone -- 32 KB,
~70 ms, no file bytes.

The label set is this shard's, never the caller's: a fetch resolves against the
set the shard itself enumerated, and tree/../../Scripts/..., Config/Bridge.cfg
and Saves/Accounts/accounts.xml are all answered `absent` before a path is built
out of them.

Protocol stays 8 and EXTRACTOR_VERSION stays 3 -- this family derives nothing,
it forwards an operator's own file unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 02:00:22 -05:00
577688b993 Merge pull request 'feat(asset-bridge): the 73 bodies action 0 could not see, and the ceiling that makes looking safe (Phase 6)' (#33) from feat/asset-bridge-p6 into edge
Reviewed-on: #33
2026-09-14 06:15:52 +00:00
a9bd18e48e feat(asset-bridge): the 73 bodies action 0 could not see, and the ceiling that makes looking safe (Phase 6)
The catalogue asked every body for action 0 and reported the rest absent. 73 of
this client's bodies have no art there and real art deeper — body 820's first
drawn action is 23, and it is a horse — so they rendered as text on the bestiary.
The catalogue now falls back to the first action that has art, and the key names
that action (`body/820/a23`). 1,022 -> 1,095 rows.

Walking the action axis is the one thing that can walk off the end of a body's
slots, and the slots after a body's band are the NEXT BODY'S. Measured here: one
action past the band, 643 of 795 legacy bodies return a fully validated picture
and 452 of those are byte-identical to body+1's action 0 (body 1 action 22 is an
ettin; body 3's is an imp, both confirmed by rendering them). Phase 0's validator
cannot catch that — the record is real — so the ceiling refuses the ADDRESS, in
ResolveAnimation where every caller already goes.

The ceiling is the index banding, never `Animations.GetAnimLength`: for a body
reaching file type 5 as id 34 that function answers 22 while the arithmetic gives
13, and the difference is nine actions of another creature's art.

A fetch serves only the key the catalogue chose for that body. `body/820/a0` and
`body/400/a2` come back `unsupported` with the chosen action alongside, never by
decoding what was asked for.

`EXTRACTOR_VERSION` 2 -> 3 (unchanged input, a different answer). Protocol stays
8 — `action` on a manifest/fetch row is additive.

Deep frame keys and the bulk-fill switch that §16 planned for this phase were
NOT built: the site displays still pictures, and a complete one-direction
animation set measures 174,453 frames / 281.5 MB against no consumer (docs
§11.2, org lead 2026-09-11).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-14 01:09:38 -05:00
b68aac41c6 Merge pull request 'feat(asset-bridge): item and land art on demand, hued where the files are (Phase 5)' (#32) from feat/asset-bridge-p5 into edge
Reviewed-on: #32
2026-09-11 11:26:40 +00:00
1be1f24562 feat(asset-bridge): item and land art on demand, hued where the files are (Phase 5)
The body catalogue is a set; this is not. This client addresses 49,152 static
ids and has art for 39,189 of them, plus 4,244 land tiles of 16,384 -- and hues
multiply the statics by three thousand. So there is no manifest and no scan:
`assets.fetch` grows two more families (`static`, `land`) and answers the keys
the website's own data names.

`assets.fetch` becomes shared plumbing. BridgeAssets now owns the command, does
the reqId/consent/key-ceiling checks once, derives the family from the keys
themselves (§5 made the key the address; a request that also named its family
would have two places to be wrong and one of them silent) and dispatches to the
reader that registered it. A batch must be of one family, because the reply
carries one `catalog` id. `assets.sources` gains `families` -- additive, so the
protocol stays 8, and EXTRACTOR_VERSION stays 2 because no existing key's bytes
change.

Two traps, both in §4.5's family -- a confident, plausible, wrong picture:

- `Art.GetStatic` memoises into a static Bitmap[0xFFFF] and hands back the SAME
  instance, while `Hue.ApplyTo` repaints in place. Hue a static once and the
  plain key comes back hued from then on, and the next hue stacks on the last.
  Measured on this client before the fix. `Files.CacheData` is now off for the
  life of the process; `TryHue` re-checks it and refuses rather than risk it,
  and the same flag decides whether a bitmap is ours to dispose.

- `PartialHue` decides whether a hue repaints every pixel or only the grey ones,
  per item id, out of `tiledata.mul` -- 13,259 of 65,536 ids on this client.
  Item 597 is a wooden screen with painted flowers: one mode reddens the
  flowers, the other the whole screen. Both decode. The first cut of this reader
  bound `TileData` to ServUO's OWN `Server.TileData` (the enclosing namespace
  beats `using Ultima;`, and it has a PartialHue flag too), which compiled and
  refused every hued key at runtime. Every such type is spelled `Ultima.` now.

Land takes no hue segment: the mode that decides how is an item flag and land
has no equivalent, so `land/3/h33` is refused rather than guessed. `h0` is not a
key either -- hue 0 means "not hued", and the plain key already names it.

Measured through the reader over the whole range: 39,189 statics and 4,244 land
tiles served, and the only refusals are the 9,963 + 12,140 empty index slots
§4.5 predicted. Nothing that carries art is refused.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016wDDVXWMDz82WqE1i969r4
2026-09-11 05:58:45 -05:00
9 changed files with 2531 additions and 238 deletions

View File

@@ -178,6 +178,26 @@ Runtime script compilation therefore had no effect, silently. `overlay/Scripts/S
| `BridgeAccountLink.cs` | `[link` account linking (Phase 5): one-time code, `link.confirm`, `WebsiteUserId` account tag. | | `BridgeAccountLink.cs` | `[link` account linking (Phase 5): one-time code, `link.confirm`, `WebsiteUserId` account tag. |
| `BridgeTownCrier.cs` | Town-crier news (Phase 6): inbound `towncrier.add` / `remove` into the global crier list, with abuse caps. | | `BridgeTownCrier.cs` | Town-crier news (Phase 6): inbound `towncrier.add` / `remove` into the global crier list, with abuse caps. |
The Asset Bridge's own files (protocol 8, `docs/link/v8.md`) — the shard reading the operator's UO
client and its own ServUO tree, and the only part of this plugin that touches files rather than the
world:
| File | Responsibility |
|------|----------------|
| `BridgeAssets.cs` | The plane's front door: the single-slot gate that answers `bridge.busy`, the 512 KiB batch budget, the paging envelope, the family registry, and the background hashing pass that fingerprints the client files without holding the slot. |
| `BridgeAssetValidator.cs` | Judges an index entry (and, for statics, the record behind it) **before** handing an id to `Ultima`. The boundary that turns 22,102 confident wrong pictures on a stock client into honest absences (§4.5). |
| `BridgeCatalog.cs` | The body catalogue: which bodies have art, at which action, and the per-body action ceiling that stops the fallback walk serving the next body's picture (§4.10). |
| `BridgeArt.cs` | Item statics and land tiles on demand, hued on the shard from `tiledata.mul`, behind a byte-bounded cache. |
| `BridgeUop.cs` | The narrow UOP animation reader, written without `System.Drawing` — the one decoder here that is not ServUO's (§4.3). |
| `BridgePng.cs` | Our own PNG encoder, for the same reason. |
| `BridgeBodies.cs` | Slug → body id, on the **Core thread**: construct the type, read `Body.BodyID`, delete it. The one question no code outside ServUO can answer (§8). |
| `BridgeCliloc.cs` | The Mythic cliloc decompressor, ported from UOFiddler (Beerware) — ServUO's own `Ultima.StringList` cannot read a modern client's compressed table (§9). |
| `BridgeTree.cs` | The shard's own `Spawns/*.xml` and friends as a `tree` key family, in gzipped 512 KiB chunks, behind its own consent `Bridge.TreeEnabled` (§10). |
**The table above is the transport plus the Asset Bridge, not all 38 files** in that directory —
the streams added by protocols 3 through 7 (visibility, leases, participation, the event plane's
world verbs) are documented in their own design docs rather than here.
`Emit()` is called from the Core thread. It enqueues and returns — it never touches the socket, never blocks, never allocates a syscall. **A wedged or absent sidecar cannot stall the shard**, and that is the property everything else depends on. `Emit()` is called from the Core thread. It enqueues and returns — it never touches the socket, never blocks, never allocates a syscall. **A wedged or absent sidecar cannot stall the shard**, and that is the property everything else depends on.
## Testing ## Testing

View File

@@ -340,6 +340,30 @@ AssetScanMs=3000
AssetPlayerDirection=0 AssetPlayerDirection=0
AssetCreatureDirection=1 AssetCreatureDirection=1
# The tree plane (docs/link/v8.md §10, phase 7). A THIRD switch, for a third consent:
# the asset switch above is about this host's UO client, which came from EA. This one is
# about the shard's own configuration -- Spawns/*.xml, Data/Regions.xml,
# Data/Locations/*.xml, Config/ChampionSpawns.xml and Data/Decoration/**.cfg -- which is
# the operator's own work and is what the website's spawn atlas is built from. Before
# protocol 8 the website read those files off a shared filesystem; that was the one place
# the platform's own rule (only the sidecar bridges the shard) was broken, and broken by
# the component that faces the internet. Turning this off closes the bridge route and
# leaves that shared-filesystem path as the only way an atlas can be built.
#
# Reads only, and only those five groups. Nothing here joins a path the website sent: a
# request names a label this shard itself enumerated, or it is refused.
TreeEnabled=true
# How much of a tree file one chunk carries, BEFORE compression. Chunking is not an
# optimisation here, it is what makes a spawn file transferable: a stock trammel.xml is
# 4.03 MB, the sidecar discards any inbound line over 1 MiB, and the whole file as one
# base64 row would time out and be re-requested forever with no error anywhere. Each
# chunk is gzipped (a spawn file compresses ~18x, so a chunk is typically 40 KB on the
# wire), but the BOUND comes from the chunk rather than the compression, because nothing
# guarantees input compresses at all. Clamped to [64 KiB, 512 KiB]: at the ceiling a
# worst-case incompressible chunk is ~683 KiB of base64, which still fits the wire.
TreeChunkBytes=524288
# The test scaffolding in tools/scaffolding/ reads its own flags from this file # The test scaffolding in tools/scaffolding/ reads its own flags from this file
# (SeedOnStart, CensusOnStart, ProbeOnStart). They are absent here on purpose: # (SeedOnStart, CensusOnStart, ProbeOnStart). They are absent here on purpose:
# Config.Get returns the default of false when a key is missing, so a deployed # Config.Get returns the default of false when a key is missing, so a deployed

View File

@@ -0,0 +1,842 @@
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Globalization;
using System.IO;
using System.Text;
using Ultima;
namespace Server.Custom.Bridge
{
/// <summary>
/// **Item and land art, on demand** (docs/link/v8.md §5, §11 — protocol 8, phase 5).
///
/// The body catalogue is a *set*: 1,022 sprites, enumerated, hashed and imported in one
/// pass because a bestiary needs all of them. This is the opposite shape. This client
/// addresses **49,152 static ids** and has real art for **39,189** of them, plus 4,244 land
/// tiles of 16,384 — and then there are hues, which multiply the statics by three thousand.
/// Nothing enumerates that. So there is no manifest here and no scan: the website asks for
/// the handful of keys its own data actually names, and this answers them.
///
/// (49,152 rather than the 81,884 entries `artidx.mul` declares: <c>FileIndex</c> sizes its
/// table from the **length argument it is constructed with**, `0x10000`, not from the idx
/// file — so the addressable range is `0x10000 - 0x4000`. Reading the ceiling off the file
/// instead would invent 16,348 ids, every one of them answered out of an array nobody
/// bounded.)
///
/// ── **The keys** (§5) ──
///
/// <code>
/// static/3922 one item graphic, as the client files hold it
/// static/3922/h33 the same graphic with hue 33 applied
/// land/3 one land tile
/// </code>
///
/// ── **Why the hue is applied HERE and not on the website** ──
///
/// Because it cannot be applied correctly anywhere else, and the incorrect version looks
/// fine.
///
/// A hue is not a tint. It is a 32-entry colour ramp out of `hues.mul` indexed by a
/// pixel's own red channel — and whether it replaces *every* pixel or only the grey ones
/// is decided by the <c>PartialHue</c> flag in <c>tiledata.mul</c>, per item id. On this
/// client **13,259 of 65,536 item ids carry that flag**. Get it wrong on one of them and
/// you do not get an error: item 597 is a wooden screen with painted flowers, and hued red
/// the right way the flowers turn red, the wrong way the whole screen turns red. Both
/// decode. Both are the right size. One is wrong.
///
/// The website has neither file and never will — shipping `Hues.mul` semantics and a
/// 65,536-row flag table into Node to answer a question the shard can answer for free is
/// the same trade §2.1 already refused. So hue is part of the key, and the key is resolved
/// where the files are.
///
/// ── **The trap this phase existed to find** ──
///
/// <c>Art.GetStatic</c> memoises into a static <c>Bitmap[0xFFFF]</c> and returns **the same
/// instance** every time; <c>Hue.ApplyTo</c> repaints a bitmap **in place**. Hue a static
/// once and the library's own copy is hued from then on — the plain key comes back hued,
/// and the next hue stacks on the last. It is §4.5's failure mode (a confident, plausible,
/// wrong picture that every success count agrees with) reached through a door §4.5 never
/// looked at, because phase 0 was auditing *records* and this is the library's *cache*.
///
/// <see cref="BridgeAssets.Initialize"/> turns <c>Files.CacheData</c> off for the life of
/// the process, which makes every bitmap this file receives its own. That invariant is
/// load-bearing enough that <see cref="Render"/> **re-checks it** before applying a hue and
/// refuses rather than risk it: an invariant nothing verifies is a comment.
///
/// ── **What is validated, and against what** ──
///
/// Everything §4.5 built, reused as-is. An index entry is judged before the id is handed to
/// <c>Ultima</c> (<see cref="BridgeAssetValidator.CheckEntry"/>), a static's record header
/// and row table are walked bounded (<c>StaticSane</c>), a land record is checked against
/// the 2,024 bytes <c>LoadLand</c> reads whatever the length says (<c>LandLengthSane</c>),
/// and the bound is taken against **whichever file <c>FileIndex</c> actually opened** —
/// <c>artLegacyMUL.uop</c> on every current client, never <c>art.mul</c> (§4.6).
///
/// Two of §4.5's measurements are this family's, not the catalogue's, and they are the
/// reason all of it is here: on a **stock** client **9,963 static ids and 12,140 land ids**
/// have an index entry reading `lookup 0, length 0`, which <c>FileIndex.Seek</c> treats as
/// a hit and the decoder answers with whatever was decoded last. Measured through this
/// reader over the whole range, those are the ONLY refusals — every one of the 39,189
/// statics and 4,244 land tiles that carries art is served, which is the half of the
/// measurement that says the boundary is in the right place (§4.5).
/// </summary>
public static class BridgeArt
{
/// <summary>Item graphics. <c>static/&lt;id&gt;</c>, optionally <c>/h&lt;hue&gt;</c>.</summary>
private const string StaticFamily = "static";
/// <summary>Land tiles. <c>land/&lt;id&gt;</c>, and no hue segment — see <see cref="TryParseKey"/>.</summary>
private const string LandFamily = "land";
/// <summary>The art index addresses land at its own id and statics at <c>0x4000 + id</c>.</summary>
private const int StaticBase = 0x4000;
/// <summary>Land is addressed with <c>index &amp; 0x3FFF</c> by the library itself.</summary>
private const int LandCount = 0x4000;
/// <summary><c>hues.mul</c> holds 3,000 slots; the wire's hue 1 is slot 0.</summary>
private const int MaxHue = 3000;
public static void Initialize()
{
if (!BridgeConfig.Enabled)
return;
BridgeAssets.RegisterFamily(StaticFamily, ReplyFetch);
BridgeAssets.RegisterFamily(LandFamily, ReplyFetch);
}
// ── the cache (§11) ──────────────────────────────────────────────────────────────────
private sealed class Rendered
{
public string Key;
public string Status;
public string Reason;
public string Sha256;
public byte[] Png;
public int Width;
public int Height;
public int Hue;
public bool PartialHue;
public string Source;
public int Weight
{
get { return Png == null ? 128 : Png.Length + 128; }
}
}
private sealed class Cache
{
public string Id;
public readonly Dictionary<string, Rendered> ByKey =
new Dictionary<string, Rendered>(StringComparer.Ordinal);
/// <summary>Insertion order, for eviction. See <see cref="Remember"/>.</summary>
public readonly Queue<string> Order = new Queue<string>();
public long Bytes;
public DateTime LastUsed;
}
private static readonly object _sync = new object();
private static Cache _cache;
private static readonly TimeSpan IdleFor = TimeSpan.FromMinutes(5);
// ── assets.fetch, the static and land half ───────────────────────────────────────────
/// <summary>
/// Both families' answer to <c>assets.fetch</c>. The correlation id, the operator's
/// consent, the key ceiling and the family decision were made by
/// <see cref="BridgeAssets.OnFetch"/>; every key here belongs to this reader.
///
/// The paging envelope, the byte budget and the `catalog` guard are §3.4's and
/// phase 3's, unchanged — a caller that already walks the body catalogue walks this
/// with the same loop.
/// </summary>
private static void ReplyFetch(string reqId, List<string> keys, string expected, string cursor)
{
string imagingReason;
if (!BridgeAssets.ImagingOk(out imagingReason))
{
// §17.9: a flat refusal, not a partial answer. Every picture in this family needs
// a decoder that goes through GDI+, so there is no half of it to serve.
BridgeAssets.Fail(reqId, "UNAVAILABLE",
"this shard host cannot render images - Mono's System.Drawing needs "
+ "libgdiplus. (" + imagingReason + ")");
return;
}
string id = SourceId();
if (BridgeAssets.CatalogMismatch(expected, id))
{
BridgeAssets.Fail(reqId, "UNREADABLE",
"the shard's client files changed since that catalogue was read (catalog "
+ expected + " is now " + id + "); ask again");
return;
}
Cache cache;
lock (_sync)
{
if (_cache == null || _cache.Id != id)
_cache = new Cache { Id = id };
cache = _cache;
cache.LastUsed = DateTime.UtcNow;
}
int from = ParseKeyCursor(cursor);
var sb = BridgeJson.Begin("assets.fetch.ok");
sb.Str("reqId", reqId)
.Str("family", BridgeAssets.FamilyOfKey(keys[0]))
.Str("catalog", cache.Id)
.Num("extractorVersion", BridgeAssets.EXTRACTOR_VERSION)
.Num("asked", keys.Count)
.Num("from", from);
var page = new BridgeAssets.PageBuilder(sb, "rows", BridgeConfig.AssetBatchBytes);
using (var readers = new Readers())
{
for (int i = from; i < keys.Count; i++)
{
string row = Row(cache, readers, keys[i]);
if (!page.TryAdd(row, "k:" + (i + 1).ToString(CultureInfo.InvariantCulture)))
break;
}
}
page.Close();
sb.Num("sent", page.Count);
BridgeLink.Emit(sb.End());
Sweep();
}
/// <summary>
/// One key to one JSON row.
///
/// A key this shard cannot serve is a **row**, never a failed request: an item id with
/// no art must not cost the other three hundred on the page. The three outcomes are the
/// ones phase 3 defined, and this family adds a `reason` beside them — additive, and
/// the only way an operator learns that eight of their records are damaged rather than
/// simply absent, which is a difference §4.5 spent a whole phase establishing.
/// </summary>
private static string Row(Cache cache, Readers readers, string key)
{
Rendered item = Resolve(cache, readers, key);
var sb = new StringBuilder(2048);
sb.Append("{\"key\":");
BridgeJson.Text(sb, key);
sb.Append(",\"status\":\"").Append(item.Status).Append('"');
if (item.Reason != null)
{
sb.Append(",\"reason\":");
BridgeJson.Text(sb, item.Reason);
}
if (item.Status != "ok")
{
sb.Append('}');
return sb.ToString();
}
sb.Append(",\"sha256\":\"").Append(item.Sha256).Append('"');
sb.Append(",\"bytes\":").Append(item.Png.Length.ToString(CultureInfo.InvariantCulture));
sb.Append(",\"width\":").Append(item.Width.ToString(CultureInfo.InvariantCulture));
sb.Append(",\"height\":").Append(item.Height.ToString(CultureInfo.InvariantCulture));
if (item.Hue > 0)
{
sb.Append(",\"hue\":").Append(item.Hue.ToString(CultureInfo.InvariantCulture));
sb.Append(",\"partialHue\":").Append(item.PartialHue ? "true" : "false");
}
sb.Append(",\"source\":\"").Append(item.Source).Append('"');
sb.Append(",\"png\":\"").Append(Convert.ToBase64String(item.Png)).Append("\"}");
return sb.ToString();
}
private static Rendered Resolve(Cache cache, Readers readers, string key)
{
lock (_sync)
{
Rendered cached;
if (cache.ByKey.TryGetValue(key, out cached))
return cached;
}
Rendered item;
try
{
item = Render(readers, key);
}
catch (Exception e)
{
Console.WriteLine("[Bridge] art: {0}: {1}: {2}", key, e.GetType().Name, e.Message);
item = new Rendered
{
Key = key,
Status = "absent",
Reason = e.GetType().Name
};
}
if (item.Status == "ok")
Remember(cache, item);
return item;
}
/// <summary>
/// Holds the encoded bytes against a byte budget, evicting oldest-first.
///
/// **Oldest-first rather than least-recently-used, deliberately.** The access pattern
/// this serves is a warm pass: the website asks for the keys it has never held, stores
/// them permanently, and does not ask again. What this cache is actually for is the
/// second page of a batch, a retry after a 425, and the same picture appearing in two
/// of a page's rows — all of which insertion order serves exactly as well as recency,
/// and with no bookkeeping on the hot path. A cache whose hit pattern has no recency in
/// it should not pretend to rank by it.
///
/// Only successes are held. An absent key costs one index lookup, which is cheaper than
/// the dictionary entry that would remember it.
/// </summary>
private static void Remember(Cache cache, Rendered item)
{
lock (_sync)
{
if (cache.ByKey.ContainsKey(item.Key))
return;
cache.ByKey[item.Key] = item;
cache.Order.Enqueue(item.Key);
cache.Bytes += item.Weight;
while (cache.Bytes > BridgeConfig.AssetArtCacheBytes && cache.Order.Count > 0)
{
string oldest = cache.Order.Dequeue();
Rendered evicted;
if (!cache.ByKey.TryGetValue(oldest, out evicted))
continue;
cache.ByKey.Remove(oldest);
cache.Bytes -= evicted.Weight;
}
}
}
// ── decode ───────────────────────────────────────────────────────────────────────────
/// <summary>
/// Validate, decode, hue, encode. In that order, and the order is the point.
/// </summary>
private static Rendered Render(Readers readers, string key)
{
bool land;
int id, hue;
if (!TryParseKey(key, out land, out id, out hue))
return Unsupported(key, "not a key this shard serves");
FileIndex index = readers.Index;
if (index == null || index.Index == null)
return Absent(key, "this shard has no art file");
int at = land ? id : StaticBase + id;
if (at < 0 || at >= index.Index.Length)
return Unsupported(key, "id " + id + " is past the end of this client's art index");
string reason;
BridgeAssetValidator.Verdict verdict =
BridgeAssetValidator.CheckEntry(index, at, readers.DataLength, readers.VerdataLength, out reason);
if (verdict == BridgeAssetValidator.Verdict.Absent)
{
// The 9,962 statics and 12,140 land tiles of §4.5: an index entry that reads
// `lookup 0, length 0`, which the library treats as a hit and answers with the
// previous asset's pixels. Absent is the true answer and the only safe one.
return Absent(key, reason);
}
if (verdict != BridgeAssetValidator.Verdict.Ok)
{
// A damaged record rather than a missing one. Still absent to the website — there
// is no picture either way — but the reason is worth carrying, because this one an
// operator can act on.
Console.WriteLine("[Bridge] art: {0} refused: {1}", key, reason);
return Absent(key, reason);
}
if (land)
{
if (!BridgeAssetValidator.LandLengthSane(index, at, out reason))
{
Console.WriteLine("[Bridge] art: {0} refused: {1}", key, reason);
return Absent(key, reason);
}
}
else if (readers.Reader == null || !readers.Reader.StaticSane(index, at, out reason))
{
Console.WriteLine("[Bridge] art: {0} refused: {1}",
key, reason ?? "the art record could not be read");
return Absent(key, reason ?? "the art record could not be read");
}
// A hue is resolved BEFORE anything is decoded, so a bad one costs no pixels and, more
// to the point, cannot half-apply to a picture that then gets cached and served.
Ultima.Hue applied = null;
bool partial = false;
if (hue > 0)
{
if (!TryHue(id, hue, out applied, out partial, out reason))
return Unsupported(key, reason);
}
Bitmap bitmap = land
? Art.GetLand(id)
// `checkmaxid: false` on purpose (§4.5): the default maps an out-of-range id to 0
// and returns ITEM ZERO'S PICTURE. The id is already bounded against the index
// that was actually opened, so this can only be loud.
: Art.GetStatic(id, false);
// **Whether this bitmap is ours to dispose is the same question as whether it is ours
// to hue**, and it has the same answer. With the library's cache off — which
// `BridgeAssets.Initialize` guarantees and `TryHue` re-checks — every call decodes a
// fresh instance that nothing else holds, so not disposing it would leak one bitmap per
// fetched key. With the cache on, that instance is the library's own copy and disposing
// it would leave a disposed `Bitmap` in a static array for the next caller to fault on.
// Both mistakes are silent; the flag decides, once, here.
bool owned = !Files.CacheData;
try
{
if (bitmap == null || bitmap.Width <= 0 || bitmap.Height <= 0)
return Absent(key, "the decoder returned no picture");
if (applied != null)
applied.ApplyTo(bitmap, partial);
byte[] png = BridgeAssets.BitmapToPng(bitmap);
if (png == null)
return Absent(key, "the picture could not be encoded");
return new Rendered
{
Key = key,
Status = "ok",
Sha256 = BridgeAssets.Sha256Hex(png),
Png = png,
Width = bitmap.Width,
Height = bitmap.Height,
Hue = hue,
PartialHue = partial,
Source = readers.Source
};
}
finally
{
if (owned && bitmap != null)
bitmap.Dispose();
}
}
/// <summary>
/// Resolves one wire hue onto a ramp, and decides whether it repaints the whole sprite
/// or only its grey pixels.
///
/// Four things have to hold, and every one of them has a way of not holding that
/// produces a picture rather than an error:
///
/// **The library's cache is off.** Re-checked here because <c>ApplyTo</c> repaints in
/// place: with the cache on, this would edit the copy <c>Art</c> hands to everyone
/// else. <see cref="BridgeAssets.Initialize"/> turns it off at boot and this refuses
/// if it somehow did not, because the failure is invisible and permanent.
///
/// **`hues.mul` is present.** When it is missing <c>Hues.Initialize</c> does not throw
/// — it fills all 3,000 slots with a <c>new Hue(index)</c> whose ramp is **all zeroes**,
/// and applying one of those paints the sprite black. An all-zero ramp is therefore
/// refused whatever the reason for it; on this client there are none.
///
/// **The index is in range.** The wire's hue is 1-based — <c>Ultima.Map</c> does the
/// same <c>GetHue(hue - 1)</c> at line 450 — and <c>GetHue</c> itself masks with
/// `0x3FFF` and falls back to slot 0 rather than failing, so an out-of-range hue would
/// silently become a different colour. Bound it here instead.
///
/// **The <c>PartialHue</c> flag decides the mode**, per item id, out of
/// <c>tiledata.mul</c>. This is the one that is invisible: both modes decode, both are
/// the right size, and 13,259 of this client's item ids need the grey-only one.
/// **Land has no such flag**, which is why <see cref="TryParseKey"/> does not accept a
/// hue on a land key at all rather than guessing a mode for it.
/// </summary>
private static bool TryHue(int id, int hue, out Ultima.Hue applied, out bool partial, out string reason)
{
applied = null;
partial = false;
reason = null;
if (Files.CacheData)
{
reason = "this shard's art cache is on, so a hue cannot be applied safely";
Console.WriteLine("[Bridge] art: refusing hue {0}: {1}", hue, reason);
return false;
}
if (hue < 1 || hue > MaxHue)
{
reason = "hue " + hue + " is outside 1-" + MaxHue;
return false;
}
Ultima.Hue[] list = Ultima.Hues.List;
if (list == null || hue - 1 >= list.Length || list[hue - 1] == null)
{
reason = "this client has no hue table";
return false;
}
Ultima.Hue candidate = list[hue - 1];
if (candidate.Colors == null || AllZero(candidate.Colors))
{
reason = "hue " + hue + " has no colours in this client's hues.mul";
return false;
}
if (!TryPartialHue(id, out partial, out reason))
return false;
applied = candidate;
return true;
}
private static bool AllZero(short[] colors)
{
for (int i = 0; i < colors.Length; i++)
{
if (colors[i] != 0)
return false;
}
return true;
}
/// <summary>
/// The <c>PartialHue</c> flag for one item id.
///
/// Refuses rather than defaults when <c>tiledata.mul</c> cannot be read. Defaulting
/// either way would be a coin flip on 13,259 ids, and the losing side of it is a
/// picture that looks deliberate.
///
/// **Every type here is spelled <c>Ultima.</c> on purpose, and it is not style.**
/// ServUO declares its own <c>Server.TileData</c>, <c>Server.ItemData</c> and
/// <c>Server.TileFlag</c> — with a <c>PartialHue</c> member — in
/// <c>Server/TileData.cs</c>. This file lives in <c>Server.Custom.Bridge</c>, so the
/// enclosing namespace beats the <c>using Ultima;</c> and the unqualified spelling
/// silently binds to the *server's* table: it compiles, the flag exists, and the answer
/// comes from a file resolved through <c>Core.DataDirectories</c> rather than through
/// <c>Ultima.Files</c>, which is the one thing §4.6 says never to do — decide a picture
/// with a file other than the one the pixels came out of. The first run of this reader
/// did exactly that and refused every hued key with a <c>TypeInitializationException</c>
/// from a class this code never meant to name.
/// </summary>
private static bool TryPartialHue(int id, out bool partial, out string reason)
{
partial = false;
reason = null;
Ultima.ItemData[] table;
try
{
table = Ultima.TileData.ItemTable;
}
catch (Exception e)
{
reason = "this client's tiledata could not be read (" + e.GetType().Name + ")";
return false;
}
if (table == null || id < 0 || id >= table.Length)
{
reason = "this client's tiledata does not describe item " + id;
return false;
}
partial = (table[id].Flags & Ultima.TileFlag.PartialHue) != 0;
return true;
}
private static Rendered Absent(string key, string reason)
{
return new Rendered { Key = key, Status = "absent", Reason = reason };
}
private static Rendered Unsupported(string key, string reason)
{
return new Rendered { Key = key, Status = "unsupported", Reason = reason };
}
// ── keys, cursors and the source id ──────────────────────────────────────────────────
/// <summary>
/// <c>static/&lt;id&gt;</c>, <c>static/&lt;id&gt;/h&lt;hue&gt;</c> and
/// <c>land/&lt;id&gt;</c>.
///
/// **A land key takes no hue segment.** The client can hue a land tile, but the mode
/// that decides how is an *item* flag and land has no equivalent — so the honest answer
/// to `land/3/h33` is that this shard does not serve it, rather than a picture produced
/// by guessing. Nothing on the wire carries a hued land tile today; if something ever
/// does, it arrives with a reason to choose.
/// </summary>
private static bool TryParseKey(string key, out bool land, out int id, out int hue)
{
land = false;
id = 0;
hue = 0;
if (key == null)
return false;
string[] parts = key.Split('/');
if (parts.Length < 2 || parts.Length > 3)
return false;
if (parts[0] == LandFamily)
land = true;
else if (parts[0] != StaticFamily)
return false;
if (!Int32.TryParse(parts[1], NumberStyles.None, CultureInfo.InvariantCulture, out id))
return false;
if (id < 0)
return false;
if (land && id >= LandCount)
return false;
if (parts.Length == 2)
return true;
if (land)
return false;
string segment = parts[2];
if (segment.Length < 2 || segment[0] != 'h')
return false;
if (!Int32.TryParse(segment.Substring(1), NumberStyles.None,
CultureInfo.InvariantCulture, out hue))
return false;
// **`h0` is not a key.** Hue 0 on the wire means "this item is not hued", so the plain
// key already names its picture. Accepting `static/3922/h0` as a synonym would have
// the website store the identical PNG twice under two names, diff them separately on
// every Update, and show whichever row it happened to join against -- for a distinction
// that does not exist. The caller drops the segment instead.
return hue > 0;
}
private static int ParseKeyCursor(string cursor)
{
if (cursor == null)
return 0;
int value;
if (cursor.StartsWith("k:", StringComparison.Ordinal)
&& Int32.TryParse(cursor.Substring(2), NumberStyles.None, CultureInfo.InvariantCulture, out value))
return Math.Max(0, value);
return 0;
}
/// <summary>
/// Everything that decides these bytes, hashed into one short id — the same guard
/// phase 3 built, over this family's inputs.
///
/// Four files, and each earns its place: the art data file holds the pixels,
/// `hues.mul` holds the ramps, `tiledata.mul` decides which of the two hue modes an
/// item gets, and `verdata.mul` can patch any record in any of them. Leaving
/// `tiledata.mul` out would be the subtle one — a client patch that only flipped
/// <c>PartialHue</c> flags changes no pixel in any source file and every hued picture
/// derived from them.
/// </summary>
private static string SourceId()
{
var sb = new StringBuilder(256);
sb.Append(BridgeAssets.EXTRACTOR_VERSION);
foreach (string path in new[]
{
BridgeAssetValidator.ArtDataPath(),
FilePath("hues.mul"),
FilePath("tiledata.mul"),
FilePath("verdata.mul")
})
{
sb.Append('|');
if (path == null)
continue;
try
{
var info = new FileInfo(path);
if (!info.Exists)
continue;
sb.Append(info.Length).Append(',').Append(info.LastWriteTimeUtc.Ticks);
}
catch
{
// An unreadable file is itself a state, and one that must not change from page
// to page without being noticed. Leaving the slot empty does that.
}
}
return BridgeAssets.Sha256Hex(Encoding.UTF8.GetBytes(sb.ToString())).Substring(0, 16);
}
private static string FilePath(string name)
{
try
{
return Files.GetFilePath(name);
}
catch
{
return null;
}
}
// ── shared plumbing ──────────────────────────────────────────────────────────────────
/// <summary>
/// The art index and its record reader, opened for one reply and closed with it — the
/// same lifetime rule phase 3's <c>Readers</c> follows, and for the same reason: a page
/// decodes hundreds of sprites through them and opening them is microseconds, so
/// holding handles on the operator's client files for the life of a cache buys nothing.
/// </summary>
private sealed class Readers : IDisposable
{
public readonly FileIndex Index;
public readonly BridgeAssetValidator.RecordReader Reader;
public readonly long DataLength;
public readonly long VerdataLength;
/// <summary>
/// Which file the pixels came out of — `uop` or `legacy` — carried on every row
/// beside the body catalogue's own `source` (§4.9). On this plane it answers §4.6's
/// operator question: art added to `art.mul` while `artLegacyMUL.uop` is present is
/// never read, and a row that says `uop` is what says so.
/// </summary>
public readonly string Source;
public Readers()
{
string data = BridgeAssetValidator.ArtDataPath();
string verdata = FilePath("verdata.mul");
DataLength = BridgeAssetValidator.MulLength(data);
VerdataLength = BridgeAssetValidator.MulLength(verdata);
Source = data != null && data.EndsWith(".uop", StringComparison.OrdinalIgnoreCase)
? "uop"
: "legacy";
try
{
Index = BridgeAssetValidator.OpenArtIndex();
if (data != null)
Reader = new BridgeAssetValidator.RecordReader(data, verdata);
}
catch (Exception e)
{
Console.WriteLine("[Bridge] art: could not open the art files: {0}", e.Message);
}
}
public void Dispose()
{
if (Reader == null)
return;
try
{
Reader.Dispose();
}
catch
{
// Closing a read-only handle. Nothing useful is left to do.
}
}
}
/// <summary>
/// Lets the held pictures go once nothing has asked for one in five minutes. The id is
/// derived from the client files rather than minted per build, so a walk that spans the
/// drop resumes against the same catalogue instead of starting over.
/// </summary>
private static void Sweep()
{
lock (_sync)
{
if (_cache == null)
return;
if (DateTime.UtcNow - _cache.LastUsed > IdleFor)
_cache = null;
}
}
public static string Status()
{
lock (_sync)
{
if (_cache == null)
return "art(empty)";
return String.Format("art(id={0} held={1} bytes={2} cap={3})",
_cache.Id, _cache.ByKey.Count, _cache.Bytes, BridgeConfig.AssetArtCacheBytes);
}
}
}
}

View File

@@ -406,11 +406,123 @@ namespace Server.Custom.Bridge
return false; return false;
} }
int actions = ActionsOf(translated, fileType);
if (action >= actions)
{
// §4.10, measured in phase 6: this is the never-sweep rule again, one axis over.
// A body's slots are contiguous and the next body's begin immediately after them,
// so `index + action * 5` past the ceiling addresses ANOTHER BODY'S action — a
// real record, at a real offset, that every check below passes. Measured on this
// client: of 795 legacy bodies, 643 return a fully validated picture one action
// past their band and **452 of those are byte-identical to body+1's action 0**.
// Body 1 action 22 is an ettin; body 3 action 22 is an imp. Nothing downstream
// can tell, which is why the refusal has to be here.
reason = "body " + body + " has " + actions + " actions in file type " + fileType
+ "; action " + action + " belongs to the next body";
return false;
}
index = AnimIndexOf(translated, fileType) + (action * 5) + direction; index = AnimIndexOf(translated, fileType) + (action * 5) + direction;
return true; return true;
} }
/// <summary>
/// How many actions the index reserves for a body — the only safe ceiling, and it is
/// the banding rather than the library's own answer.
///
/// <c>Animations.GetAnimLength</c> exists and looks like the right source. It is not:
/// for a body reaching file type 5 as id 34 it answers **22** while
/// <see cref="AnimIndexOf"/> puts that body in the 65-slot band, which is **13**. The
/// two disagree on exactly one body of this client (reached by translation from body
/// 276), and taking the larger number is nine actions of somebody else's art. So the
/// count is derived from the same arithmetic that produces the offset, in the same
/// file, where the two cannot drift apart.
/// </summary>
public static bool ActionCount(int body, out int actions, out int fileType, out string reason)
{
reason = null;
actions = 0;
fileType = 0;
if (body <= 0)
{
reason = "body " + body + " is not addressable";
return false;
}
int translated = body;
int hue = 0;
try
{
Animations.Translate(ref translated, ref hue);
fileType = BodyConverter.Convert(ref translated);
}
catch (Exception e)
{
reason = "body.def/bodyconv.def lookup failed: " + e.GetType().Name;
return false;
}
if (AnimDataPath(fileType) == null)
{
reason = "bodyconv sends body " + body + " to file type " + fileType
+ ", which this client does not have";
return false;
}
actions = ActionsOf(translated, fileType);
return true;
}
/// <summary>
/// The banding of <see cref="AnimIndexOf"/>, read as an action count: a body's slots
/// are five directions per action, so the band size divided by five is how many
/// actions it owns.
/// </summary>
private static int ActionsOf(int body, int fileType)
{
return SlotsOf(body, fileType) / 5;
}
/// <summary>
/// How many index slots <see cref="AnimIndexOf"/>'s arithmetic gives this body. The
/// bands are transcribed there and their sizes here, from the same source and in the
/// same order, because a ceiling that disagrees with an offset is worse than no
/// ceiling at all.
/// </summary>
private static int SlotsOf(int body, int fileType)
{
switch (fileType)
{
case 2:
return body < 200 ? 110 : 65;
case 3:
if (body < 300)
return 65;
return body < 400 ? 110 : 175;
case 5:
// Body 34's exclusion again — it is in the second band here, so it owns 13
// actions and not 22. This is the one body `GetAnimLength` is wrong about.
if (body < 200 && body != 34)
return 110;
return body < 400 ? 65 : 175;
default: // 1 and 4 share their banding
if (body < 200)
return 110;
return body < 400 ? 65 : 175;
}
}
/// <summary> /// <summary>
/// <c>Animations.GetFileIndex</c>'s own arithmetic, which is private. The banding is /// <c>Animations.GetFileIndex</c>'s own arithmetic, which is private. The banding is
/// per file type and the boundaries differ between them, so this is transcribed rather /// per file type and the boundaries differ between them, so this is transcribed rather

View File

@@ -1,7 +1,10 @@
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
using System.Drawing;
using System.Drawing.Imaging;
using System.Globalization; using System.Globalization;
using System.IO; using System.IO;
using System.Runtime.InteropServices;
using System.Security.Cryptography; using System.Security.Cryptography;
using System.Text; using System.Text;
using System.Threading; using System.Threading;
@@ -82,8 +85,13 @@ namespace Server.Custom.Bridge
/// stock client is 235 new sprites and two of them player-character bodies; and the /// stock client is 235 new sprites and two of them player-character bodies; and the
/// player-body set no longer carries ghost ids. Every client file is byte-identical /// player-body set no longer carries ghost ids. Every client file is byte-identical
/// and the answer is different, which is precisely what this number exists to say. /// and the answer is different, which is precisely what this number exists to say.
///
/// **3** — phase 6 (§4.10, §11.2). A body with no art at action 0 is catalogued at
/// the first action that has any, and its key names that action. 74 more bodies on a
/// stock client, no existing key's bytes changed — but a body that was absent is now
/// a row, which is the same "unchanged input, different answer" this number covers.
/// </summary> /// </summary>
public const int EXTRACTOR_VERSION = 2; public const int EXTRACTOR_VERSION = 3;
// ── the one slot (§3.2) ────────────────────────────────────────────────────────────── // ── the one slot (§3.2) ──────────────────────────────────────────────────────────────
@@ -123,7 +131,62 @@ namespace Server.Custom.Bridge
if (!BridgeConfig.Enabled) if (!BridgeConfig.Enabled)
return; return;
DisableTheLibraryCache();
BridgeBoot.RegisterHandler("assets.sources", OnSources); BridgeBoot.RegisterHandler("assets.sources", OnSources);
BridgeBoot.RegisterHandler("assets.fetch", OnFetch);
// Owned here since phase 7, for the same reason `assets.fetch` moved here in phase 5:
// it is the transport, and more than one family has something to enumerate.
BridgeBoot.RegisterHandler("assets.manifest", OnManifest);
}
/// <summary>
/// **Turns <c>Ultima.Files.CacheData</c> off for the life of the process** (phase 5,
/// §17.10). One line, and it answers two separate problems that both end in a
/// confident wrong picture or an out-of-memory shard.
///
/// **The poisoning.** <c>Art.GetStatic</c> and <c>Art.GetLand</c> memoise into a
/// <c>Bitmap[0xFFFF]</c> and hand back **the same instance** on every call, while
/// <c>Hue.ApplyTo</c> repaints a bitmap **in place**. So hueing a static edits the
/// library's cached copy: measured on this client, hue item 3922 once and every later
/// request for the *plain* 3922 comes back hued, and a second hue stacks on the first.
/// Nothing downstream can see it — the row is the right size, the right shape and the
/// right id. It is §4.5's failure mode arriving through a completely different door.
///
/// **The retention.** That array is never trimmed. Decoding this client's 39,189
/// statics once would leave 74 MB of <c>Bitmap</c> in a static field of a game server,
/// kept for as long as the process lives, to serve pictures nobody asked for twice.
///
/// The obvious alternative — copy each bitmap before hueing — was rejected, and not
/// only for the retention: <c>new Bitmap(src)</c> **throws** on the
/// <c>Format16bppArgb1555</c> these decoders produce, so the copy has to name the
/// source pixel format explicitly, which is a subtlety on the wrong side of a
/// correctness boundary.
///
/// **What it costs is nothing measurable here.** <c>Animations</c> — the whole of the
/// body catalogue — does not consult this flag at all, and
/// <see cref="BridgeCatalog"/> and <see cref="BridgeArt"/> each keep their own cache of
/// *encoded PNG bytes*, which is the thing worth holding: a tenth of the size, already
/// hashed, and released when it goes idle.
///
/// It is a process-global on a library nothing else in this overlay reads, which is why
/// setting it here rather than saving and restoring it around each decode is safe —
/// and a save/restore would not be, because the asset worker is a thread.
/// </summary>
private static void DisableTheLibraryCache()
{
try
{
Files.CacheData = false;
}
catch (Exception e)
{
// A client this library cannot even open. The families report that for themselves,
// per key, with a reason; it must not stop the plugin booting.
Console.WriteLine("[Bridge] assets: could not disable the Ultima bitmap cache: {0}",
e.Message);
}
} }
public static string Status() public static string Status()
@@ -162,7 +225,11 @@ namespace Server.Custom.Bridge
return; return;
} }
if (!BridgeConfig.AssetsEnabled) // Stage 1 answers for the whole plane, not for the client files alone: since phase 7
// an operator can serve the shard's own configuration tree while declining to serve
// their UO client, and `families` is where a website discovers which. Refused only
// when there is nothing at all to report.
if (Families().Count == 0)
{ {
Fail(reqId, "DISABLED", "asset extraction is disabled on this shard"); Fail(reqId, "DISABLED", "asset extraction is disabled on this shard");
return; return;
@@ -228,6 +295,423 @@ namespace Server.Custom.Bridge
BridgeLink.Emit(sb.End()); BridgeLink.Emit(sb.End());
} }
// ── assets.fetch, and the families behind it (§5, phase 5) ───────────────────────────
/// <summary>
/// One family's answer to a fetch. Runs on the asset worker, never the Core thread.
/// </summary>
internal delegate void FamilyFetch(string reqId, List<string> keys, string catalog, string cursor);
/// <summary>
/// One family's answer to a manifest walk — everything it can serve, no payload.
/// Runs on the asset worker, never the Core thread. A family with nothing to
/// enumerate (statics and land are addressed, not listed) registers none.
/// </summary>
internal delegate void FamilyManifest(string reqId, string cursor);
/// <summary>
/// What one §5 key family registered: how to serve it, how to list it, and — since
/// phase 7 — which operator consent it answers to.
///
/// The gate is per family rather than per plane because the planes are not one
/// consent. `body`, `static` and `land` are the operator's UO CLIENT, licensed from
/// EA and read off their disk; `tree` is the shard's OWN configuration, which they
/// wrote. An operator can reasonably want the second published and not the first, and
/// before this the atlas would have been what silently disappeared when they said so.
/// </summary>
private sealed class FamilyReader
{
public FamilyFetch Fetch;
public FamilyManifest Manifest;
public Func<bool> Enabled;
public string DisabledReason;
}
private static readonly Dictionary<string, FamilyReader> _families =
new Dictionary<string, FamilyReader>(StringComparer.Ordinal);
/// <summary>
/// Claims one §5 key family for a reader.
///
/// Phase 3 gave <c>assets.fetch</c> to the body catalogue outright, which was right
/// while there was one family and wrong the moment there were three: the command is
/// the *transport*, and the family is a property of the key. So the shared parts — the
/// correlation id, the operator's consent, the key-count ceiling, and deciding which
/// reader a request belongs to — live here once, and a family only ever sees a list of
/// keys it owns.
///
/// Registration is order-independent on purpose: ServUO calls every
/// <c>Initialize</c> in an order nothing here controls, and this fills a dictionary the
/// handler does not read until a request arrives.
/// </summary>
internal static void RegisterFamily(string name, FamilyFetch fetch)
{
RegisterFamily(name, fetch, null, null, null);
}
/// <summary>
/// The full registration: a fetch reader, an optional manifest reader, and the
/// consent this family answers to.
///
/// <paramref name="enabled"/> null means the asset plane's own gate
/// (<c>Bridge.AssetsEnabled</c>), which is what every client-file family wants.
/// A family that reads something else entirely passes its own.
/// </summary>
internal static void RegisterFamily(string name, FamilyFetch fetch, FamilyManifest manifest,
Func<bool> enabled, string disabledReason)
{
lock (_families)
{
_families[name] = new FamilyReader
{
Fetch = fetch,
Manifest = manifest,
Enabled = enabled,
DisabledReason = disabledReason
};
}
}
/// <summary>
/// The families this shard can serve **right now**, for §6's stage 1 and for
/// diagnostics.
///
/// Filtered by consent rather than by registration, because that is the question the
/// website is actually asking: a family it can see in this list is one it can fetch.
/// Listing a family the operator has switched off would turn one clear refusal at
/// import time into a per-key refusal on every pass, forever — which is exactly the
/// failure `families` was added in phase 5 to prevent.
/// </summary>
internal static List<string> Families()
{
var names = new List<string>();
lock (_families)
{
foreach (var pair in _families)
{
if (EnabledFor(pair.Value))
names.Add(pair.Key);
}
}
names.Sort(StringComparer.Ordinal);
return names;
}
private static bool EnabledFor(FamilyReader reader)
{
if (reader == null)
return false;
try
{
return reader.Enabled == null ? BridgeConfig.AssetsEnabled : reader.Enabled();
}
catch
{
// A gate that throws is a gate that has not consented.
return false;
}
}
private static FamilyReader FamilyFor(string name)
{
lock (_families)
{
FamilyReader reader;
return _families.TryGetValue(name, out reader) ? reader : null;
}
}
/// <summary>
/// Resolves a named family and answers the request itself when it cannot.
///
/// Shared by <c>assets.fetch</c> and <c>assets.manifest</c> so the two cannot drift
/// apart about what "this shard does not serve that" means — and so the consent check
/// happens in exactly one place for both.
/// </summary>
private static bool Resolve(string reqId, string family, out FamilyReader reader)
{
reader = FamilyFor(family);
if (reader == null)
{
Fail(reqId, "BAD_REQUEST",
"this shard serves no '" + family + "' asset family (it serves "
+ String.Join(", ", Families().ToArray()) + ")");
return false;
}
if (!EnabledFor(reader))
{
Fail(reqId, "DISABLED", reader.DisabledReason
?? "asset extraction is disabled on this shard");
return false;
}
return true;
}
/// <summary>
/// The family segment of a §5 key: everything before the first `/`.
/// </summary>
internal static string FamilyOfKey(string key)
{
if (String.IsNullOrEmpty(key))
return null;
int slash = key.IndexOf('/');
return slash <= 0 ? null : key.Substring(0, slash);
}
/// <summary>
/// §14's `assets.fetch`, for every family.
///
/// **The family is derived from the keys and is not a request field.** §5 made the key
/// the address of an asset, so a request that had to name its family as well would have
/// two places to be wrong and one of them silent. A batch must be of one family —
/// mixing them is refused rather than split — because the reply carries a single
/// `catalog` id, and that id is what stops an operator patching their client mid-import
/// from stitching one asset set out of two. Two families, two fingerprints, and a reply
/// that claimed one of them would be lying about the other.
/// </summary>
private static void OnFetch(Dictionary<string, object> o)
{
var reqId = BridgeJson.GetString(o, "reqId");
if (reqId == null)
{
Fail(null, "BAD_REQUEST", "assets.fetch requires a reqId");
return;
}
// The consent check is NOT here any more (phase 7). It cannot be: which consent this
// request needs is a property of the keys, and the keys have not been read yet. So the
// shape checks come first and the gate happens in `Resolve`, once the family is known.
var keys = BridgeJson.GetStringList(o, "keys");
if (keys.Count == 0)
{
Fail(reqId, "BAD_REQUEST", "assets.fetch requires a non-empty `keys` array");
return;
}
if (keys.Count > BridgeConfig.AssetFetchKeys)
{
Fail(reqId, "BAD_REQUEST",
"assets.fetch takes at most " + BridgeConfig.AssetFetchKeys
+ " keys per request (asked for " + keys.Count + ")");
return;
}
string family = FamilyOfKey(keys[0]);
for (int i = 1; i < keys.Count; i++)
{
if (String.Equals(FamilyOfKey(keys[i]), family, StringComparison.Ordinal))
continue;
Fail(reqId, "BAD_REQUEST",
"assets.fetch takes keys of one family per request; this one mixes '"
+ family + "' with '" + FamilyOfKey(keys[i]) + "'");
return;
}
FamilyReader reader;
if (!Resolve(reqId, family, out reader))
return;
if (reader.Fetch == null)
{
Fail(reqId, "BAD_REQUEST",
"the '" + family + "' family cannot be fetched by key on this shard");
return;
}
var catalog = BridgeJson.GetString(o, "catalog");
var cursor = BridgeJson.GetString(o, "cursor");
FamilyFetch fetch = reader.Fetch;
Accept(reqId, "assets.fetch", () => fetch(reqId, keys, catalog, cursor));
}
/// <summary>
/// §14's `assets.manifest`, for every family that has one.
///
/// Phase 3 gave this command to the body catalogue outright and phase 5 learned, for
/// `assets.fetch`, that the command is the transport and the family is a property of
/// the key. Phase 7 is where the same lesson lands one level up: the tree family
/// enumerates its files exactly the way the catalogue enumerates its bodies, and
/// nothing about the envelope, the cursor or the consent differs between them.
///
/// **`family` still defaults to `body`.** A phase-3 website asks without naming one
/// and must keep getting the catalogue it asked for.
/// </summary>
private static void OnManifest(Dictionary<string, object> o)
{
var reqId = BridgeJson.GetString(o, "reqId");
if (reqId == null)
{
Fail(null, "BAD_REQUEST", "assets.manifest requires a reqId");
return;
}
var family = BridgeJson.GetString(o, "family") ?? "body";
FamilyReader reader;
if (!Resolve(reqId, family, out reader))
return;
if (reader.Manifest == null)
{
// Named rather than defaulted: statics and land are ADDRESSED (§11.1) rather than
// listed, and a website that asked for a list of 49,152 item graphics has made a
// mistake it needs told about rather than an empty page it will read as "none".
Fail(reqId, "BAD_REQUEST",
"the '" + family + "' family is fetched by key and has no manifest");
return;
}
var cursor = BridgeJson.GetString(o, "cursor");
FamilyManifest manifest = reader.Manifest;
Accept(reqId, "assets.manifest", () => manifest(reqId, cursor));
}
/// <summary>
/// ARGB1555 to a PNG with a transparent background.
///
/// <c>Frame</c> writes 16-bit ARGB1555: a pixel the sprite does not cover is left as
/// zero and a pixel it does cover carries the top bit set. Saving that format straight
/// to PNG asks GDI+ to make the conversion, and what it does with a one-bit alpha
/// channel varies by platform — on Mono it is a different implementation entirely. A
/// sprite that came back with a black rectangle behind it would look fine in a test
/// that only checked the bytes decoded, and wrong on every page that showed it.
///
/// So the expansion is done here, explicitly: alpha bit clear becomes fully
/// transparent, and each 5-bit channel is widened to 8 bits by repeating its high bits
/// (<c>(c &lt;&lt; 3) | (c &gt;&gt; 2)</c>) rather than by shifting alone, which would
/// cap white at 248 and tint the whole catalogue.
/// </summary>
internal static byte[] BitmapToPng(Bitmap source)
{
var rect = new Rectangle(0, 0, source.Width, source.Height);
if (source.PixelFormat != PixelFormat.Format16bppArgb1555)
{
// Not what this library has ever produced. Save it rather than reinterpret it:
// guessing at an unknown layout is how a catalogue fills with confident nonsense.
using (var ms = new MemoryStream())
{
source.Save(ms, ImageFormat.Png);
return ms.ToArray();
}
}
using (var target = new Bitmap(source.Width, source.Height, PixelFormat.Format32bppArgb))
{
BitmapData src = source.LockBits(rect, ImageLockMode.ReadOnly, PixelFormat.Format16bppArgb1555);
BitmapData dst = null;
try
{
dst = target.LockBits(rect, ImageLockMode.WriteOnly, PixelFormat.Format32bppArgb);
var line = new short[source.Width];
var outLine = new int[source.Width];
for (int y = 0; y < source.Height; y++)
{
Marshal.Copy(new IntPtr(src.Scan0.ToInt64() + ((long)y * src.Stride)),
line, 0, source.Width);
for (int x = 0; x < source.Width; x++)
{
int p = line[x] & 0xFFFF;
if ((p & 0x8000) == 0)
{
outLine[x] = 0;
continue;
}
int r = (p >> 10) & 0x1F;
int g = (p >> 5) & 0x1F;
int b = p & 0x1F;
outLine[x] = unchecked((int)0xFF000000)
| (((r << 3) | (r >> 2)) << 16)
| (((g << 3) | (g >> 2)) << 8)
| ((b << 3) | (b >> 2));
}
Marshal.Copy(outLine, 0, new IntPtr(dst.Scan0.ToInt64() + ((long)y * dst.Stride)),
source.Width);
}
}
finally
{
if (dst != null)
target.UnlockBits(dst);
source.UnlockBits(src);
}
using (var ms = new MemoryStream())
{
target.Save(ms, ImageFormat.Png);
return ms.ToArray();
}
}
}
/// <summary>
/// Does a caller's asserted catalog id disagree with what this shard holds?
///
/// **An absent fingerprint and an empty one mean the same thing**, and that is the
/// whole reason this is a function rather than an inline `expected != null`. A caller
/// with nothing to assert sends the field absent or empty depending on how its own
/// client serialises a missing value, and treating `""` as a real id refuses **every**
/// fetch it makes — with a sentence naming no catalog at all ("catalog is now
/// 8159778b"), which reads as a shard fault rather than a caller one.
///
/// Phase 7 found this on the tree family, where a probe passed an empty string by
/// accident. It was inline in three places by then; it is one function now, because
/// three copies of a comparison are three chances for the next family to get it wrong
/// in a way only a differently-written client would ever reveal.
///
/// Note this is deliberately NOT the shape `BridgeLeases` uses for its own `expected`:
/// there the value is a world property, where an empty string is a legitimate thing to
/// expect and `!= null` is correct.
/// </summary>
internal static bool CatalogMismatch(string expected, string actual)
{
return !String.IsNullOrEmpty(expected) && !String.Equals(expected, actual, StringComparison.Ordinal);
}
/// <summary>
/// SHA-256, lowercase hex. Shared because the hash in a manifest row, the hash in a
/// fetch row and the hash the website stores must be one function.
/// </summary>
internal static string Sha256Hex(byte[] bytes)
{
using (var sha = SHA256.Create())
{
byte[] digest = sha.ComputeHash(bytes);
var sb = new StringBuilder(digest.Length * 2);
foreach (byte b in digest)
sb.Append(b.ToString("x2", CultureInfo.InvariantCulture));
return sb.ToString();
}
}
/// <summary> /// <summary>
/// The asset plane's one refusal frame, shared by every family on it. /// The asset plane's one refusal frame, shared by every family on it.
/// ///
@@ -343,7 +827,13 @@ namespace Server.Custom.Bridge
var sb = BridgeJson.Begin("assets.sources.ok"); var sb = BridgeJson.Begin("assets.sources.ok");
sb.Str("reqId", reqId) sb.Str("reqId", reqId)
.Num("extractorVersion", EXTRACTOR_VERSION); .Num("extractorVersion", EXTRACTOR_VERSION)
// Which of the two consents this shard has given (phase 7). Without it a website
// whose operator switched client-file extraction off would read an empty `files`
// array as "your client has no cliloc.enu" — a sentence that sends them looking at
// their client install for a setting that lives on their shard.
.Bool("assetsEnabled", BridgeConfig.AssetsEnabled)
.Bool("treeEnabled", BridgeConfig.TreeEnabled);
WriteImaging(sb); WriteImaging(sb);
@@ -354,10 +844,32 @@ namespace Server.Custom.Bridge
sb.Str("artDataFile", artData == null ? null : Path.GetFileName(artData)); sb.Str("artDataFile", artData == null ? null : Path.GetFileName(artData));
// Which §5 families this shard can be asked for. Additive, so the protocol stays 8: a
// consumer that does not read it behaves exactly as it did. One that does can tell an
// older overlay (bodies only) from this one without discovering it as a refused fetch
// halfway through a warm pass.
var families = Families();
sb.Append(",\"families\":[");
for (int i = 0; i < families.Count; i++)
{
if (i > 0)
sb.Append(',');
BridgeJson.Text(sb, families[i]);
}
sb.Append(']');
var page = new PageBuilder(sb, "files", BridgeConfig.AssetBatchBytes); var page = new PageBuilder(sb, "files", BridgeConfig.AssetBatchBytes);
bool anyMissingHash = false; bool anyMissingHash = false;
for (int i = 0; i < SourceFiles.Length; i++) // The client files are the asset plane's own subject, so they are listed under the
// asset plane's own consent. A tree-only shard answers this call — that is how its
// website learns the `tree` family exists — and reports no client files at all,
// which is the truthful answer to "what may I read here".
for (int i = 0; BridgeConfig.AssetsEnabled && i < SourceFiles.Length; i++)
{ {
string name = SourceFiles[i]; string name = SourceFiles[i];
string path = ResolvePath(name); string path = ResolvePath(name);

View File

@@ -262,6 +262,8 @@ namespace Server.Custom.Bridge
e.Mobile.SendMessage("Bridge: {0}", BridgeOneShots.Status()); e.Mobile.SendMessage("Bridge: {0}", BridgeOneShots.Status());
e.Mobile.SendMessage("Bridge: {0}", BridgeAssets.Status()); e.Mobile.SendMessage("Bridge: {0}", BridgeAssets.Status());
e.Mobile.SendMessage("Bridge: {0}", BridgeCatalog.Status()); e.Mobile.SendMessage("Bridge: {0}", BridgeCatalog.Status());
e.Mobile.SendMessage("Bridge: {0}", BridgeArt.Status());
e.Mobile.SendMessage("Bridge: {0}", BridgeTree.Status());
break; break;
} }
} }

View File

@@ -1,11 +1,8 @@
using System; using System;
using System.Collections.Generic; using System.Collections.Generic;
using System.Drawing; using System.Drawing;
using System.Drawing.Imaging;
using System.Globalization; using System.Globalization;
using System.IO; using System.IO;
using System.Runtime.InteropServices;
using System.Security.Cryptography;
using System.Text; using System.Text;
using Ultima; using Ultima;
@@ -17,11 +14,20 @@ namespace Server.Custom.Bridge
/// ///
/// One thumbnail per creature body: the working set that makes a bestiary, a marketplace /// One thumbnail per creature body: the working set that makes a bestiary, a marketplace
/// listing and a character sheet render. Everything deeper — every action, every frame — /// listing and a character sheet render. Everything deeper — every action, every frame —
/// is the same addressing scheme at a deeper key, fetched on demand in a later phase; this /// is the same addressing scheme at a deeper key and is **not served** (§11.2, phase 6:
/// is the set that is worth importing before anything asks for it, because on this /// the site shows still pictures, so frames wait for a consumer that wants them). This is
/// machine's client it is **1,022 sprites at about a kilobyte each** — 787 out of the /// the set that is worth importing before anything asks for it, because on this machine's
/// legacy `anim*.mul` files and, since phase 4, 235 more out of `AnimationFrame*.uop`, /// client it is **1,096 sprites at about a kilobyte each** — 787 out of the legacy
/// which ServUO's vendored decoder never opens (§4.3, §4.9). /// `anim*.mul` files, 235 more out of `AnimationFrame*.uop` since phase 4 (§4.3, §4.9),
/// and 74 more since phase 6, which have no art at action 0 and real art deeper.
///
/// ── **One picture per body, at the first action that has one** ──
///
/// A key carries the action it came from — `body/820/a23` for a horse whose action 0 is
/// empty — so the catalogue is still exactly one row per body, and the row says which
/// picture it is. What it never does is decode an action the walk did not choose: a fetch
/// for `body/820/a0` is `unsupported`, not a second attempt, because the slots past a
/// body's band belong to the next body and every check passes on them (§4.10).
/// ///
/// Two request kinds, which are §6's two stages for assets rather than for sources: /// Two request kinds, which are §6's two stages for assets rather than for sources:
/// ///
@@ -92,16 +98,34 @@ namespace Server.Custom.Bridge
/// <summary>Bodies are addressable to 2047; the sweep behind §4.8 covered exactly this.</summary> /// <summary>Bodies are addressable to 2047; the sweep behind §4.8 covered exactly this.</summary>
private const int MaxBody = 2047; private const int MaxBody = 2047;
/// <summary>The catalogue is first frames only. Deep keys are phase 6.</summary> /// <summary>
private const int CatalogAction = 0; /// The action a thumbnail comes from when the body has one, which is nearly always.
/// Everything deeper than a first frame is deferred — see §11.2.
/// </summary>
private const int PreferredAction = 0;
/// <summary>
/// How far the fallback looks for a body with no art at <see cref="PreferredAction"/>.
///
/// 35 because that is the largest band any file type gives a body, so an action beyond
/// it is not something the client's own layout can name. The legacy arm is bounded
/// tighter still and per body, by <see cref="BridgeAssetValidator.ActionCount"/> —
/// this is only the scan's outer stop, and it is the UOP arm's real one, where an
/// action is a named entry rather than an offset.
/// </summary>
private const int MaxAction = 35;
public static void Initialize() public static void Initialize()
{ {
if (!BridgeConfig.Enabled) if (!BridgeConfig.Enabled)
return; return;
BridgeBoot.RegisterHandler("assets.manifest", OnManifest); // Both commands are shared plumbing: `assets.fetch` since phase 5 and
BridgeBoot.RegisterHandler("assets.fetch", OnFetch); // `assets.manifest` since phase 7 (§5, §10). BridgeAssets owns the correlation id, the
// operator's consent, the key ceiling and the family decision; what is registered here
// is only this family's two readers, and each is called on the asset worker with work
// it owns.
BridgeAssets.RegisterFamily(Family, ReplyFetch, ReplyManifest, null, null);
} }
// ── the cache ──────────────────────────────────────────────────────────────────────── // ── the cache ────────────────────────────────────────────────────────────────────────
@@ -110,6 +134,14 @@ namespace Server.Custom.Bridge
{ {
public string Key; public string Key;
public int Body; public int Body;
/// <summary>
/// Which action this body's thumbnail came from — <see cref="PreferredAction"/>
/// for all but 74 bodies on this client, and on the wire because the key names it
/// (§5, §11.2). A consumer that assumes `a0` would build a dead URL for a horse.
/// </summary>
public int Action;
public int Direction; public int Direction;
public int FileType; public int FileType;
public string Sha256; public string Sha256;
@@ -130,8 +162,15 @@ namespace Server.Custom.Bridge
private sealed class Catalog private sealed class Catalog
{ {
public string Id; public string Id;
public readonly Dictionary<string, Sprite> ByKey =
new Dictionary<string, Sprite>(StringComparer.Ordinal); /// <summary>
/// Keyed by **body**, not by asset key, since phase 6: a body's key now carries
/// the action its picture came from, so the key cannot be spelled until the body
/// has been resolved. A fetch arrives holding a key and has to reach the same
/// sprite, which it does by parsing the body out of it and comparing.
/// </summary>
public readonly Dictionary<int, Sprite> ByBody = new Dictionary<int, Sprite>();
public readonly List<Sprite> Order = new List<Sprite>(); public readonly List<Sprite> Order = new List<Sprite>();
/// <summary>The next body the scan has yet to look at.</summary> /// <summary>The next body the scan has yet to look at.</summary>
@@ -148,32 +187,6 @@ namespace Server.Custom.Bridge
// ── assets.manifest ────────────────────────────────────────────────────────────────── // ── assets.manifest ──────────────────────────────────────────────────────────────────
private static void OnManifest(Dictionary<string, object> o)
{
string reqId;
if (!Admit(o, "assets.manifest", out reqId))
return;
var family = BridgeJson.GetString(o, "family") ?? Family;
if (!String.Equals(family, Family, StringComparison.Ordinal))
{
// Named rather than ignored: `family` exists so §5's statics and land can join
// this envelope in phase 5 without a second request kind, and a website that
// asked for one of those against a phase-3 overlay must be told it asked too
// early rather than handed a body catalogue it did not request.
BridgeAssets.Fail(reqId, "BAD_REQUEST",
"this shard serves the '" + Family + "' asset family only (asked for '"
+ family + "')");
return;
}
var cursor = BridgeJson.GetString(o, "cursor");
BridgeAssets.Accept(reqId, "assets.manifest", () => ReplyManifest(reqId, cursor));
}
/// <summary> /// <summary>
/// Worker thread. Scans forward from the cursor until the byte budget or the time /// Worker thread. Scans forward from the cursor until the byte budget or the time
/// budget is spent, hashing what it decodes and keeping the bytes for the fetch. /// budget is spent, hashing what it decodes and keeping the bytes for the fetch.
@@ -264,6 +277,7 @@ namespace Server.Custom.Bridge
item.Append(",\"width\":").Append(sprite.Width.ToString(CultureInfo.InvariantCulture)); item.Append(",\"width\":").Append(sprite.Width.ToString(CultureInfo.InvariantCulture));
item.Append(",\"height\":").Append(sprite.Height.ToString(CultureInfo.InvariantCulture)); item.Append(",\"height\":").Append(sprite.Height.ToString(CultureInfo.InvariantCulture));
item.Append(",\"body\":").Append(sprite.Body.ToString(CultureInfo.InvariantCulture)); item.Append(",\"body\":").Append(sprite.Body.ToString(CultureInfo.InvariantCulture));
item.Append(",\"action\":").Append(sprite.Action.ToString(CultureInfo.InvariantCulture));
item.Append(",\"direction\":").Append(sprite.Direction.ToString(CultureInfo.InvariantCulture)); item.Append(",\"direction\":").Append(sprite.Direction.ToString(CultureInfo.InvariantCulture));
item.Append(",\"source\":\"").Append(sprite.Source).Append('"'); item.Append(",\"source\":\"").Append(sprite.Source).Append('"');
item.Append('}'); item.Append('}');
@@ -324,36 +338,11 @@ namespace Server.Custom.Bridge
// ── assets.fetch ───────────────────────────────────────────────────────────────────── // ── assets.fetch ─────────────────────────────────────────────────────────────────────
private static void OnFetch(Dictionary<string, object> o) /// <summary>
{ /// The `body` family's half of <c>assets.fetch</c>. The correlation id, the operator's
string reqId; /// consent, the key ceiling and the family decision have already been made by
/// <see cref="BridgeAssets.RegisterFamily"/>'s caller; every key here is this family's.
if (!Admit(o, "assets.fetch", out reqId)) /// </summary>
return;
var keys = BridgeJson.GetStringList(o, "keys");
if (keys.Count == 0)
{
BridgeAssets.Fail(reqId, "BAD_REQUEST",
"assets.fetch requires a non-empty `keys` array");
return;
}
if (keys.Count > BridgeConfig.AssetFetchKeys)
{
BridgeAssets.Fail(reqId, "BAD_REQUEST",
"assets.fetch takes at most " + BridgeConfig.AssetFetchKeys
+ " keys per request (asked for " + keys.Count + ")");
return;
}
var catalog = BridgeJson.GetString(o, "catalog");
var cursor = BridgeJson.GetString(o, "cursor");
BridgeAssets.Accept(reqId, "assets.fetch", () => ReplyFetch(reqId, keys, catalog, cursor));
}
private static void ReplyFetch(string reqId, List<string> keys, string expected, string cursor) private static void ReplyFetch(string reqId, List<string> keys, string expected, string cursor)
{ {
string imagingReason; string imagingReason;
@@ -368,7 +357,7 @@ namespace Server.Custom.Bridge
string id = SourceId(); string id = SourceId();
if (expected != null && expected != id) if (BridgeAssets.CatalogMismatch(expected, id))
{ {
// The client files moved between the manifest and this fetch. Refusing is the only // The client files moved between the manifest and this fetch. Refusing is the only
// honest answer: the keys were chosen against a catalogue that no longer describes // honest answer: the keys were chosen against a catalogue that no longer describes
@@ -438,9 +427,9 @@ namespace Server.Custom.Bridge
/// </summary> /// </summary>
private static string Render(Catalog catalog, Readers readers, string key) private static string Render(Catalog catalog, Readers readers, string key)
{ {
int body; int body, action;
if (!TryParseKey(key, out body)) if (!TryParseKey(key, out body, out action))
{ {
var bad = new StringBuilder(96); var bad = new StringBuilder(96);
bad.Append("{\"key\":"); bad.Append("{\"key\":");
@@ -462,12 +451,26 @@ namespace Server.Custom.Bridge
return item.ToString(); return item.ToString();
} }
if (sprite.Action != action)
{
// The body has a picture, but not at the action this key names. Two ways to get
// here and both are the caller's: an old manifest that catalogued this body at
// `a0` before the client was patched, or a key someone built by assuming the
// action. Neither is served — decoding the asked-for action instead would be
// §4.10's wrong picture, arrived at politely.
item.Append(",\"status\":\"unsupported\"");
item.Append(",\"action\":").Append(sprite.Action.ToString(CultureInfo.InvariantCulture));
item.Append('}');
return item.ToString();
}
item.Append(",\"status\":\"ok\""); item.Append(",\"status\":\"ok\"");
item.Append(",\"sha256\":\"").Append(sprite.Sha256).Append('"'); item.Append(",\"sha256\":\"").Append(sprite.Sha256).Append('"');
item.Append(",\"bytes\":").Append(sprite.Png.Length.ToString(CultureInfo.InvariantCulture)); item.Append(",\"bytes\":").Append(sprite.Png.Length.ToString(CultureInfo.InvariantCulture));
item.Append(",\"width\":").Append(sprite.Width.ToString(CultureInfo.InvariantCulture)); item.Append(",\"width\":").Append(sprite.Width.ToString(CultureInfo.InvariantCulture));
item.Append(",\"height\":").Append(sprite.Height.ToString(CultureInfo.InvariantCulture)); item.Append(",\"height\":").Append(sprite.Height.ToString(CultureInfo.InvariantCulture));
item.Append(",\"body\":").Append(sprite.Body.ToString(CultureInfo.InvariantCulture)); item.Append(",\"body\":").Append(sprite.Body.ToString(CultureInfo.InvariantCulture));
item.Append(",\"action\":").Append(sprite.Action.ToString(CultureInfo.InvariantCulture));
item.Append(",\"direction\":").Append(sprite.Direction.ToString(CultureInfo.InvariantCulture)); item.Append(",\"direction\":").Append(sprite.Direction.ToString(CultureInfo.InvariantCulture));
item.Append(",\"source\":\"").Append(sprite.Source).Append('"'); item.Append(",\"source\":\"").Append(sprite.Source).Append('"');
item.Append(",\"png\":\"").Append(Convert.ToBase64String(sprite.Png)).Append("\"}"); item.Append(",\"png\":\"").Append(Convert.ToBase64String(sprite.Png)).Append("\"}");
@@ -484,13 +487,11 @@ namespace Server.Custom.Bridge
/// </summary> /// </summary>
private static Sprite Resolve(Catalog catalog, Readers readers, int body) private static Sprite Resolve(Catalog catalog, Readers readers, int body)
{ {
string key = Key(body);
lock (_sync) lock (_sync)
{ {
Sprite cached; Sprite cached;
if (catalog.ByKey.TryGetValue(key, out cached)) if (catalog.ByBody.TryGetValue(body, out cached))
return cached; return cached;
} }
@@ -498,38 +499,88 @@ namespace Server.Custom.Bridge
? BridgeConfig.AssetPlayerDirection ? BridgeConfig.AssetPlayerDirection
: BridgeConfig.AssetCreatureDirection; : BridgeConfig.AssetCreatureDirection;
// Legacy first, always. The vendored decoder is what 787 of this client's bodies come Sprite sprite = ResolveAny(readers, body, direction);
// out of, it is what phase 3 measured, and the UOP packages hold a different and
// mostly disjoint set (measured: of the 244 bodies they carry, 8 also have legacy
// art). So this is a fallback rather than a choice, and no body changes reader while
// a client sits still.
Sprite sprite = ResolveLegacy(key, readers, body, direction)
?? ResolveUop(key, readers, body, direction);
if (sprite == null) if (sprite == null)
return null; return null;
lock (_sync) lock (_sync)
{ {
if (!catalog.ByKey.ContainsKey(key)) if (!catalog.ByBody.ContainsKey(body))
{ {
catalog.ByKey[key] = sprite; catalog.ByBody[body] = sprite;
catalog.Order.Add(sprite); catalog.Order.Add(sprite);
} }
return catalog.ByKey[key]; return catalog.ByBody[body];
} }
} }
/// <summary>
/// One body's thumbnail: action 0 if it has one, otherwise the first action that does.
///
/// ── **Why there is a fallback at all** ──
///
/// Through phase 5 a body with no art at action 0 was simply absent, and on this
/// client **74 bodies are in exactly that state while carrying real art deeper** — 66
/// of them UOP, 8 legacy. Body 820's first drawn action is 23, and it is a horse.
/// They rendered as text on the bestiary for want of looking one action further.
///
/// ── **Why the key says which action it is** ──
///
/// The fallback's picture is `body/820/a23`, not `body/820/a0`. Naming it `a0` would
/// have been fewer changes downstream and a key that lies about its content, which is
/// the failure this protocol keeps meeting from other directions (§4.5, §4.8, §11.1).
///
/// ── **Why the ceiling is not a detail** ──
///
/// Scanning actions is the one thing that can walk off the end of a body's slots, and
/// the slots immediately after a body's are the **next body's**. Measured in phase 6:
/// 643 of 795 legacy bodies return a fully validated, correctly-sized picture one
/// action past their band, and 452 of those are byte-identical to body+1's action 0.
/// <see cref="BridgeAssetValidator.ResolveAnimation"/> refuses past the ceiling, so
/// this walk cannot produce one — see §4.10.
/// </summary>
private static Sprite ResolveAny(Readers readers, int body, int direction)
{
int actions, fileType;
string reason;
// The legacy ceiling. A body the legacy path cannot place at all still gets the UOP
// arm below, where an action is a named entry rather than an offset into a band.
if (!BridgeAssetValidator.ActionCount(body, out actions, out fileType, out reason))
actions = 0;
for (int action = PreferredAction; action < MaxAction; action++)
{
// Legacy first, always. The vendored decoder is what 787 of this client's bodies
// come out of, it is what phase 3 measured, and the UOP packages hold a different
// and mostly disjoint set (measured: of the 244 bodies they carry, 8 also have
// legacy art). So this is a fallback rather than a choice, and no body changes
// reader while a client sits still.
Sprite sprite = action < actions
? ResolveLegacy(Key(body, action), readers, body, action, direction)
: null;
if (sprite == null)
sprite = ResolveUop(Key(body, action), readers, body, action, direction);
if (sprite != null)
return sprite;
}
return null;
}
/// <summary> /// <summary>
/// ServUO's vendored <c>Animations</c> over <c>anim*.mul</c>, behind §4.5's validator. /// ServUO's vendored <c>Animations</c> over <c>anim*.mul</c>, behind §4.5's validator.
/// </summary> /// </summary>
private static Sprite ResolveLegacy(string key, Readers readers, int body, int direction) private static Sprite ResolveLegacy(string key, Readers readers, int body, int action, int direction)
{ {
int fileType, at; int fileType, at;
string reason; string reason;
if (!BridgeAssetValidator.ResolveAnimation(body, CatalogAction, direction, if (!BridgeAssetValidator.ResolveAnimation(body, action, direction,
out fileType, out at, out reason)) out fileType, out at, out reason))
return null; return null;
@@ -559,12 +610,12 @@ namespace Server.Custom.Bridge
try try
{ {
return Decode(key, body, direction, fileType); return Decode(key, body, action, direction, fileType);
} }
catch (Exception e) catch (Exception e)
{ {
Console.WriteLine("[Bridge] catalogue: body {0}: {1}: {2}", Console.WriteLine("[Bridge] catalogue: body {0} action {1}: {2}: {3}",
body, e.GetType().Name, e.Message); body, action, e.GetType().Name, e.Message);
return null; return null;
} }
} }
@@ -583,9 +634,9 @@ namespace Server.Custom.Bridge
/// hash of a name carrying the body id, and the payload repeats that id in its own /// hash of a name carrying the body id, and the payload repeats that id in its own
/// header for <see cref="BridgeUop.Group.TryOpen"/> to check. A miss is a miss. /// header for <see cref="BridgeUop.Group.TryOpen"/> to check. A miss is a miss.
/// </summary> /// </summary>
private static Sprite ResolveUop(string key, Readers readers, int body, int direction) private static Sprite ResolveUop(string key, Readers readers, int body, int action, int direction)
{ {
ulong hash = BridgeUop.HashOf(body, CatalogAction); ulong hash = BridgeUop.HashOf(body, action);
byte[] payload = null; byte[] payload = null;
string reason = null; string reason = null;
@@ -599,8 +650,8 @@ namespace Server.Custom.Bridge
if (!package.TryRead(hash, out payload, out reason)) if (!package.TryRead(hash, out payload, out reason))
{ {
Console.WriteLine("[Bridge] catalogue: body {0} in {1}: {2}", Console.WriteLine("[Bridge] catalogue: body {0} action {1} in {2}: {3}",
body, BridgeUop.PackageName(n), reason); body, action, BridgeUop.PackageName(n), reason);
return null; return null;
} }
@@ -614,7 +665,8 @@ namespace Server.Custom.Bridge
if (!BridgeUop.Group.TryOpen(payload, body, out group, out reason)) if (!BridgeUop.Group.TryOpen(payload, body, out group, out reason))
{ {
Console.WriteLine("[Bridge] catalogue: body {0} uop: {1}", body, reason); Console.WriteLine("[Bridge] catalogue: body {0} action {1} uop: {2}",
body, action, reason);
return null; return null;
} }
@@ -632,7 +684,8 @@ namespace Server.Custom.Bridge
// exactly the same condition — so it is absent, silently. Anything else is a // exactly the same condition — so it is absent, silently. Anything else is a
// record this reader refused, and that is worth a line. // record this reader refused, and that is worth a line.
if (!empty) if (!empty)
Console.WriteLine("[Bridge] catalogue: body {0} uop: {1}", body, reason); Console.WriteLine("[Bridge] catalogue: body {0} action {1} uop: {2}",
body, action, reason);
return null; return null;
} }
@@ -646,24 +699,25 @@ namespace Server.Custom.Bridge
{ {
Key = key, Key = key,
Body = body, Body = body,
Action = action,
Direction = direction, Direction = direction,
FileType = 0, FileType = 0,
Png = png, Png = png,
Width = pixels.Width, Width = pixels.Width,
Height = pixels.Height, Height = pixels.Height,
Sha256 = Hash(png), Sha256 = BridgeAssets.Sha256Hex(png),
Source = "uop" Source = "uop"
}; };
} }
private static Sprite Decode(string key, int body, int direction, int fileType) private static Sprite Decode(string key, int body, int action, int direction, int fileType)
{ {
int hue = 0; int hue = 0;
// `preserveHue: false` — the catalogue is the creature's own art, and a body-level hue // `preserveHue: false` — the catalogue is the creature's own art, and a body-level hue
// from Body.def belongs to a specific mob rather than to the species. §5's key scheme // from Body.def belongs to a specific mob rather than to the species. §5's key scheme
// is where a hued variant is expressed (`static/3922/h33`), not here. // is where a hued variant is expressed (`static/3922/h33`), not here.
Frame[] frames = Animations.GetAnimation(body, CatalogAction, direction, ref hue, false, true); Frame[] frames = Animations.GetAnimation(body, action, direction, ref hue, false, true);
if (frames == null || frames.Length == 0 || frames[0] == null) if (frames == null || frames.Length == 0 || frames[0] == null)
return null; return null;
@@ -673,7 +727,7 @@ namespace Server.Custom.Bridge
if (bitmap == null || bitmap.Width <= 0 || bitmap.Height <= 0) if (bitmap == null || bitmap.Width <= 0 || bitmap.Height <= 0)
return null; return null;
byte[] png = ToPng(bitmap); byte[] png = BridgeAssets.BitmapToPng(bitmap);
if (png == null) if (png == null)
return null; return null;
@@ -682,117 +736,17 @@ namespace Server.Custom.Bridge
{ {
Key = key, Key = key,
Body = body, Body = body,
Action = action,
Direction = direction, Direction = direction,
FileType = fileType, FileType = fileType,
Png = png, Png = png,
Width = bitmap.Width, Width = bitmap.Width,
Height = bitmap.Height, Height = bitmap.Height,
Sha256 = Hash(png), Sha256 = BridgeAssets.Sha256Hex(png),
Source = "legacy" Source = "legacy"
}; };
} }
/// <summary>
/// ARGB1555 to a PNG with a transparent background.
///
/// <c>Frame</c> writes 16-bit ARGB1555: a pixel the sprite does not cover is left as
/// zero and a pixel it does cover carries the top bit set. Saving that format straight
/// to PNG asks GDI+ to make the conversion, and what it does with a one-bit alpha
/// channel varies by platform — on Mono it is a different implementation entirely. A
/// sprite that came back with a black rectangle behind it would look fine in a test
/// that only checked the bytes decoded, and wrong on every page that showed it.
///
/// So the expansion is done here, explicitly: alpha bit clear becomes fully
/// transparent, and each 5-bit channel is widened to 8 bits by repeating its high bits
/// (<c>(c &lt;&lt; 3) | (c &gt;&gt; 2)</c>) rather than by shifting alone, which would
/// cap white at 248 and tint the whole catalogue.
/// </summary>
private static byte[] ToPng(Bitmap source)
{
var rect = new Rectangle(0, 0, source.Width, source.Height);
if (source.PixelFormat != PixelFormat.Format16bppArgb1555)
{
// Not what this library has ever produced. Save it rather than reinterpret it:
// guessing at an unknown layout is how a catalogue fills with confident nonsense.
using (var ms = new MemoryStream())
{
source.Save(ms, ImageFormat.Png);
return ms.ToArray();
}
}
using (var target = new Bitmap(source.Width, source.Height, PixelFormat.Format32bppArgb))
{
BitmapData src = source.LockBits(rect, ImageLockMode.ReadOnly, PixelFormat.Format16bppArgb1555);
BitmapData dst = null;
try
{
dst = target.LockBits(rect, ImageLockMode.WriteOnly, PixelFormat.Format32bppArgb);
var line = new short[source.Width];
var outLine = new int[source.Width];
for (int y = 0; y < source.Height; y++)
{
Marshal.Copy(new IntPtr(src.Scan0.ToInt64() + ((long)y * src.Stride)),
line, 0, source.Width);
for (int x = 0; x < source.Width; x++)
{
int p = line[x] & 0xFFFF;
if ((p & 0x8000) == 0)
{
outLine[x] = 0;
continue;
}
int r = (p >> 10) & 0x1F;
int g = (p >> 5) & 0x1F;
int b = p & 0x1F;
outLine[x] = unchecked((int)0xFF000000)
| (((r << 3) | (r >> 2)) << 16)
| (((g << 3) | (g >> 2)) << 8)
| ((b << 3) | (b >> 2));
}
Marshal.Copy(outLine, 0, new IntPtr(dst.Scan0.ToInt64() + ((long)y * dst.Stride)),
source.Width);
}
}
finally
{
if (dst != null)
target.UnlockBits(dst);
source.UnlockBits(src);
}
using (var ms = new MemoryStream())
{
target.Save(ms, ImageFormat.Png);
return ms.ToArray();
}
}
}
private static string Hash(byte[] bytes)
{
using (var sha = SHA256.Create())
{
byte[] digest = sha.ComputeHash(bytes);
var sb = new StringBuilder(digest.Length * 2);
foreach (byte b in digest)
sb.Append(b.ToString("x2", CultureInfo.InvariantCulture));
return sb.ToString();
}
}
// ── player bodies (§5.2) ───────────────────────────────────────────────────────────── // ── player bodies (§5.2) ─────────────────────────────────────────────────────────────
/// <summary> /// <summary>
@@ -884,20 +838,28 @@ namespace Server.Custom.Bridge
// ── keys, cursors and the source id ────────────────────────────────────────────────── // ── keys, cursors and the source id ──────────────────────────────────────────────────
private static string Key(int body) private static string Key(int body, int action)
{ {
return "body/" + body.ToString(CultureInfo.InvariantCulture) return "body/" + body.ToString(CultureInfo.InvariantCulture)
+ "/a" + CatalogAction.ToString(CultureInfo.InvariantCulture); + "/a" + action.ToString(CultureInfo.InvariantCulture);
} }
/// <summary> /// <summary>
/// `body/&lt;id&gt;/a0`, and nothing else in this phase. A deeper key /// `body/&lt;id&gt;/a&lt;n&gt;`, and nothing else in this phase. A deeper key
/// (`body/400/a2/f3`) is well-formed under §5 and simply not served yet, so it comes /// (`body/400/a2/f3`) is well-formed under §5 and simply not served, so it comes back
/// back `unsupported` rather than being silently read as its own first frame. /// `unsupported` rather than being silently read as its own first frame.
///
/// The action is parsed rather than required to be zero — 74 of this client's bodies
/// are catalogued at a different one (§11.2) — but a parsed action is not an accepted
/// one. <see cref="Render"/> serves a key only when it is the key the catalogue itself
/// chose for that body, which is what keeps §4.10's ceiling from being reachable
/// through a request: nothing the website can ask makes this decode an action the
/// catalogue did not already pick.
/// </summary> /// </summary>
private static bool TryParseKey(string key, out int body) private static bool TryParseKey(string key, out int body, out int action)
{ {
body = 0; body = 0;
action = -1;
if (key == null) if (key == null)
return false; return false;
@@ -913,7 +875,14 @@ namespace Server.Custom.Bridge
if (body < 1 || body > MaxBody) if (body < 1 || body > MaxBody)
return false; return false;
return parts[2] == "a" + CatalogAction.ToString(CultureInfo.InvariantCulture); if (parts[2].Length < 2 || parts[2][0] != 'a')
return false;
if (!Int32.TryParse(parts[2].Substring(1), NumberStyles.None,
CultureInfo.InvariantCulture, out action))
return false;
return action >= 0 && action < MaxAction;
} }
private static int ParseBodyCursor(string cursor) private static int ParseBodyCursor(string cursor)
@@ -997,34 +966,11 @@ namespace Server.Custom.Bridge
} }
} }
return Hash(Encoding.UTF8.GetBytes(sb.ToString())).Substring(0, 16); return BridgeAssets.Sha256Hex(Encoding.UTF8.GetBytes(sb.ToString())).Substring(0, 16);
} }
// ── shared plumbing ────────────────────────────────────────────────────────────────── // ── shared plumbing ──────────────────────────────────────────────────────────────────
/// <summary>
/// The two gates every request on this plane passes: a correlation id, and the
/// operator's consent. Both refuse rather than answer.
/// </summary>
private static bool Admit(Dictionary<string, object> o, string kind, out string reqId)
{
reqId = BridgeJson.GetString(o, "reqId");
if (reqId == null)
{
BridgeAssets.Fail(null, "BAD_REQUEST", kind + " requires a reqId");
return false;
}
if (!BridgeConfig.AssetsEnabled)
{
BridgeAssets.Fail(reqId, "DISABLED", "asset extraction is disabled on this shard");
return false;
}
return true;
}
/// <summary> /// <summary>
/// The five anim files' index and record readers — and, since phase 4, the five UOP /// The five anim files' index and record readers — and, since phase 4, the five UOP
/// packages beside them — opened for one reply and closed with it. Holding them across /// packages beside them — opened for one reply and closed with it. Holding them across

View File

@@ -124,6 +124,43 @@ namespace Server.Custom.Bridge
public static int AssetPlayerDirection { get; private set; } public static int AssetPlayerDirection { get; private set; }
public static int AssetCreatureDirection { get; private set; } public static int AssetCreatureDirection { get; private set; }
// ---- the tree plane (docs/link/v8.md §10, phase 7) ----
//
// Its OWN gate, and the third one on this link for the third kind of consent. The asset
// gate above is the operator agreeing that the website may read THEIR UO CLIENT -- art
// and animations and a string table that came from EA. This one is the operator agreeing
// that it may read THE SHARD'S OWN CONFIGURATION: the spawn files, the region and
// location definitions, the champion table, the decoration lists. Those are the
// operator's own work rather than a licensed client, and they are what the spawn atlas is
// built out of -- so a shard that declines to serve client art must still be able to
// publish where its creatures live. One switch could not have expressed both, and the
// atlas would have been the thing that silently disappeared.
//
// Reads only, and only the five labelled groups SPAWN_ATLAS.md already names. Nothing
// here joins a path the website sent: a request names a label this shard enumerated, or
// it is refused.
public static bool TreeEnabled { get; private set; }
// How much of a tree file one chunk carries, BEFORE compression (§10). The chunk is the
// thing that makes this transferable at all: a stock Spawns/trammel.xml is 4.03 MB and
// the sidecar discards any inbound line over 1 MiB, so the file as a single base64 row
// could never arrive -- it would time out and be re-requested forever, which is a failure
// with no error in it anywhere.
//
// Compression is what makes it cheap (a spawn file gzips ~18x, so a chunk is typically
// 40 KB on the wire) and the chunk is what makes it BOUNDED: gzip cannot be relied on to
// shrink anything, so the ceiling has to hold for input that does not compress at all.
// At 512 KiB a worst-case incompressible chunk is ~683 KiB of base64, which still fits
// the wire under AssetBatchBytes' deliberate factor of two.
public static int TreeChunkBytes { get; private set; }
// How many bytes of rendered item and land art the shard holds between requests (§11,
// phase 5). This is a convenience, not a store: the website keeps every picture it fetches
// and does not ask twice, so what this actually buys is the second page of a batch, a
// retry after a 425, and the same item appearing in two rows of one page. Sized so a
// full 512 KB batch and the one before it both fit with room over.
public static int AssetArtCacheBytes { get; private set; }
public static int LeaseMaxDurationSec { get; private set; } public static int LeaseMaxDurationSec { get; private set; }
public static int LeaseGraceSec { get; private set; } public static int LeaseGraceSec { get; private set; }
@@ -222,6 +259,29 @@ namespace Server.Custom.Bridge
AssetPlayerDirection = Clamp(Config.Get("Bridge.AssetPlayerDirection", 0), 0, 4); AssetPlayerDirection = Clamp(Config.Get("Bridge.AssetPlayerDirection", 0), 0, 4);
AssetCreatureDirection = Clamp(Config.Get("Bridge.AssetCreatureDirection", 1), 0, 4); AssetCreatureDirection = Clamp(Config.Get("Bridge.AssetCreatureDirection", 1), 0, 4);
// The floor is one batch: a cache that cannot hold the page being built evicts rows
// while they are still being written, which is a cache that costs and never pays. The
// ceiling is a game server's memory, and 64 MB of PNG is already ~34,000 sprites --
// most of this client's art, held for a working set that is measured in hundreds.
AssetArtCacheBytes = Config.Get("Bridge.AssetArtCacheBytes", 16 * 1024 * 1024);
if (AssetArtCacheBytes < AssetBatchBytes)
AssetArtCacheBytes = AssetBatchBytes;
if (AssetArtCacheBytes > 64 * 1024 * 1024)
AssetArtCacheBytes = 64 * 1024 * 1024;
TreeEnabled = Config.Get("Bridge.TreeEnabled", true);
// Floor and ceiling both matter. Below 64 KiB a stock tree is thousands of chunks and
// the per-row overhead starts to dominate the payload; above 512 KiB an incompressible
// chunk stops fitting inside the sidecar's inbound line cap, which is the one bound
// this number exists to respect. Kept equal to AssetBatchBytes' own ceiling so the two
// budgets cannot drift into disagreeing about the same wire.
TreeChunkBytes = Config.Get("Bridge.TreeChunkBytes", 512 * 1024);
if (TreeChunkBytes < 64 * 1024)
TreeChunkBytes = 64 * 1024;
if (TreeChunkBytes > 512 * 1024)
TreeChunkBytes = 512 * 1024;
StatSweepSeconds = Config.Get("Bridge.StatSweepSeconds", 30); StatSweepSeconds = Config.Get("Bridge.StatSweepSeconds", 30);
DecaySweepSeconds = Config.Get("Bridge.DecaySweepSeconds", 60); DecaySweepSeconds = Config.Get("Bridge.DecaySweepSeconds", 60);
EconomySweepSeconds = Config.Get("Bridge.EconomySweepSeconds", 300); EconomySweepSeconds = Config.Get("Bridge.EconomySweepSeconds", 300);

View File

@@ -0,0 +1,775 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.IO.Compression;
using System.Text;
namespace Server.Custom.Bridge
{
/// <summary>
/// **The shard's own configuration, over the bridge** (docs/link/v8.md §10 — protocol 8,
/// phase 7).
///
/// Everything else on the asset plane reads the operator's UO CLIENT. This family reads
/// the shard's own files: the spawn tables, the region and location definitions, the
/// champion list and the decoration lists. The website parses those into its spawn atlas —
/// where every creature lives, which regions exist, what this shard calls scenery — and
/// until protocol 8 it did so by **reading the ServUO tree off a shared filesystem**:
/// same host, a bind mount, or a shared volume.
///
/// That was the one place the platform's own rule was broken, and broken by the component
/// that faces the internet. This closes it. The parsers do not move — `spawnAtlasParse.js`
/// is pure, fs-free and covered by CI without a ServUO tree anywhere near it, and every
/// quirk it handles stays exactly where it is. The shard sends bytes; the website still
/// decides what they mean.
///
/// ── What phase 7 measured, and the shape it forced ────────────────────────────────
///
/// §10 said "the shard serves `tree/&lt;label&gt;` → bytes". Measured against a stock 57.4
/// tree, it cannot: `Spawns/trammel.xml` is **4.03 MB**, the sidecar discards any inbound
/// line over **1 MiB** (`shard.rs` `MAX_INBOUND_LINE_BYTES`), and that file as a single
/// base64 row is 5.4 MiB. It would never arrive — the reply would be discarded, the
/// request would time out, and the import would retry forever with no error anywhere in
/// it. Two files on a *stock* tree are in that state; a shard with hand-built spawn tables
/// has more.
///
/// So a file crosses as **chunks, each gzipped**:
///
/// <code>
/// tree/Spawns/trammel.xml the manifest row — size, hash, chunk count
/// tree/Spawns/trammel.xml/c0 the first 512 KiB of it, gzipped
/// tree/Spawns/trammel.xml/c1 the next
/// </code>
///
/// which is §5's depth scheme at work a second time, exactly as `body/400/a0/f0` is —
/// and, as there, nothing about it needed a protocol change.
///
/// **The chunk is the bound and the compression is the saving**, and it matters which is
/// which. Compression is what makes this cheap: the stock tree is 11.34 MB and gzips to
/// 927 KB, so the whole atlas source arrives in about three pages instead of thirty-one.
/// But nothing guarantees that an operator's files compress at all, so the ceiling has to
/// hold when they do not — and it does, because a 512 KiB chunk that refuses to compress
/// is still only ~683 KiB of base64, inside the wire cap that
/// <see cref="BridgeConfig.AssetBatchBytes"/>' deliberate factor of two leaves room for.
/// A design that leaned on the ratio would work on every tree anyone tested and fail on
/// the first one nobody did.
///
/// ── Two rules that are not negotiable here ────────────────────────────────────────
///
/// **1. The label set is this shard's, never the caller's.** This is the only family on
/// this link whose keys look like paths, and the website is the internet-facing component.
/// So nothing here joins a path that arrived on the wire: a fetch resolves its label
/// against the set <see cref="Enumerate"/> itself produced, and a label that is not in it
/// is refused — before any file is opened, and whatever it spells. The five groups are
/// fixed in code, the extensions are fixed in code, and the resolved path is checked to be
/// under the tree root even after all of that.
///
/// **2. A row re-declares its own address.** Each chunk carries its label, its index, its
/// byte offset and the hash of its own (uncompressed) bytes, and the manifest carries the
/// hash of the whole file. That is the §4.10 lesson on a fourth axis: a reassembly that
/// silently put chunk 3 where chunk 4 belongs would produce a file that parses — XML is
/// forgiving about what it skips — and a spawn atlas subtly missing a facet. Per-chunk
/// hashes make it a named error instead.
/// </summary>
public static class BridgeTree
{
/// <summary>The §5 key family this serves.</summary>
private const string Family = "tree";
/// <summary>
/// The five labelled groups `spawnAtlasSource.js` reads, and nothing else.
///
/// Fixed in code rather than configured, because a configurable list is a way for the
/// website to ask for a file this shard never meant to publish. An operator who wants
/// a different tree served wants a different feature.
/// </summary>
private static readonly string[] SingleFiles =
{
"Data/Regions.xml",
"Config/ChampionSpawns.xml"
};
private const string LocationsDir = "Data/Locations";
private const string SpawnsDir = "Spawns";
private const string DecorationDir = "Data/Decoration";
public static void Initialize()
{
if (!BridgeConfig.Enabled)
return;
// Its own consent, not the asset plane's (§10, phase 7). An operator who declines to
// serve their UO client still gets a spawn atlas, because these are their own files.
BridgeAssets.RegisterFamily(Family, ReplyFetch, ReplyManifest,
() => BridgeConfig.TreeEnabled,
"the shard's configuration tree is not served (Bridge.TreeEnabled is off)");
}
// ── the file set ─────────────────────────────────────────────────────────────────────
private sealed class TreeFile
{
public string Label;
public string Path;
public long Bytes;
public long MTime;
}
/// <summary>
/// Every atlas source file this shard has, tree-relative and forward-slashed.
///
/// The labels are `spawnAtlasSource.js`'s own, character for character, because they
/// are what the website keys its stored fingerprint on: the same tree read here and
/// read there has to produce the same label or every import looks like a change.
/// Forward slashes for the same reason — a Windows shard and a Linux one must agree.
/// </summary>
private static List<TreeFile> Enumerate()
{
string root = Core.BaseDirectory;
var files = new List<TreeFile>();
foreach (string label in SingleFiles)
Add(files, root, label);
foreach (string label in ListByExtension(root, LocationsDir, ".xml"))
Add(files, root, label);
foreach (string label in ListByExtension(root, SpawnsDir, ".xml"))
Add(files, root, label);
foreach (string label in ListTree(root, DecorationDir, ".cfg"))
Add(files, root, label);
return files;
}
private static void Add(List<TreeFile> files, string root, string label)
{
string path = Resolve(root, label);
if (path == null)
return;
try
{
var info = new FileInfo(path);
if (!info.Exists)
return;
files.Add(new TreeFile
{
Label = label,
Path = path,
Bytes = info.Length,
MTime = ToUnixMs(info.LastWriteTimeUtc)
});
}
catch (Exception e)
{
// A file the shard cannot stat is a file it cannot serve. Say so once, here,
// rather than as a refused row on every import pass forever.
Console.WriteLine("[Bridge] tree: cannot read {0}: {1}", label, e.Message);
}
}
/// <summary>One directory's files with the given extension, sorted, as labels.</summary>
private static List<string> ListByExtension(string root, string dir, string extension)
{
var labels = new List<string>();
string full = Path.Combine(root, dir.Replace('/', Path.DirectorySeparatorChar));
try
{
if (!Directory.Exists(full))
return labels;
foreach (string path in Directory.GetFiles(full))
{
string name = Path.GetFileName(path);
if (name.EndsWith(extension, StringComparison.OrdinalIgnoreCase))
labels.Add(dir + "/" + name);
}
}
catch (Exception e)
{
Console.WriteLine("[Bridge] tree: cannot list {0}: {1}", dir, e.Message);
}
labels.Sort(StringComparer.Ordinal);
return labels;
}
/// <summary>
/// One directory tree's files with the given extension, recursively.
///
/// Recursive because `Data/Decoration` nests two deep in places (`Magincia/Trammel`,
/// `Stygian Abyss/Ter Mur`, `Old/Britannia`), and the website's own reader says why
/// that matters: a flat read indexes a third of what the shard has, and the failure is
/// an authoring dropdown quietly missing whole expansions rather than an error anyone
/// would notice.
/// </summary>
private static List<string> ListTree(string root, string dir, string extension)
{
var labels = new List<string>();
string full = Path.Combine(root, dir.Replace('/', Path.DirectorySeparatorChar));
try
{
if (!Directory.Exists(full))
return labels;
foreach (string path in Directory.GetFiles(full, "*", SearchOption.AllDirectories))
{
if (!path.EndsWith(extension, StringComparison.OrdinalIgnoreCase))
continue;
string rel = path.Substring(full.Length).Replace('\\', '/').TrimStart('/');
if (rel.Length > 0)
labels.Add(dir + "/" + rel);
}
}
catch (Exception e)
{
Console.WriteLine("[Bridge] tree: cannot walk {0}: {1}", dir, e.Message);
}
labels.Sort(StringComparer.Ordinal);
return labels;
}
/// <summary>
/// A label to a path on this host, or null if it is not one this shard serves.
///
/// Rule 1 of the class doc lives here. The label has already been matched against the
/// enumerated set by the time a fetch calls this, and this still refuses anything with
/// a traversal segment, a drive or a root in it, and still checks that what
/// <c>Path.GetFullPath</c> produced is under the tree root. Three checks for one rule
/// because the cost of being wrong once is the website reading an arbitrary file off a
/// game server's disk.
/// </summary>
private static string Resolve(string root, string label)
{
if (String.IsNullOrEmpty(label) || label.IndexOf('\\') >= 0)
return null;
string[] segments = label.Split('/');
foreach (string segment in segments)
{
if (segment.Length == 0 || segment == "." || segment == "..")
return null;
}
if (Path.IsPathRooted(label))
return null;
try
{
string rootFull = Path.GetFullPath(root);
string full = Path.GetFullPath(Path.Combine(rootFull,
label.Replace('/', Path.DirectorySeparatorChar)));
if (!rootFull.EndsWith(Path.DirectorySeparatorChar.ToString(CultureInfo.InvariantCulture),
StringComparison.Ordinal))
{
rootFull += Path.DirectorySeparatorChar;
}
return full.StartsWith(rootFull, StringComparison.OrdinalIgnoreCase) ? full : null;
}
catch
{
return null;
}
}
// ── the fingerprint ──────────────────────────────────────────────────────────────────
/// <summary>
/// What the whole tree currently is, in sixteen hex characters.
///
/// The same job <c>BridgeCatalog.SourceId</c> does for client files, and the same
/// reason: it goes on every page of a walk, and a page whose id differs from the
/// first's means the operator edited a spawn file while it was being read. Half of
/// what arrived then describes a tree that no longer exists and nothing later can tell
/// which half, so the website refuses the import outright rather than stitching one.
///
/// Built from (label, size, mtime) rather than from content hashes, because it is
/// computed on every page and hashing the tree's contents each time would spend a
/// tenth of a second per page to answer a question (size, mtime) answers for free.
/// The CONTENT hashes are still sent — once, per file, on the manifest — which is
/// where the website's own drift gate reads them from.
/// </summary>
private static string FingerprintOf(List<TreeFile> files)
{
var sb = new StringBuilder(256);
sb.Append(files.Count);
foreach (TreeFile file in files)
{
sb.Append('|').Append(file.Label)
.Append(':').Append(file.Bytes.ToString(CultureInfo.InvariantCulture))
.Append(':').Append(file.MTime.ToString(CultureInfo.InvariantCulture));
}
return BridgeAssets.Sha256Hex(Encoding.UTF8.GetBytes(sb.ToString())).Substring(0, 16);
}
// ── assets.manifest, for this family ─────────────────────────────────────────────────
/// <summary>
/// Worker thread. Every file this shard would serve, with its size, its content hash
/// and how many chunks it takes — and no bytes.
///
/// That separation is what makes the normal case free. The website stores these
/// hashes; on the next import it asks for this list again, compares, and fetches
/// nothing at all when nothing moved — which on a shard whose maps are not being
/// edited is every import.
///
/// A stock tree is 141 rows and fits in one page comfortably. It pages anyway, by the
/// same envelope as every other family, because the day a shard has three thousand
/// decoration files is not the day to discover this was the one walk that could not
/// end.
/// </summary>
private static void ReplyManifest(string reqId, string cursor)
{
List<TreeFile> files = Enumerate();
string fingerprint = FingerprintOf(files);
int from = ParseCursor(cursor);
if (from < 0 || from > files.Count)
from = 0;
var sb = BridgeJson.Begin("assets.manifest.ok");
sb.Str("reqId", reqId)
.Str("family", Family)
.Str("catalog", fingerprint)
.Num("chunkBytes", BridgeConfig.TreeChunkBytes)
.Num("total", files.Count)
.Num("from", from);
var page = new BridgeAssets.PageBuilder(sb, "rows", BridgeConfig.AssetBatchBytes);
int i = from;
for (; i < files.Count; i++)
{
TreeFile file = files[i];
string hash = HashFile(file.Path);
var item = new StringBuilder(256);
item.Append("{\"key\":");
BridgeJson.Text(item, Family + "/" + file.Label);
item.Append(",\"label\":");
BridgeJson.Text(item, file.Label);
item.Append(",\"bytes\":").Append(file.Bytes.ToString(CultureInfo.InvariantCulture));
item.Append(",\"mtime\":").Append(file.MTime.ToString(CultureInfo.InvariantCulture));
item.Append(",\"chunks\":").Append(
ChunkCount(file.Bytes).ToString(CultureInfo.InvariantCulture));
item.Append(",\"sha256\":");
BridgeJson.Text(item, hash);
item.Append('}');
if (!page.TryAdd(item.ToString(), "t:" + (i + 1).ToString(CultureInfo.InvariantCulture)))
break;
}
page.Close();
sb.Num("sent", page.Count);
BridgeLink.Emit(sb.End());
}
/// <summary>
/// How many chunks a file of this size takes.
///
/// **An empty file is one chunk, not none.** `Data/Locations` can legitimately hold an
/// empty file, and zero chunks would make it a manifest row the website could never
/// fetch: it would wait for content that has no address, and report the import
/// incomplete forever.
/// </summary>
private static int ChunkCount(long bytes)
{
long chunk = BridgeConfig.TreeChunkBytes;
long count = (bytes + chunk - 1) / chunk;
return count < 1 ? 1 : (int)count;
}
// ── assets.fetch, for this family ────────────────────────────────────────────────────
/// <summary>
/// Worker thread. The bytes for an explicit list of chunk keys.
///
/// Chunks are read with a seek rather than by holding the file, so the memory this
/// costs a running game server is one chunk regardless of how large an operator's
/// spawn tables are. A 4 MB file served eight times over is eight seeks and eight
/// 512 KiB reads — cheaper than caching it would be, and with no cache to invalidate
/// when the operator edits it mid-pass.
/// </summary>
private static void ReplyFetch(string reqId, List<string> keys, string expected, string cursor)
{
List<TreeFile> files = Enumerate();
string fingerprint = FingerprintOf(files);
// Shared with every other family on this plane, because an absent fingerprint and an
// empty one have to mean the same thing here and there — see
// `BridgeAssets.CatalogMismatch` for what treating them differently costs.
if (BridgeAssets.CatalogMismatch(expected, fingerprint))
{
// The tree moved between the manifest and this fetch. The same refusal the
// catalogue makes for a patched client, and for the same reason: these keys were
// chosen against a listing that no longer describes what is on disk.
BridgeAssets.Fail(reqId, "UNREADABLE",
"the shard's configuration tree changed since that manifest was read (catalog "
+ expected + " is now " + fingerprint + "); start the import again");
return;
}
var byLabel = new Dictionary<string, TreeFile>(StringComparer.Ordinal);
foreach (TreeFile file in files)
byLabel[file.Label] = file;
int from = ParseCursor(cursor);
if (from < 0 || from > keys.Count)
from = 0;
var sb = BridgeJson.Begin("assets.fetch.ok");
sb.Str("reqId", reqId)
.Str("family", Family)
.Str("catalog", fingerprint)
.Num("chunkBytes", BridgeConfig.TreeChunkBytes)
.Num("asked", keys.Count)
.Num("from", from);
var page = new BridgeAssets.PageBuilder(sb, "rows", BridgeConfig.AssetBatchBytes);
int i = from;
for (; i < keys.Count; i++)
{
string item = Render(byLabel, keys[i]);
if (!page.TryAdd(item, "t:" + (i + 1).ToString(CultureInfo.InvariantCulture)))
break;
}
page.Close();
sb.Num("sent", page.Count);
BridgeLink.Emit(sb.End());
}
/// <summary>
/// One key to one row.
///
/// A key this shard cannot serve is a row rather than a failed request, exactly as in
/// every other family, and `status` keeps the two kinds apart: `absent` is a file this
/// shard does not have (a tree with no `ChampionSpawns.xml` is a normal tree), and
/// `unsupported` is a key shape this family does not serve — which is a website bug,
/// and is counted separately so it cannot hide inside the expected gaps.
/// </summary>
private static string Render(Dictionary<string, TreeFile> byLabel, string key)
{
string label;
int chunk;
if (!ParseKey(key, out label, out chunk))
return Refusal(key, "unsupported", "not a tree chunk key (tree/<label>/c<n>)");
TreeFile file;
if (!byLabel.TryGetValue(label, out file))
{
// Rule 1: the label has to be one THIS shard enumerated. Anything else is refused
// here, before a path is built out of it, whatever it spells.
return Refusal(key, "absent", "this shard does not serve that file");
}
int chunks = ChunkCount(file.Bytes);
if (chunk < 0 || chunk >= chunks)
{
return Refusal(key, "unsupported",
"chunk " + chunk.ToString(CultureInfo.InvariantCulture) + " of "
+ chunks.ToString(CultureInfo.InvariantCulture));
}
long offset = (long)chunk * BridgeConfig.TreeChunkBytes;
byte[] raw;
try
{
raw = ReadChunk(file.Path, offset, BridgeConfig.TreeChunkBytes);
}
catch (Exception e)
{
Console.WriteLine("[Bridge] tree: cannot read {0} chunk {1}: {2}", label, chunk, e.Message);
return Refusal(key, "absent", e.GetType().Name);
}
byte[] packed;
try
{
packed = Gzip(raw);
}
catch (Exception e)
{
Console.WriteLine("[Bridge] tree: cannot compress {0} chunk {1}: {2}", label, chunk, e.Message);
return Refusal(key, "absent", e.GetType().Name);
}
var item = new StringBuilder(packed.Length * 2);
item.Append("{\"key\":");
BridgeJson.Text(item, key);
item.Append(",\"status\":\"ok\",\"label\":");
BridgeJson.Text(item, label);
item.Append(",\"chunk\":").Append(chunk.ToString(CultureInfo.InvariantCulture));
item.Append(",\"chunks\":").Append(chunks.ToString(CultureInfo.InvariantCulture));
item.Append(",\"offset\":").Append(offset.ToString(CultureInfo.InvariantCulture));
item.Append(",\"bytes\":").Append(raw.Length.ToString(CultureInfo.InvariantCulture));
item.Append(",\"sha256\":");
BridgeJson.Text(item, BridgeAssets.Sha256Hex(raw));
item.Append(",\"gzip\":");
BridgeJson.Text(item, Convert.ToBase64String(packed));
item.Append('}');
return item.ToString();
}
private static string Refusal(string key, string status, string reason)
{
var item = new StringBuilder(128);
item.Append("{\"key\":");
BridgeJson.Text(item, key);
item.Append(",\"status\":");
BridgeJson.Text(item, status);
item.Append(",\"reason\":");
BridgeJson.Text(item, reason);
item.Append('}');
return item.ToString();
}
/// <summary>
/// `tree/&lt;label&gt;/c&lt;n&gt;` into its label and chunk index.
///
/// The label itself contains slashes, so the chunk segment is taken off the END rather
/// than by counting segments from the front. That is unambiguous here and not by
/// luck: every label this family serves ends in `.xml` or `.cfg`, so no label's last
/// segment can be spelled `c` followed by digits.
/// </summary>
private static bool ParseKey(string key, out string label, out int chunk)
{
label = null;
chunk = -1;
if (String.IsNullOrEmpty(key))
return false;
string prefix = Family + "/";
if (!key.StartsWith(prefix, StringComparison.Ordinal))
return false;
int slash = key.LastIndexOf('/');
if (slash <= prefix.Length - 1)
return false;
string last = key.Substring(slash + 1);
if (last.Length < 2 || last[0] != 'c')
return false;
for (int i = 1; i < last.Length; i++)
{
if (last[i] < '0' || last[i] > '9')
return false;
}
if (!Int32.TryParse(last.Substring(1), NumberStyles.None, CultureInfo.InvariantCulture, out chunk))
return false;
label = key.Substring(prefix.Length, slash - prefix.Length);
return label.Length > 0;
}
private static int ParseCursor(string cursor)
{
if (String.IsNullOrEmpty(cursor) || !cursor.StartsWith("t:", StringComparison.Ordinal))
return 0;
int value;
return Int32.TryParse(cursor.Substring(2), NumberStyles.None,
CultureInfo.InvariantCulture, out value) ? value : 0;
}
// ── bytes ────────────────────────────────────────────────────────────────────────────
private static byte[] ReadChunk(string path, long offset, int length)
{
using (var stream = new FileStream(path, FileMode.Open, FileAccess.Read,
FileShare.ReadWrite, 1 << 16))
{
long remaining = stream.Length - offset;
if (remaining < 0)
remaining = 0;
if (remaining > length)
remaining = length;
var buffer = new byte[remaining];
stream.Seek(offset, SeekOrigin.Begin);
int filled = 0;
while (filled < buffer.Length)
{
int read = stream.Read(buffer, filled, buffer.Length - filled);
// A short read is not the end of the file here — the length was taken from the
// stream itself. Stopping on one would hand back a chunk whose declared length
// and real length disagree, which the website would only see as a hash
// mismatch on a file it cannot name a cause for.
if (read <= 0)
break;
filled += read;
}
if (filled == buffer.Length)
return buffer;
var exact = new byte[filled];
Buffer.BlockCopy(buffer, 0, exact, 0, filled);
return exact;
}
}
/// <summary>
/// A complete gzip member for exactly one empty chunk.
///
/// **`GZipStream` writes NOTHING for zero bytes of input**, on .NET Framework and on
/// Mono: the gzip header is emitted lazily on the first write, so a stream that is
/// opened and closed without one produces a zero-length buffer rather than the 20-byte
/// empty member. That is not a valid gzip stream, and the reader at the other end says
/// so — `zlib: unexpected end of file`.
///
/// It is not a hypothetical: **stock ServUO 57.4 ships two empty decoration files**
/// (`Felucca/ambitious solen queen quest.cfg` and
/// `Tokuno/terrible hatchlings quest.cfg`), so every import off an untouched tree hit
/// it. Worth knowing how it was found, because it says something about probes: an
/// offline harness reassembled all 141 files and reported success, since .NET's own
/// decompressor treats an empty stream as empty data and the chunk's declared length
/// (0) and hash (of nothing) both agreed with that. Only the live walk, through a
/// reader on a different runtime, disagreed.
///
/// The alternative — letting an empty chunk carry an empty payload and teaching the
/// reader to expect it — was rejected: it puts a special case on the wire, where every
/// future reader has to know it, instead of in the one place that builds the bytes.
/// Header (magic, deflate, no flags, no mtime, no XFL, unknown OS), one empty stored
/// block, then CRC32 and ISIZE of nothing.
/// </summary>
private static readonly byte[] EmptyGzip =
{
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xff,
0x03, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00
};
private static byte[] Gzip(byte[] raw)
{
if (raw.Length == 0)
return EmptyGzip;
using (var ms = new MemoryStream())
{
using (var gz = new GZipStream(ms, CompressionMode.Compress, true))
gz.Write(raw, 0, raw.Length);
return ms.ToArray();
}
}
/// <summary>
/// The content hash of one file, streamed.
///
/// Streamed rather than <c>File.ReadAllBytes</c> because this runs once per file per
/// manifest, and a stock tree's spawn files are 10 MB between them: reading them whole
/// would put that much through a game server's large object heap to produce 141 short
/// strings.
/// </summary>
private static string HashFile(string path)
{
try
{
using (var sha = System.Security.Cryptography.SHA256.Create())
using (var stream = new FileStream(path, FileMode.Open, FileAccess.Read,
FileShare.ReadWrite, 1 << 16))
{
var buffer = new byte[1 << 16];
int read;
while ((read = stream.Read(buffer, 0, buffer.Length)) > 0)
sha.TransformBlock(buffer, 0, read, null, 0);
sha.TransformFinalBlock(buffer, 0, 0);
var sb = new StringBuilder(64);
foreach (byte b in sha.Hash)
sb.Append(b.ToString("x2", CultureInfo.InvariantCulture));
return sb.ToString();
}
}
catch (Exception e)
{
Console.WriteLine("[Bridge] tree: cannot hash {0}: {1}", path, e.Message);
return null;
}
}
private static long ToUnixMs(DateTime utc)
{
return (long)(utc - new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)).TotalMilliseconds;
}
/// <summary>For `[Bridge] status`, the same one-line shape every other family reports.</summary>
public static string Status()
{
if (!BridgeConfig.TreeEnabled)
return "tree(disabled)";
List<TreeFile> files = Enumerate();
long bytes = 0;
foreach (TreeFile file in files)
bytes += file.Bytes;
return String.Format("tree(files={0} bytes={1} catalog={2})",
files.Count, bytes, FingerprintOf(files));
}
}
}