docs/modules/uo/SCHEMA.md

module-uo — the tables it owns

The 27 tables module-uo creates and owns, and the reasoning behind their shapes. 26 shard_* plus uo_link_config, all created by the idempotent server/db/schema.sql fragment core replays after its own schema on every boot, and all dropped by server/db/purge.sql when an operator purges the module's data.

This page moved out of BACKEND_DESIGN.md §3 when Phase 4 closed (MODULE_SYSTEM.md §2.7.2): core's schema reference describes core's tables, and these are not core's. The text is unchanged — where it says "the shard", it means the Ultima Online shard this module fronts. Core's own module bookkeeping table, installed_modules, stays in BACKEND_DESIGN.md where it belongs.

The shard_ and uo_link_ prefixes are grandfathered (MODULE_API.md §6.5). A new module's tables are prefixed with its own id; these predate the rule, and they are read by the shipped Android app, so renaming them would be a data migration plus a client break.

Not every table has a section here. The pre-3.0 ingest tables (shard_events, shard_online, shard_economy, shard_houses, shard_champs, shard_guilds, shard_governors, shard_governor_terms, shard_presence, shard_pages, shard_account_links, uo_link_config) are described where their wire frames are — link/PLAN.md §5 and link/INTEGRATION.md. What follows is everything that carried a design note worth keeping.

---

shard_ruleset — the shard's published ruleset (Protocol 3.0)

Singleton row (id = 1, CHECK-constrained) holding the latest world.ruleset frame: rev, expansion, payload JSON (the whole frame), t, updated_at. The shard re-emits the complete ruleset on every sidecar connect, so this is an overwrite, not an append — and the kind is deliberately not in LOGGED_KINDS, since logging it would put a duplicate row in shard_events on every reconnect while server.hello already marks each of those.

The frame is stored whole rather than normalized into columns: it is a flat description of server config that is read as one page, so splitting it up would mean a schema change every time the shard grows a new block. rev (the shard's FNV-1a of the body) and expansion are hoisted only because they are cheap to display — the same payload-plus-hoisted-columns shape shard_champs uses.

No row means the shard has never published one (an older plugin, or Bridge.RulesetEnabled=false), served as null rather than {}: "not published yet" and "published, everything off" are different answers and the page renders them differently.

shard_points_boards — points / loyalty leaderboards (Protocol 3.0)

One row per point system, keyed by the shard's own PointsType name (QueensLoyalty, CleanUpBritannia, …). The shard carries ~25 of these, each a standing players build over months. Columns: system (PK), name, name_cliloc, max_points, players, show_on_gump, payload JSON (the whole points.board frame), t, updated_at.

The top-N list stays inside payload rather than being normalized into a shard_points_entries table. It is a fixed-size list (10 by default) that is only ever read whole — exactly like shard_governors.candidates — so normalizing buys nothing until something needs a per-character reverse lookup, and a character's own standings already ride inside char.profile instead.

Board state, not events: points.board is not in LOGGED_KINDS, for the same reason guild.update isn't. The shard emits a frame every time anyone's score moves a top ten, so logging would grow shard_events without bound for something whose only interesting value is its latest version. There is also no delete path — the shard's set of systems is fixed at startup, so there is no points.remove to mirror.

Two values carry non-obvious meanings, both set by the plugin and both documented in link/INTEGRATION.md §4:

• max_points = 0 means uncapped, and on a real shard that is the common case (ServUO's uncapped idiom is double.MaxValue, which the plugin normalises to 0). Anything rendering points / max_points must special-case it.
• name is usually NULL, with name_cliloc set instead — most systems name themselves with a cliloc rather than a literal. Listing therefore orders by COALESCE(name, system), so boards awaiting cliloc resolution sort by their own key rather than clumping together under NULL.

shard_vendors / shard_vendor_items — the player-vendor marketplace (Protocol 3.0)

The shard-wide shop index, fed by vendor.listing / vendor.listing.remove. One row per player vendor and one per priced listing. Full operator detail in MARKETPLACE.md; the design is docs/link/v3.md §8.

Table	Shape
shard_vendors	serial (PK), shop_name, owner_serial, owner_name, map/x/y/z, region, house, item_count, item_total, truncated, t, updated_at. Indexes on owner, map, region and updated_at.
shard_vendor_items	id (PK), vendor_serial, serial, item_id, hue, amount, price, name, cliloc, display_name, child. Indexes on vendor_serial, price, item_id, display_name, and (display_name, price).

Ingest is per-vendor and authoritative: the frame is the whole shop, so ingest is delete-then-insert of that vendor's listings inside one transaction. All-or-nothing matters specifically because the two writes are "the shop" and "what is in it" — a failure between them leaves a shop advertising an inventory it no longer has, which is visibly wrong and indistinguishable from a genuinely empty shop. No foreign keys, consistent with every other shard_* table.

There is deliberately no payload column, unlike shard_points_boards directly above. The board's top-N is a fixed-size list read whole, so it lives in JSON; here the items are the searchable rows, so they are normalized and nothing is left worth duplicating. The sidecar keeps the whole blob — outage resilience is its job, search is ours.

Market state, not events: neither kind is in LOGGED_KINDS, and this is the strongest case of the three v3 kinds. One frame carries up to 250 listings and the sweep re-emits a shop on any price change, so logging would turn shard_events into a price history nobody reads.

Two columns carry non-obvious meanings:

• item_count vs item_total. item_count is what the frame published; item_total is what the shop actually holds. They differ when truncated — the shard caps listings per frame (Bridge.MarketMaxListings, 250 by default), and a commodity reseller with thousands of stacks genuinely exceeds it. Any UI must show both or it presents a partial shop as complete.
• display_name is denormalized at ingest, resolved from the item's literal name (preferred — a player set it, so it is more specific) else its cliloc against shard_clilocs. Resolving at query time would put the cliloc table on the hot path and make search-by-name impossible. Because the shard's diff sweep will not re-send an unchanged shop just because the site learned what its items are called, a cliloc import triggers a bulk re-resolution of this column (after a boot import and after an admin import; ~50 ms per thousand rows, never throws).

updated_at is written explicitly on every upsert rather than left to ON UPDATE CURRENT_TIMESTAMP, which MariaDB does not fire when every column is written back unchanged. A shop re-published identically is still freshly confirmed, and without this the staleness banner would age a perfectly current shop forever.

shard_feature_visibility — per-feature audience config (Protocol 3.0)

One row per shard feature: feature (PK), enabled, audience (a rung on the ladder in API.md), stream (whether the feature's kinds fan out over SSE at all), field_rules JSON ({field: rung} for the sensitive fields only), updated_by, updated_at.

An absent row means "use the compiled default", and the compiled defaults reproduce pre-3.0 behavior — so an empty table is a no-op and there is nothing to seed. Stored rows are merged over the defaults on read, which is also where the invariants are re-applied: a row naming an unknown feature is ignored (a stale row must not resurrect a removed feature), an invalid rung falls back to the default rather than failing open, and a rule touching a locked field (acct / webId) is discarded. See API.md.

shard_spawn_* / shard_regions / shard_landmarks / shard_champion_spawns / shard_atlas_meta — the spawn atlas (Protocol 3.0)

Static shard content, not live shard state. Nothing here comes from the sidecar: the atlas is derived from the shard's own ServUO tree, re-read on every server boot and hash-gated so an unchanged tree costs one read pass and no write. Nothing is precomputed and committed — a shard's maps change over its life, and a snapshot in the repo would silently drift from the world players actually see. These tables stay populated whether the shard is up or not. Full operator detail in SPAWN_ATLAS.md; the design is docs/link/v3.md §6.

No facet name appears anywhere in the code. A shard may add facets, replace them, or rename them when its maps are updated; the facet set is discovered from the tree, and the loose spellings in Data/Locations are matched against it rather than looked up in a table.

Table	Key columns
shard_spawn_creatures	slug PK, name, total, points, facets JSON, art NULL
shard_spawn_points	id PK, facet, name, x, y, width, height, spawn_range, max_count, min_delay, max_delay, tod_start/end/mode, region, landmark, label
shard_spawn_point_types	(point_id, slug) PK, max_count
shard_regions	facet, name, type, priority, parent, rects JSON
shard_landmarks	facet, name, grp, x, y, z
shard_champion_spawns	slug PK, name, grp, type, random_type, facet, x, y, z, radius, label
shard_atlas_meta	Singleton (id = 1), payload JSON (counts, a sha256 per source file, parserVersion), imported_at
shard_atlas_pending	Singleton (id = 1), status (pending/rejected), payload JSON, detected_at

The first seven are import-owned: a refresh empties and reloads every one inside a single transaction, so a failed reload leaves the previous atlas intact rather than a half-loaded world. Nothing else writes to them, and nothing holds a foreign key to them — no FKs at all, consistent with every other shard_* table.

shard_atlas_pending is the security-relevant one. A refresh that would REMOVE a facet is never applied automatically: facet loss is indistinguishable at boot from a half-copied or mid-update tree, so it is staged here for an admin to approve or reject, and startup is never blocked by it. Only the decision is stored — source hashes plus the facet diff, a few KB — and approving re-parses the tree, so a multi-megabyte blob never lands in the database and what gets applied matches the tree at approval time. A rejection is remembered against those exact hashes so a declined refresh does not re-prompt on every restart. Everything else (new facets, renamed regions, changed spawns) applies immediately, since none of it can destroy data an operator would miss.

The boot refresh is best-effort by contract: no configured path, an unreadable mount, a malformed file or a database error is caught and logged, and the site comes up serving whatever atlas it had. The tree path comes from the spawn_atlas_servuo_path setting, falling back to SERVUO_PATH.

A refresh re-derives when the tree changed OR the parser did. spawnAtlasSource.PARSER_VERSION is stored in shard_atlas_meta beside the source hashes and bumped whenever the parser produces different data from identical files. Hashing the tree alone would strand an install whose maps never change on whatever an older build derived — a corrected parse would ship and never reach the data.

Four column choices worth stating, because each one is a trap:

• spawn_range, not range, and grp, not group — both are reserved words.
• DELETE, not TRUNCATE. TRUNCATE is DDL in MariaDB and implicitly commits, which would defeat the all-or-nothing reload. At ~7k rows the difference does not matter.
• Point ids are assigned explicitly, not left to AUTO_INCREMENT: the shard_spawn_point_types rows need to know them, and conn.batch() reports no usable insertId for a multi-row insert.
• Plain INDEX on name, deliberately not FULLTEXT. ~800 creature rows makes a LIKE scan free, and FULLTEXT's minimum token length would break searches for names like "orc".

shard_champion_spawns is the configured altar roster ("there is an Unholy Terror altar in Deceit"). The live champ.update feed in shard_champs is the separate answer to "it is on level 3 right now". Both exist; they are not the same data.

shard_spawn_creatures.art is DERIVED, never written by the atlas import itself — see shard_assets below. The project still ships no creature artwork: sprites live in the operator's own client .mul/.uop files and are theirs, not ours to redistribute. What changed in Protocol 8 is who extracts them: the shard does, from its own client, over the bridge. An operator's hand-drawn map plus images under the (already gitignored) server/uploads/atlas/ still wins over anything imported. Text-only is still the normal, supported state — an install with no shard link never imports one, and even a complete import leaves two thirds of the playable ghost and gargoyle bodies without art.

shard_assets / shard_creature_bodies / shard_asset_meta — the Asset Bridge (Protocol 8)

Creature artwork read from the shard's own UO client (link/v8.md §6, §8, §12).

Table	Shape
shard_assets	asset_key VARCHAR PK (§5's key, e.g. body/34/a0, body/820/a23, static/3922/h33, land/3), family, sha256, bytes, width, height, body, action, direction, file, catalog, imported_at
shard_creature_bodies	slug PK, type_name (the ServUO class name asked), body nullable, status, resolved_at
shard_asset_meta	Singleton (id = 1), payload JSON (catalogue id, extractor version, source fingerprint, counts), imported_at

These are the one part of this schema that is deliberately NOT import-owned, and the reason is the atlas tables sitting directly above them. replaceAtlas empties and refills shard_spawn_creatures on every refresh, and a refresh runs on every boot; an imported filename stored on that row would be destroyed by an ordinary re-parse of the ServUO tree, with the next asset Update finding the client files unchanged, reporting "nothing to do", and never restoring it. So the assets live out here, upserted per key, and replaceAtlas re-derives art from them on the way past — { ...derived, ...operatorMap }, which is the one place "the operator's map wins" is enforced.

Four details that are load-bearing rather than incidental:

• file is a filename under the uploads directory, never a path, and it is content-addressed (uo-body-34-a0-<sha8>.png). A stable name overwritten in place would leave every browser and CDN serving the previous client's sprite from cache, with the row perfectly correct.
• The art derivation joins on the catalogue key, a.asset_key = CONCAT('body/', b.body, '/a0'), not a.body = b.body. The simpler join is correct today and stops being correct the moment deeper animation keys (body/400/a2/f0) arrive, at which point one slug matches dozens of rows.
• shard_creature_bodies IS replaced whole, unlike shard_assets: it is derived from the atlas's creature list, so a slug that has left the atlas has no meaning, and the pass that rebuilds it is a shard round trip rather than a file transfer.
• status keeps the negative answers — unknown (the spawn files name a type this shard's scripts do not define, which is real drift), notCreature (a spawn entry for an item or decoration, a permanent answer), failed. Without them the next pass asks again, and each name costs a real constructor on the shard's Core thread.

shard_spawn_creatures.name already holds the ServUO class name — the atlas build picks the winning spelling of the spawn type token rather than inventing a display label — which is why the body pass needs no extra column to ask its question.

Item and land art: the same table, a different shape of use (phase 5)

The creature catalogue is a set: one manifest walk covers every key, so one stored fingerprint in shard_asset_meta describes all of them and an Update is a hash diff. Item art has no set — the client addresses 49,152 item graphics times 3,000 hues — so those rows arrive one at a time, because something on this site named the key.

Three consequences in this schema:

• catalog is per row, and it is what phase 5 added. It records the shard's art catalogue id (a hash of artLegacyMUL.uop/art.mul, hues.mul, tiledata.mul, verdata.mul and its extractor version), so staleness is a column comparison rather than a manifest diff. A client patch changes it; a restart does not. NULL means "written before the column existed", which is stale by the same test and costs one re-fetch. The body import fills it too, so one column answers the question everywhere.
• shard_asset_meta stays the body catalogue's alone. A warm pass writing there would tell the body import that a client it never looked at is unchanged, and the creature catalogue would quietly stop updating.
• family is now load-bearing, not decoration: body, static and land rows share the table and have different lifetimes. The admin status counts them separately for the same reason — there is no "how many are there" to compare static against, so the only honest number is how many the site has been asked for and holds.

Pictures land in server/uploads/items/ (gitignored like uploads/atlas/), content-addressed the same way — uo-static-3922-h33-<sha8>.png. A key the shard has no art for writes no row at all: an empty row would make it "held" and it would never be asked again, including after the operator patches in the graphic that was missing.

shard_clilocs / shard_cliloc_meta — UO's localization table (Protocol 3.0)

Items on the wire carry a LabelNumber, not a name. The bridge has always sent it — char.profile.equipment.cliloc, reward titles as a cliloc number in string form, and one per marketplace listing — but with no table to resolve it against, the character sheet could only render id 1023721 where the game renders "quarter staff".

Table	Shape
shard_clilocs	number INT PK, flag, text TEXT
shard_cliloc_meta	Singleton (id = 1), payload JSON (source — bridge or file — the base's fingerprint under base, the overlay hashes, per-source counts, parserVersion), imported_at

Import-owned and all-or-nothing in one transaction, same contract as the atlas — including DELETE, not TRUNCATE, for the same reason.

The base table comes from the SHARD on any install with uo-link configured (Protocol 8, phase 2): it reads its own client's compressed Cliloc.enu and serves the table paged over the bridge, so nothing is converted and nothing is copied to the web host. Without a shard link it falls back to a converted file on disk at a path from the cliloc_client_path setting, then UO_CLIENT_PATH — the pre-protocol-8 pipeline, deprecated rather than removed so an existing install keeps working.

Overlays are always the filesystem's, either way: ServUO has no server-side notion of a custom cliloc, so custom/ is the only place shard-added ids exist and there is nothing on the shard to ask for. That gap is in the game, not in this pipeline.

Nothing client-derived is committed: UO's strings are EA's, exactly as the creature sprites are. A shard with nothing configured is fully supported — names render as ids. Full design and operator guide: CLILOCS.md.

It reads a SET of sources, not one file, because shards edit items and add new ones and those carry cliloc ids no stock client table has. A base (from the shard, or a converted file) plus every overlay under custom/ are hash-gated together, exactly as the atlas re-reads Regions.xml + Locations/*.xml + Spawns/*.xml + ChampionSpawns.xml. Later sources win, so an overlay both adds ids and overrides stock ones, and adding one custom item never means re-exporting a 5 MB client file. Scale, measured on the live shard: its script tree references 16,434 cliloc ids and only 37 are absent from stock — tens of entries against a 67k base, which is why this is an overlay and not a second table.

The conversion step used to be unavoidable, and is not any more. Every current client ships its cliloc files compressed (first DWORD's high byte 0x8E) and ServUO's own bundled Ultima.StringList cannot read that either — which is why, for two protocol versions, the operator had to install UOFiddler, build a converter against its Ultima.dll and copy a 5 MB file to the web host. Protocol 8 phase 2 ported the Mythic decompressor into the overlay, so the shard reads its own client and supplies the names. The file half survives only as the fallback above, where the plain layout and a delimited text export are both accepted, sniffed by header rather than extension.

Two consequences for what this table holds. shard_cliloc_meta.payload carries the base's fingerprint under meta.base (the shard's file size, mtime, hash and EXTRACTOR_VERSION) separately from the overlay hashes, because on the bridge the old clilocs.plain label is supposed to disappear and a single hash map would read that upgrade as a vanished source. And boot does not import on the bridge path: a file could be re-hashed locally on every restart, but asking the shard would put a sidecar round trip in the boot sequence for a table that changes only when an operator patches their client. Importing is an admin action.

Three decisions worth stating:

• text is TEXT, not VARCHAR. Long property descriptions reach 12 KB. The index that matters for marketplace search is the denormalized shard_vendor_items.display_name, not this table.
• Blank entries are dropped at import — 123,490 parsed → 67,496 stored. Roughly half a cliloc table is empty strings for ids the client reserves and never uses; a row that resolves to no name is indistinguishable from no row at all, and dropping them makes the binary and text imports converge on identical content.
• Two refusals, one of them the atlas's. A corrupt source fails the parse on a truncated record, so it is caught outright and leaves the previous table serving. But a source that has vanished parses perfectly and imports a table quietly missing everything it contributed — an unmounted volume and a deliberate deletion are indistinguishable from here, which is precisely the ambiguity the atlas stages a facet removal for. So it is escalated: status: 'needsReview', nothing applied, missingSources reported by both the import and status(), and an admin accepts it with {approve:true}. That is a flag rather than the atlas's approve/reject pair because the atlas stores a pending decision so that approving re-parses the tree; here nothing is stored, so re-reading at approval time is automatic.

Resolution is server-side and there is no public route. The table is never served as a table: 67k rows would dwarf any page using them, and the Android client consumes the same already-resolved JSON. resolveMany() returns only ids that resolved to something displayable — placeholders like ~1_val~ are stripped, since the bridge sends the id and never the property packet that carries the arguments — and it never throws, because a cliloc lookup is decoration on a character sheet.