docs/website/UPGRADE_NOTES.md

# Upgrade notes — website

**Operator-facing. Newest first.** One entry per change that requires an operator to *do* something,
or that changes behaviour they would otherwise discover by accident.

Most upgrades need nothing: the schema is replayed on every boot with `IF NOT EXISTS` migrations, and
a deployment that pulls a new image and restarts is done. This file exists for the exceptions — and
the exceptions worth writing down are almost always the ones that **fail quietly**, because a loud
failure announces itself and a quiet one does not.

**Read the entries between the version you are on and the version you are moving to.** Each names the
action, whether it is required, and what happens if you do nothing.

---

## The Asset Bridge: client artwork and names come off the shard now

**Required on a Linux shard host; one action for everyone else.** Protocol 8, the Asset Bridge
([`../link/v8.md`](../link/v8.md)). The shard reads its own UO client and serves what is in it, so
three things that used to need a desktop, a converter or a shared filesystem no longer do.

### What changed

- **Creature portraits on the bestiary** come from the shard's client, not from files you draw or
  place. Portraits you *have* drawn and named in `spawnAtlas.art.json` still win over an imported
  one — nothing you made by hand is overwritten.
- **Item and land pictures** appear beside marketplace listings and on character sheets. They arrive
  one at a time, shortly after a page asks for one, and refresh themselves after a client patch.
- **The cliloc table** (UO's id → name table, which is what makes an item read "a katana" rather
  than "1023721") is decompressed **on the shard**. The old path — install UOFiddler, build a
  converter against its `Ultima.dll`, copy a 5 MB file onto the web host — is gone, and so is
  `server/tools/cliloc-export/` and `UOFIDDLER.md`.
- **The spawn atlas no longer needs to read your ServUO tree.** It used to require a bind mount or a
  shared volume so the *website* could read the shard's `Spawns/*.xml` — the one place the rule
  "only the sidecar talks to the shard" was broken, by the internet-facing component. Those files
  now cross the bridge like everything else. An existing mount keeps working and is still supported;
  you no longer need one.

### What you must do

1. **On a Linux shard host, install `libgdiplus`** — `sudo apt-get install libgdiplus`, or see
   [`../link/SHARD_PREREQS.md`](../link/SHARD_PREREQS.md). ServUO runs under Mono there, and Mono's
   `System.Drawing` is a thin layer over that library, which sits in the **decode** path: without it
   the shard cannot read a single sprite. It is not a crash — the panel reports `NO_IMAGING` with
   the command to fix it — and names and spawn files are unaffected, because neither touches a
   pixel. **Windows shard hosts need nothing.** `installer doctor` checks this for you.
2. **Import once, from Admin → Client Files.** Creature portraits, item pictures and the cliloc
   table are all on that page, with the same two buttons: *Update* transfers only what a client
   patch actually changed, *Re-import everything* fetches the lot.

### If you do nothing

The site runs exactly as it did — every one of these is additive, and a shard with the asset plane
switched off simply has no pictures. The bestiary shows names as text, item listings show numbers
instead of names, and nothing errors.

### The one thing that will not tell you about itself

**Nothing here happens on a restart.** Boot deliberately never calls the shard for client files: a
client patch is an event you know about and the website does not, and a site that re-read 343 MB of
client files on every restart to discover nothing had changed would pay for the rare case forever.
So after you patch your UO client, the site keeps serving the old pictures and the old names until
somebody presses a button on that page. That is the whole reason the page exists.

---

## Engagement: rules, an in-app inbox, and three things that arrive switched off

**Required.** The engagement workstream, Phases 1–12. Three separate actions, and the two that matter
most are the ones nothing will tell you about.

### What changed

The site now has a general notification engine. A **trigger** fires (a forum post, a house going
IDOC, a governor being appointed), a **rule** decides who hears about it and through which
**channel**, and a **template** decides what it says. All of it is admin-editable under
**Admin → Engagement** — Rules, Audiences, Templates, Triggers, Send Log and Suppressions.

Players get an in-app inbox at **Account → Notifications**, with per-channel switches at **Account →
Notification settings**, and the same inbox in the Android app.

### What you must do

Three things, in this order. The first is a required upgrade step for anyone running a shard; the
second and third are only required if you were relying on the behaviour they change.

#### 1. Update the shard — the wire protocol is now 5

**Required if this deployment has a shard connected.** Update in this order:

1. the website (this image),
2. the **UO module to 0.5.0** — **Admin → Modules**, paste the 0.5.0 release install-manifest
   URL, then **restart the server**: modules are read from disk at boot, so nothing changes until
   you do. It declares `coreApi ^1.9.0`, and a core older than that will refuse to *load* it (the
   install itself still succeeds) — the module then shows as failed on that screen, naming the
   version it needs. That is the check working, but it means step 1 has to be finished first;
3. the **shard**, with `installer update` on the shard host. That is what pairs a protocol-5 sidecar
   with a protocol-5 overlay; the installer refuses to pair a mismatched one by design.

**The order matters, and step 2 is the one that starts the clock.** The protocol pin lives in the UO
module's schema: on the first boot after the module updates, `uo_link_config.protocol` moves to 5 and
the site starts sending `X-UOLink-Version: 5`. A shard still on 4 answers `409` from that moment on,
so shard data stops. Nothing is lost and nothing is corrupted — the site degrades to showing the
shard offline, exactly as it does when the sidecar is down — but it stays that way until the shard is
updated. If you cannot update the shard in the same window, update the site and leave the module on
its current version until you can.

Protocol 5 adds a decay schedule and owner account to `house.decay`, fee and owner state to
`vendor.listing`, and a new `account.login.result`. Those three fields are what the shard-driven
rules are built on; a shard on 4 could not feed them.

#### 2. Team notification email arrives switched off

**Required if your Teams rely on email.** Team email used to send automatically. It now runs through
the engagement engine like everything else, as four rules — and like every rule on the platform, they
are seeded **disabled**. Nothing mails until an operator switches one on.

Open **Admin → Engagement → Rules** and enable the Team rules you want. A banner on that screen says
the same thing while they are all off, and disappears when one is enabled.

**Only email moved.** The push tickle and the Discord bridge are unchanged and kept working through
the upgrade, so the app and your Discord channel behave exactly as before. Per-member Team
preferences and unsubscribe links still apply above the rules.

#### 3. News push stops until you switch it on

**Required if you publish news and expect it to notify.** Publishing a news post used to send a push
tickle directly. That call is now an engagement trigger, seeded — again — as a **disabled** rule. A
deployment that upgrades and does nothing else tickles nobody on the next publish.

Same screen: **Admin → Engagement → Rules**, enable the news rule.

This is deliberate rather than an oversight. The alternative was to keep the old direct tickle beside
the new trigger for one release, and an exception with no owner is one nobody removes.

### Behaviour changes that need no action

- **An unsubscribe link now turns off one channel, not everything.** The old link muted a user
  outright, which silenced push as well as mail. It now switches off exactly the channel the link was
  in.
- **Email and push are opt-in; the in-app inbox is on.** A user who changes nothing receives in-app
  items and no mail. That matches the Team preference model already in use, and it is why enabling a
  rule does not by itself produce a mail storm.
- **A rule cannot mail more people than its trigger allows.** Every trigger declares an audience
  ceiling, and a rule is checked against it — so a staff-only event cannot be given a public
  audience, whatever the rule says.
- **Bounces now suppress an address.** A hard bounce or a rejected recipient stops further mail to
  that address and is listed under **Admin → Engagement → Suppressions**, which is also the only
  screen that can lift one. An address that has stopped receiving mail is the first thing to check
  there.

### If you do nothing

The site upgrades, the in-app inbox works, and no mail is sent — because every rule is off and email
needs both a transport (see the SMTP entry below) and a user who opted in. What you lose silently is
**Team email** and **news push**, both of which were sending before the upgrade and stop at it. If
you also leave the UO module on its current version, the shard keeps working on protocol 4 and none
of the shard-driven rules exist yet.

### Data

Nothing is dropped and no row is deleted. New tables cover rules, segments, templates, the outbox,
the send log, cooldowns, suppressions, per-channel preferences and the in-app inbox.
`team_notification_prefs` is kept and still consulted — per-Team granularity did not move.

**Nothing in this schema expires on its own.** Finished outbox rows, the send log and the suppression
list are kept until you remove them; there is no retention sweep. On a busy deployment those tables
grow without limit, which is worth knowing before it is worth acting on.

---

## Email addresses are now unique, and some accounts may lose theirs

**Required only if the dashboard says so.** Engagement Phase 1b.

### What changed

An account's email address is now the destination for account recovery and, in time, for notification
mail, so an address may belong to **one account only**. Until now it could repeat — SSO sign-up wrote
whatever address the provider returned, without checking — so a database that has been running with
SSO enabled may well hold duplicates.

Users can now also **set and change their own address**, under *Account → Email address*, confirming
it by opening a link. Before this there was no way for anyone to change their own address at all.

### What the upgrade does on first boot

Nothing you need to trigger, and **no account is deleted**. Where several accounts share an address:

- the **earliest-created** account keeps it — not the "verified" one, because SSO used to mark an
  address verified merely for existing, so that flag cannot arbitrate anything;
- every later account has its address **cleared** (set to nothing) and is listed in a report;
- the dashboard then shows a warning naming how many accounts were affected.

Case is folded — `Foo@x.com` and `foo@x.com` are the same mailbox — but **accents are not**:
`josé@x.com` and `jose@x.com` are correctly treated as two different addresses.

### What you have to do

If the warning appears, open **Admin → Users** and read the report. It names each affected account and
the address it lost. **Those users are the reason this warning exists:** they can still sign in
normally, but they can no longer receive password-reset or notification email until they set a new
address themselves, and nothing tells them that. Contact them, and point them at *Account → Email
address*.

Dismissing the warning keeps the report — it is the record of what the upgrade did, and the only place
the lost addresses survive.

### Verification email

Confirmation links are sent through the same transport as everything else, so **if outbound email is
not configured, nobody can confirm an address.** The screen says so honestly rather than claiming a
mail was sent. If you have not configured SMTP yet, see the entry below first.

### The verification setting

A new setting, *require a confirmed address before sending notification email*, is seeded **off on an
existing deployment** and **on for a fresh install**. The asymmetry is deliberate: switching it on
retroactively would silently stop mailing every user who had already opted in, on the day you
upgraded. Nothing reads it yet — it takes effect when engagement notifications ship — so there is no
hurry to change it.

### If you do nothing

The de-duplication has already run; it runs on the first boot whether or not you read this. What you
lose by ignoring it is the chance to tell the affected users, who will otherwise discover it the next
time they try to reset a password.

### Data

Nothing is dropped and no row is deleted. `users` gains `email_norm` (a generated lowercase copy of
`email`, which carries the uniqueness index) and `email_pending`. Two tables are added:
`email_verifications` and `email_dedupe_report`. Cleared addresses survive in the report.

---

## Outbound email: the Gmail connect flow is gone; configure SMTP

**Required, if this deployment currently sends email.** Engagement Phase 1.

### What changed

The "Connect Gmail" consent flow has been **removed**. Email now sends through an ordinary mail
transport whose credentials an admin types in — SMTP is the baseline and the only transport shipped.

Removed with it: `GET /api/v1/admin/email/connect/start`, `GET /api/v1/admin/email/connect/callback`,
and email's reuse of the `google` authentication provider's client id and secret. That reuse is worth
a sentence of its own, because it was a coupling nobody could see: rotating the Google **SSO** secret
silently broke outbound **mail**, with nothing on either screen relating the two. Single sign-on is
otherwise unaffected — the `google` provider row exists for SSO in its own right and is untouched.

### What you must do

On the first boot after the upgrade, `email_config.transport` backfills to `smtp` with **no
credentials**. Until you supply some, **no mail is sent** — and nothing errors, because every sender
is written to degrade politely:

- the contact form falls back to a `mailto:` link,
- an invite returns the accept link for you to send by hand,
- a password reset still answers its usual generic success,
- Team notifications are logged and dropped.

That is the risk this note exists for: **email stops and nothing complains.** The admin dashboard
shows a warning while it is true, and the warning goes away by itself once a credential is saved.

Go to **Admin → Settings → Email**, choose the posture that fits, then use **Send test** — which is
now the real verification of the whole configuration, not a formality.

| Posture | Settings | Notes |
| --- | --- | --- |
| **A relay** — recommended | The relay's host, port `587`, *Implicit TLS* **off**, your API key as the password | Mailgun, SES, Postmark and equivalents. Reputation, DKIM and bounce handling are the relay's problem. The right answer for anything with volume |
| **Gmail over SMTP** — the shortest migration | `smtp.gmail.com`, port `587`, *Implicit TLS* **off**, your Google address as the username, and an **app password** | This is the direct replacement for the removed connect flow. It needs an [app password](https://support.google.com/accounts/answer/185833) — 2-Step Verification must be on, and your ordinary account password will not work. Google's per-day send caps apply |
| **Your own MTA** | Its host, port `25`, *Implicit TLS* **off**, username and password left blank | For an operator already running mail on the same host |

Two fields cause most failures:

- **Implicit TLS** is *on* only for port **465**. On port 587 leave it **off** — the connection still
  upgrades to TLS, via STARTTLS. Port 587 with it on does not error; it hangs.
- **Send from** is now typed by you rather than read back from Google, so nothing guarantees the
  server will accept it. It must be an address the account is allowed to send as, or the mail is
  refused for SPF/DMARC reasons that look like nothing at all. Send test names this specifically when
  it happens.

### If you do nothing

Mail stays off. Nothing breaks and no data is lost — the site, the admin panel, logins, SSO and the
shard integration all behave exactly as before. The dashboard warning stays up until you either
configure a transport or press **Clear credentials**, which is how you say "this deployment
deliberately sends no mail".

### Data

Nothing is dropped. `email_config` gains `transport`, `credential_enc` and `reply_to`; the old
`provider` and `refresh_token_enc` columns are kept but no longer read. Your stored Gmail refresh
token is not used again and is cleared the first time you press **Clear credentials**.