Also remember that if the untrusted SVG file is served from the same origin and is missing a `Content-Disposition: attachment` header (or a CSP that disables scripts), an attacker could upload a malicious SVG and send the SVG URL to an unsuspecting user with pretty bad consequences.

That SVG can then do things like history.replaceState() and include <foreignObject> with HTML to change the URL shown to the user away from the SVG source and show any web UI it would like.

how is that special/different from an HTML URL?

Because displaying user-submitted images is pretty common and doesn't feel like a security footgun, but displaying user-submitted HTML is less common (and will raise more careful security scrutiny).