Shouldn't the ignoring of scripting be done at the user agent level? Maybe some kind of HTTP header to allow sites to disable scripts in SVG ala CORS?

It's definitely a possible solution if you control how the file are displayed. In my case I preferred the files to be safe regardless of the mechanism used to view them (less risk of misconfiguration).

Content-Security-Policy: default-src 'none'