The presence of the code itself is a threat. There's no good reason it shouldn't be an extension, beholden to all the same "security" restrictions other extensions are.

Yes, a web browser should prioritize security and simplicity, and put optional features in a sandbox.