> But I also think that the state of the art in small LLM and user device capabilities aren't there yet to put a "good enough to be actually useful" local-only LLM as a prepackaged thing in a mass market distributed browser.

Then just allow it to be enabled on high end devices? But it must be local only. As hardware advances and people upgrade, more people will be able to turn on the feature.