It’s actually really useful for web devs to have access to a local model. Whether or not browsers should bundle their own rather than using the system-provided model(s) is up for debate, however. For the time being, though, Google does have some of the better small ones.
Furthermore, users aren’t going to want to have to wait for an extra thing to download before their web apps can use AI.
That’s the thing… Without context of why, users probably wouldn’t want a 4 GB download. But they do want their web apps to work properly. When there’s a specific use case they’re interested in, they will want to have it, and they won’t want to wait.
You haven't even tried to provide a hypothetical example of what a web app should try to do using a local LLM, nor addressed the obvious questions about how that kind of thing should be standardized, what level of local LLM capability is reasonable for a web app to expect, or how permissions for that should be managed given that a local LLM is not just a tax on local storage capacity.
So why should anyone take it as a foregone conclusion that this is an instance where web devs should get what they want? In general, the browser should be acting in the best interests of the user and not automatically granting the wishes of every web site that wants to drain your battery.
One example I have that made me excited for this feature is the free recipe manager website I run.
Many of the paid-for competitors give users the ability to import unstructured recipe data these days from sites like instagram or at least text-only websites.
I can't afford to offer this as a feature since my website has no advertising and I just pay for it out of pocket, but it's an incredibly easy feature to add if you have the money to pay for tokens.
If I could use a local llm to do it though that runs in the person's own browser then I think it would definitely be valuable.
That said, I'm not sure the state of local llms provides a good enough experience yet (small models and slow) but that doesn't mean that in the future it might not be useful.
The propsosed apis do work for this purpose, albeit more slowly and lower quality
> not automatically granting the wishes of every web site that wants to drain your battery.
Pretty sure that ship sailed way back when Flash ruled the Internet, and it's still sailing more than ever today.
Browsers are just weird sandboxed VMs now. They have nothing to do with their original purpose. Don't be mad at me, I like shipping webapps that render documents server-side and use even JS incredibly sparingly. I'm just reporting what I see. The browser exists as a way to make developing completely proprietary apps with proprietary UIs for several platforms cheaper, and Chromium exists to help further that goal including, if necessary, being packaged up and shipped with those apps (Electron).
There is a link elsewhere in this comment tree addressing all of that:
https://developer.chrome.com/docs/ai/prompt-api
Google's marketing for their latest new browser feature nobody asked for shouldn't be taken at face value. Somebody outside Google needs to provide a well-reasoned assessment of the feature proposal.
And having skimmed that page, it really doesn't answer most of the important questions. Are other browsers expected to ship Google's model, or put a different model behind the API that Google has documented as being specifically for Gemini Nano?
> Somebody outside Google needs to provide a well-reasoned assessment of the feature proposal.
I'm sure that exists already, I've personally been waiting for some version of this to make it into browsers since like GPT 3.5. Every day on HN there is conversation about the tradeoffs of local vs. hosted models; the uses this API is intended for are perfectly within the capabilities of local models.
> And having skimmed that page, it really doesn't answer most of the important questions. Are other browsers expected to ship Google's model, or put a different model behind the API that Google has documented as being specifically for Gemini Nano?
Most of this is answered under the tag "Intent to Experiment" and the associated link. It's not a mandate that they're forcing on the web today, it's a public experiment intended in part to solicit feedback for a potential spec: https://groups.google.com/a/chromium.org/g/blink-dev/c/6uBwi...
> Most of this is answered under the tag "Intent to Experiment" and the associated link. It's not a mandate that they're forcing on the web today, it's a public experiment intended in part to solicit feedback for a potential spec: https://groups.google.com/a/chromium.org/g/blink-dev/c/6uBwi...
That does pretty much settle the question: if it's still just an experiment, Google should be asking for consent and web developers should not be assuming that this is the future of browsers, and nobody should be acting like this is a foregone conclusion.
> really useful for web devs to have access to a local model
I’m not opposed to this. I don’t want Google, an advertising quasi monopoly, to be auto-installing its own AIs on everyone’s computers.