What part of "run locally" (inference on my machine) did I misrepresent as "run remotely"? You're saying they'd never do it because they can (somehow) make it cost effective to buy a few new graphics cards for every user that signs up?
What part of "run locally" (inference on my machine) did I misrepresent as "run remotely"? You're saying they'd never do it because they can (somehow) make it cost effective to buy a few new graphics cards for every user that signs up?
You would want them to strictly train models and then I guess give them away to you so that you can run them locally?
If for free, won't happen, as I said, you'd just turn them into a dumb pipe.
If paid, yeah, maybe that could work as they offload the compute, but then they would need to figure out how to push ads to you and probably also sell your data
What is going on here.
Mechanical engineers pay gobs of money for software suites that make them productive. They do automatic FEA, renders, etc. There's no ads in those. The parent companies can spend millions / y to improve that software and sell upgraded seats.
Windows as a software product has _barely_ dipped its does into ads, and survived forever at about 100/machine costs.
There are lots of examples. The only reason some software ends up cloud is so you can extort subscription feeds and the software is _cheap_ to run for the user.
LLMs are not like that. They are fundamentally expensive to run for users. An obvious end objective here is to slash all the operational budget / api teams / web ui teams / inferrence infra costs and roll that into just "build better models". It's trivial to wrap a model in a sign-in workflow and ship it to users as a core piece of software that all their other local software can use to gain LLM super powers. Selling upgrades every year keeps people paying 100/machine/user. The market size here is as big as the PC + phone market. It's enormous.
It just seems like the obvious end game is to focus on making good models and productionizing them vs running them as a service with all the headaches that takes while also building useful tools while also training new models.
I have a graphics card my employer bought, they aren't going to care if it costs 100/y more if I gain this productivity boost.
This is how it works (ad free) in almost every other engineering discipline, from matlab to autocad to adobe etc etc.
The future of LLMs is basically Windows XP: A software tax on all machines sold + Ios: A software tax on all phones sold.