It's not hard to imagine a future where I license their network for inference on my own machine, and they can focus on training.
It's not hard to imagine a future where I license their network for inference on my own machine, and they can focus on training.
The problem with this is that the temptation to do more us too big. Nobody wants to be a "dumb pipe", a utility.
I don't understand the dumb pipe analogy. Sure they can sell a "cloud inference" as well, but it's way cheaper for them to license the model and let me run it. Let me decide. If I have hardware from a gaming addiction, or a privacy fetish, I'll pay $200/m to use it. But if I want to ramp up use, then maybe 100k/y is the price.
Data we send to LLMs is super valuable. Code bases, counseling (emotional, financial, etc).
The temptation to monetize it in shady ways will be irresistible.
And giving up control will reduce income sources.
What part of "run locally" (inference on my machine) did I misrepresent as "run remotely"? You're saying they'd never do it because they can (somehow) make it cost effective to buy a few new graphics cards for every user that signs up?
You would want them to strictly train models and then I guess give them away to you so that you can run them locally?
If for free, won't happen, as I said, you'd just turn them into a dumb pipe.
If paid, yeah, maybe that could work as they offload the compute, but then they would need to figure out how to push ads to you and probably also sell your data
What is going on here.
Mechanical engineers pay gobs of money for software suites that make them productive. They do automatic FEA, renders, etc. There's no ads in those. The parent companies can spend millions / y to improve that software and sell upgraded seats.
Windows as a software product has _barely_ dipped its does into ads, and survived forever at about 100/machine costs.
There are lots of examples. The only reason some software ends up cloud is so you can extort subscription feeds and the software is _cheap_ to run for the user.
LLMs are not like that. They are fundamentally expensive to run for users. An obvious end objective here is to slash all the operational budget / api teams / web ui teams / inferrence infra costs and roll that into just "build better models". It's trivial to wrap a model in a sign-in workflow and ship it to users as a core piece of software that all their other local software can use to gain LLM super powers. Selling upgrades every year keeps people paying 100/machine/user. The market size here is as big as the PC + phone market. It's enormous.
It just seems like the obvious end game is to focus on making good models and productionizing them vs running them as a service with all the headaches that takes while also building useful tools while also training new models.
I have a graphics card my employer bought, they aren't going to care if it costs 100/y more if I gain this productivity boost.
This is how it works (ad free) in almost every other engineering discipline, from matlab to autocad to adobe etc etc.
The future of LLMs is basically Windows XP: A software tax on all machines sold + Ios: A software tax on all phones sold.