I fully expect token vendors to go the way of mobile data providers. Increasing throughput and efficiency, yet commoditized and not reaping the massive rewards of the economic gains their technologies enable.
Can't wait for the unlimited plans we will have in due time.
They will not come. You can only spend so much time speaking on the phone.
But if you get unlimited tokens, with scheduled/automated jobs and subagents you will burn a LOT of money.
"Unlimited" for a consumer will probably take a form like limited parallel agents at max intelligence for X amount of tokens, followed by either single threaded agents or nerfed intelligence or both.
This is the same mechanism used in unlimited mobile internet for example.
I rarely use more than 25% of my subscription's weekly allowance. So, it's already unlimited.
> if you get unlimited tokens, with scheduled/automated jobs and subagents you will burn a LOT of money
Maybe, maybe not. There could be a saturation point of compute demand from ordinary consumers. (See: hot-water analogy.)
> You can only spend so much time speaking on the phone.
Tell that to my dial up modem as I played Active Worlds in 1996.
Yeu were still limited to about 4GiB/month usage as a time/bandwidth ceiling
Not where I am, it's 20G mobile and internet is unlimited, the level below mine has 1Tb cap.
The parent poster was talking about the amount of data transfer you could do on dialup over a month in 1996.
Mobile video data and tethered devices would tell a different story. Usage growth has been significant.
I expect they will be "unlimited" in the same sense the mobile / ISPs do that 99% of people never reach the limits.
We could have "unlimited" electricity too, if we choose, by printing solar panels (like the Chinese) and hopefully fusion before long.
> Can't wait for the unlimited plans we will have in due time.
I don't see how this can happen. The cost of serving a model grows linearly with use (energy). The cost of serving a mobile data plan is approximately zero.
> The cost of serving a model grows linearly with use (energy)
The cost of processing tokens however does not. Throughput and efficiency are improving around 10x per year. Hardware is not the only factor.
I'm curious, why would this not turn into an unlimited electricity flatrate? In most parts of the world, consumer electricity is charged by kWh, not as flat payment.
As long as tokens are as energy intense to compute as they are right now, I don't see a flatrate being a viable model.
Power and memory are apt to become the biggest billing constraints.
Something close to how we rent other cloud resources today makes sense. We don't worry about electricity costs, it's built into the hourly prices, use as much of the hardware as you can. Electricity rates vary over the course of the day as well.
People don't like complex pricing. This would be for businesses running larger workloads. For individuals, the unlimited plan seems the most probable to me. This stuff is going to get real cheap in the coming years.
The energy needed per token is in dropping rapidly, even though the energy per-GPU is going up. We are seeing something like 10x improvements per year.
[flagged]