https://arcprize.org/leaderboard
$13.62 per task - so we need another 5-10 years for the price to run this to become reasonable?
But the real question is if they just fit the model to the benchmark.
https://arcprize.org/leaderboard
$13.62 per task - so we need another 5-10 years for the price to run this to become reasonable?
But the real question is if they just fit the model to the benchmark.
Why 5-10 years?
At current rates, price per equivalent output is dropping at 99.9% over 5 years.
That's basically $0.01 in 5 years.
Does it really need to be that cheap to be worth it?
Keep in mind, $0.01 in 5 years is worth less than $0.01 today.
Wow that's incredible! Could you show your work?
https://epoch.ai/data-insights/llm-inference-price-trends
A grad student hour is probably more expensive…
In my experience, a grad student hour is treated as free :(
You never applied for a grant, have you?
What’s reasonable? It’s less than minimum hourly wage in some countries.
Burned in seconds.
Getting the work done faster for the same money doesn't make the work more expensive.
You could slow down the inference to make the task take longer, if $/sec matters.
That's not a long time in the grand scheme of things.
Speak for yourself. Five years is a long time to wait for my plans of world domination.
This concerns me actually. With enough people (n>=2) wanting to achieve world domination, we have a problem.
It’s not that I want to achieve world domination (imagine how much work that would be!), it’s just that it’s the inevitable path for AI and I’d rather it be me than then next shmuck with a Claude Max subscription.
I mean everyone with prompt access to the model says these things, but people like Sam and Elon say these things and mean it.
n = 2 is Pinky and the Brain.
I'm convinced that a substantial fraction of current tech CEOs were unwittingly programmed as children by that show.
Yes, you better hurry.