Why is the insane speed of 13KTPS of this site is not more on the the top of the AI conversations?
Because there's been nothing to discuss since their announcement. Their API access immediately closed due to overwhelming demand and they didn't fab newer models than Llama3 yet.
Probably they will make bank selling to HFT for a while.
It's pretty well known by now.
I asked it for a block of C++ code and it hit 14,189 tok/s. I assume it cached someone else's session?
No - it's custom silicon https://news.ycombinator.com/item?id=48693490
Because I just tested it and it took 3-4 clarifications before it actually gave a correct response vs gemini/google search. It's not great, but good.
I'd rather wait 3x as long.
Because there's been nothing to discuss since their announcement. Their API access immediately closed due to overwhelming demand and they didn't fab newer models than Llama3 yet.
Probably they will make bank selling to HFT for a while.
It's pretty well known by now.
I asked it for a block of C++ code and it hit 14,189 tok/s. I assume it cached someone else's session?
No - it's custom silicon https://news.ycombinator.com/item?id=48693490
Because I just tested it and it took 3-4 clarifications before it actually gave a correct response vs gemini/google search. It's not great, but good.
I'd rather wait 3x as long.