Is their hardware programming interface reasonable for implementing inference of frontier models: no quantization, several tera params?
BTW, how many many params open weight frontier models have? A few teras, 100s of teras?
Is their hardware programming interface reasonable for implementing inference of frontier models: no quantization, several tera params?
BTW, how many many params open weight frontier models have? A few teras, 100s of teras?
Yes, ROCm can be used to run frontier models and is being used by OpenAI, Anthropic, and Meta.
I would prefer direct hardware kernel interface.
Like linux DMABUFs with userland hardware command ring buffers (I guess this hardware ring buffer instance would be specific to a VMID and a PASID).
You can do that (tinygrad style) if you want.
Kimi-K3: 2.8T
Qwen3.8-Max: 2.4T
DeepSeek V4 Pro: 1.6T
DeepSeek V4 Flash: 284B
(all are total parameter counts, not active parameters)
Rumors say chatgpt/claude/gemini/etc are in the 100s of teras. True?
I'll ask my uncle (he works for Nintendo) and get back to you on that one
My question is that wrong?
Sorry if the joke didn't land; I have heard a lot of different numbers for the size of US labs' models, but never seen any of them substantiated, so I think you're likely to just get more rumours in answer to this question.
My personal take, with no sources: 100T sounds excessively high given they need to be able to actually serve these things on commercially available hardware. I would guess they are in the same order of magnitude as the Chinese frontier models. It's possible their edge is in RL training methods, training-time compute, and access to data (e.g. from customers' CC/Codex sessions), not in model size.
No, rumors say 5-10T.