> Chip Design
> As an early proof of concept, Kimi K3 designed a chip to serve a nano model built on its own architecture. In a single 48-hour autonomous run, K3 built, optimized, and verified the chip using open-source EDA tools on the Nangate 45nm library. Within 4 mm², the chip closes timing at 100 MHz and sustains over 8,700 tokens/s decode throughput in simulation, packing 1.46M standard cells, 0.277 MB of SRAM, and an INT4 MAC array with fused dequantization. A chip built by a model, for a model, reflects K3's long-horizon agentic capabilities.
Absolutely wild.
Really feels like end game type stuff - AI designing its own next versions, designing its own chips, etc..
The advancement is slow, but fast - like a plant growing. We really are the boiling frogs now aren’t we?
And the people with eyes wide open are us, and anyone that frequents this site really. Is this Milliways?
GPT 5.6 Pro one shot a perfect score to the new IMO today and nobody cares. We are in the end game.
go watch the music uptown funk music videos they generated. We still got a few years.
groq did an ASIC for llama and now for nvidia. Their cloud service is fast.
> NVIDIA Groq 3 LPU Inference Accelerator > The NVIDIA Groq 3 LPU is the next generation of Groq’s innovative language processing unit. Each LPX rack features 256 interconnected LPU accelerators that, together with the NVIDIA Vera Rubin platform, supercharge inference. Each LPU accelerator delivers 500 megabytes (MB) of SRAM, 150 terabytes per second (TB/s) of SRAM bandwidth, and 2.5 TB/s scale-up bandwidth.
https://www.nvidia.com/en-us/data-center/lpx/
I wonder where those now worthless ASICs are rotting
I had a thought a while back: sell large local models burned onto fused compute / ROM chips. Like cartridges for old game consoles. Slot (or probably plug into USB-C) and go.
It’s an ASIC with the model wired into it so it’s very low power and fast.
I’d buy these. Say $100 for a frontier class model. Maybe more.
Taalas is developing this, but not for Frontier class models. I hope that if we can least get the easy 80% of work done on that sort of hardware, we can greatly reduce the demand for GPUs, HBM and energy to some extent.
There is an amount of brute forcing that becomes possible at those speeds that I think could even take us beyond 80%. If we could have Qwen3.6-27B running at 15k t/s, run 100 attempts concurrently, select top-K solutions and synthesize a final result from them.
There was a paper a while back that showed top-K selection like that with tiny models was able to reliably solve some 1M-step Tower of Hanoi when no frontier model could. Very big level up in capability just from horizontally scaling compute.
100 dumb folks don't make an Einstein
You pull out Einstein when you need a breakthrough.
Taalas[0] seems to be what you're talking about.
0. https://taalas.com/
I love this for the popular sci-fi trope too, where you see some ship engineer swap one glowing crystal "compute core" for another.
We could have the photonic AI model ASICs for real!
Interestingly, you could easily run them from the said old consoles! You'd just need a bit of console code to interface (text input/output) with your fully independent LLM subsystem. Imagine Claude for the NES without Internet?
This would be very compelling. Can anyone share more details on how it would work? Only issue is that you are stuck at a certain point in time but that’s not a huge deal. Even just a good 27b model would be useful.
Talaas have done this with a llama 3 model. Runs at like, 16k/tokens a second oror something obscene. Very little power draw too.
Doesn’t need hbm or lots of memory, because the hardware can just forward the data straight to the next layer and you don’t need to round trip through memory.
They claim to be working on an approach to make the underlying hardware a bit more reusable between models.
Yeah, if you have a fixed llm topology, you can just effectively burns 2 top layers of the chip as Rom (model weights) - which has a per area density even better than dram - so it’s just attention and kv streaming that is hbm to sram transfer.
Most big model weights will not fit a single reticle sized chip - so you’d have prob 30 different chips to split the model .
And you’d need super fast chip to chip comms for the all-reduce and similar.
So scaling to 1T models is hard - and a long lead time - but can be very power efficient.
Kinda already exists.
demo https://chatjimmy.ai/
https://news.ycombinator.com/item?id=47103661
Hallucinates on the first question I ask, as 90% of these models that try to take shortcuts.
You’re expecting the wrong thing. The demo demonstrates the insane inference rate of dedicated hardware. Iirc it’s llama 3 or something. Not a very good model by today’s standards. But it runs at 16k tokens per second, an order of magnitude above the competition.
Imagine what’s possible if you had GLM-5.2 turned into a hardware chip like this.
Your statement reminds me of Avenger's scene where Tony choosing Friday among other AI's catridges to use.
That sounds good and practical to happen!
> I’d buy these. Say $100 for a frontier class model. Maybe more.
Sure you would. Running frontier class models on current hardware costs in the order of tens of thousands of dollars. It is more likely that these custom ASICs will be priced competitively with that, and not with Super Mario Bros.
Oh, and energy consumption will be in the same order.
https://chatjimmy.ai
You need terabytes of memory to run a frontier class model
I wonder, if you can run at 8k or 15k t/s, you could in theory run 10 or 20 agents (or more) at the same time and generate hundreds of versions, then just analyze them. Think thinking mode x1000 at least... Would be interesting to see how good it would be
How very Cyberdyne.
[dead]
Wow, I'd really love it if that were the case. I'm already pretty satisfied with just GPT 5.6 as it is.
How nano are we talking about here? A single transformer head and a few dense layers?