In theory that would be ideal, I feel like FPGA's haven't kept up compared to GPU's. The latest GPU's will be at 4nm, while FPGA's will be still at 28nm. The pipelines are huge, it would take many FPGA's to fit one LLM if everything is kept on-die. Cerebras is attempting this, but has to use a whole silicon wafer:
We need FPGA's at the latest process node, with many GB's of HBM in the package. Fast reconfigurability would also be a nice have.
I feel like the FPGA has stagnated over the last decade as the two largest companies in this space were acquired by Intel and AMD. Those companies haven't kept up the pace of innovation in this space, as it isn't their core business.
7nm for Achronix
https://www.achronix.com/product/speedster7t-fpgas
> The latest GPU's will be at 4nm, while FPGA's will be still at 28nm.
16 nm (or “14 nm”) for Ultrascale+.