Not to mention you could at that point burn/etch the weights into silicon directly, and have models as 'ROM cartridges' that could perform at thousands of tokens/second, enabling entirely new use-cases.