The benefit in KANs is interpretability, not expressivity. It's a structure that lends itself well to performing symbolic regression or other interpretable downstream tasks. This can make it better suited for scientific tasks, for example. You can easily replicate the practical performance of any KAN with an MLP, and it will train and run faster on modern architectures. This proposes a method it might be faster, but it's early days to me.
Precision in the activation function is targetting a part of neural networks that you don't want. There are many other methods that work with high precision. You use neural networks because of their implicit bias toward regular solutions. That means there is a sweet spot at low precision that you're targetting.
A key benefit of KANs is expressivity, as each layer is significantly more expressive than an MLP layer. This can be seen in our benchmarks: KAN networks need fewer layers than MLPs to match or beat their performance, even in software.
However, on GPUs, KAN implementations are far less efficient than MLPs: since B-spline locality is hard to exploit and lookup operations aren't as efficient. This is your original point about MLPs training and running faster on modern architectures: each KAN layer is more expressive, but its poor hardware efficiency makes it a net negative (at least for current approaches).
On FPGAs, LUT lookups are cheap, so KANs' expressive layers map to very hardware-efficient implementations, and the resulting networks are thus much more compact and efficient than equivalent MLPs.
On your second point: low precision is certainly viable for both inference and learning (as shown in our work), and quantization can even have a mild regularizing effect. However, task performance generally worsens with lower precision (here and across the literature): the use of low precision is fundamentally a result of the efficiency-performance tradeoff.
I generally agree with this rebuttal. Each KAN layer is more expressive on a per-layer basis, although there is a mapping to an MLP with more layers. With the current hardware implementations, yes, MLPs have an advantage overall. I can certainly respect the intention to make KANs faster, since it is a serious issue for more widespread adoption, and KANs certainly have their value.
I'm still very skeptical of arguing for KANs as an eventual replacement, like I've seen some papers on the subject argue. The reduced depth may not be an advantage. For example, higher depth for standard neural networks doesn't just add to expressivity, it actually induces spectral sparsity bias. KANs have a bias of their own, but it is different, and is sometimes better, sometimes worse, depending on the task. If increasing depth turns out to be important, KANs might remain less efficient overall.