All theories are wrong.

Some are useful.

Having theories that only give answers, but you can't reason about is not as useful. Having a theory where you don't know the limits of it's applicability, can be very dangerous.

At least in the physical realm there is not yet anything that combines relativity with QM so they can only be approximations. Even in math so far there seem to be similar challenges using programatic and "AI" driven solutions and proofs.

Still, I know that LLMs will be useful for Verilog/VHDL and particularly with verification, where they are already heavily used. Defined outputs and complete test coverage is already such a big part digital/asic design, I'd be surprised if it isn't used a lot more. Many software people would say that hardware is badly written copy-pasta, as it is. That said, higher velocity slop and hardware "technical debt" isn't something you can fix with an update. And no matter how fast you "ship", you won't get parts back in less than a few months. Poorly used, it will lead to expensive failures.