I don‘t understand why you would train a NN for an operation like sqrt that the GPU supports in silicon.

I see it as a practical joke or a fun hack, like CPUs implemented in the Game of Life, or in Minecraft.

It’s been done already. Have a look at Quest for Tetris: https://codegolf.stackexchange.com/questions/11880/build-a-w...

I actually ran Sokoban under EForth running on top of subleq/muxleq with a VM interpreted under few lines of AWK.