> the fact that “will this AI cause harm” is, mathematically, an unanswerable question.

No it isn't -- this is a fundamental misunderstanding. What is unanswerable is "For all x where x is an AI, will x cause harm?" ... but there is an infinity of specific AIs that provably will or won't cause harm.

Likewise this a very common misunderstanding of the Halting Problem -- Turing proved that there is no TM that can prove whether m will halt for all possible TMs m ... but there are myriad TMs that provably do halt or provably don't halt.