Can we unlock FP64 so we can do serious work on second order methods? Even at enterprise grade the last good fp64 gpu was the H100. B100 focused on lower fp due to llms.

I don't believe so. As I understand it, the only Ampere GPU with a full complement of FP64-capable cores is the GA100 used in the A-series server cards (which, as you suggest, still underperforms the H-series for FP64).