The ANE is not designed to be fast. Nothing on the M-series is designed to be fast. Anytime it's fast is a lucky accident.

It's designed to have optimal performance/power ratios. The GPU is faster but it uses more power.