So why would CPU speed be the bottleneck then?

Because this board doesn't have specialized routing hardware and will have to do everything in CPU.