people who don't have aphantasia are demonstrably and measurably better at .. something, right?

Look up memory castles

so you're saying people with aphantasia test worse for memory?

The phrase is "memory palace": https://en.wikipedia.org/wiki/Method_of_loci