It doesn't quite make sense to keep object identity at the language level. Inherently the data in the arrays cannot be the same memory of the data in the objects fields.

To get the speed up, you can't just abstract it as an access pattern because it's tied to the specific way the memory is laid out.

If you were trying to make some kind of collection type that could be queried by both row and column, you would need to store it both ways at all times and also keep both representations in sync, which also defeats the purpose, somewhat.

I feel like if you're trying to do this pattern then it doesn't make sense to also keep the objects.