Why are the images always generated in the same orientation (upright)? Do the cells have awareness of what is "up"?

IIRC training starts with the initial state and the end state, and the end state is always oriented the same way. It would be interesting to see what would happen if the end state was rotated randomly though I suspect it wouldn't work so well.

Checkout the Isotropic NCA blog from the Google Zurich team: https://google-research.github.io/self-organising-systems/is...

yeah normally NCAs have a sense of up and left. There are some isotropic variants that make the perception fully rotation-invariant.