I don't think they ignored gaming; they just knew they would fail if they designed a gaming-first device.
Their best bet to bootstrap the platform is to make it computing-first around the iPad platform, and that means leading with multitasking, AR/MR, and with computing-oriented inputs.
Actually they should’ve limited it to being a spatial computing productivity headset to enhance the productivity of actual professional Mac users, like 3D artists, music producers, programmers, etc. They could’ve severely reduced the size and weight of the headset, maybe even made it glasses form, if they simply anchored it, wirelessly even, to a nearby Mac.
From there they could’ve expanded the spatial capabilities over time and eventually create a consumer version. The Vision Pro is the Newton of mixed reality.