More people creates more economic activity and higher productivity, which is deflationary. Fewer people lowers productivity and depresses economic activity, causing inflation. You have it backwards - the real economy is not a closed system with fixed amounts of positions and finite money needing zero sum thinking.
And it’s a good thing all that wealth is evenly distributed and not hoarded nearly exclusively by a small class of families.
I can assure you mass immigration is not good for the working class families of this country despite what an economist might say in aggregate. The reality is more people drives up the costs of food, shelter, medicine, and other resources that are not very elastic.
Don’t overthink it.