Based on this research, the impact of these populations on the allocation of representatives is probably not particularly large: https://www.pewresearch.org/short-reads/2020/07/24/how-remov...

Sure, the House is almost evenly split, so a few seats here or there would have an impact. But the net result would probably be further mitigated by gerrymandering, other population shifts, and so on.

One other thing I appreciated from this article is how it touches on comments about simply following the law. Just because something is legal, does not make it morally questionable (at best). From the article:

> The apportionment of seats in Congress is required by the U.S. Constitution, which says that the census will be used to divide the House of Representatives “among the several States according to their respective numbers, counting the whole number of persons in each State,” except for enslaved people, who, until the late 1800s, were counted as three-fifths of a person, and certain American Indians.