Take pcaps on your client and your server and compare...

I've seen some nasty things happen with broken NICs/drivers. One side gets nonsense data that the peer didn't send, and has poor behavior as a result.

Maybe add iperf3 to the test case to really get the network working...