If they are behind a NAT/ stateful firewall there is just so much connections it will handle at once. I think OpenWRT has like 16K max by default, f.ex. So for less than 16K requests by different users/IPs… each is kept for about 1 minute I think… it quickly will go down, I guess. :)
cat /proc/sys/net/netfilter/nf_conntrack_max
Should give some details.
Do you know if using the DMZ feature on most routers instead of port forwarding would get around this limit, or if there's any other way?