> But my students weren’t as happy as I was - they wanted to build something genuinely useful, and they were really disappointed that our “product” had strong architectural limits and couldn’t outperform titans like nginx and haproxy.
I took a (very brief) look at the github repo [1], it doesn't look like you're doing anything with cpu pinning.
You can probably eke (thanks) out a bit more performance if you cpu pin your threads and cpu pin your listen sockets (sockopt SO_INCOMING_CPU).
If you also cpu align your outgoing sockets, you should get a significant boost, but afaik, there's no great api for that. Linux does have an api for compatible NICs (traffic steering/flow steering) which can work, but if you know what hash your NIC uses (it's probably toeplitz) and you manage source port selection to your backend, you can pick ports that will hash properly.
The goal is for your proxy to be able to handle packets without any cross cpu communication.
Basically, v0 and v1 of the repo is completely different implementations, written almost from scratch. Now working on the 3rd one implementation, I believe the last one. :) Completely different architectural choices was made.
If it's still running on more than a single core, and your students want it to go faster, aligning the work to cpus will almost certainly be useful.
I saw you mentioned windows development elsewhere. You might be interested to know that Microsoft pionered Receive Side Scaling and Send Side Scaling. If you try your proxy out on Windows, be sure to hook into those systems there.
The less work your proxy does, the more important avoiding cross core communication is.
Pin threads to cores, and make sure threads different cores aren’t writing to the same 64 or 128 byte block. Lookup “false sharing”
Thanks for the write up. So the first version was synchronous, second version was using epoll and third one will be use io_uring?
I would be interested to see benchmarks for that patch
I don't have the right setup to make good benchmarks for this right now, but when I had the chance to put it into practice, the improvement between no cpu alignment and full alignment was quite large. That was on a 28 core machines (with 16 nic queues); many years ago, but IIRC, I got at least 10x the connections/sec out of the boxes after tuning and after tuning 12 cores were idle ... the machines were repurposed, if they were ordered for this, they should have had one core per nic queue in a single socket. The difference is likely smaller on a 4 core machine as described in the article.
The hardest part is going to be generating enough load. I had production load, which has the benefit that you don't need to generate it. Otoh, it was a transitional need, and I couldn't reasonably test above 50% of peak traffic on a single machine ... I hit that mark around the time traffic started dropping, and then it wasn't fun anymore.
eke