Limiting total number of incoming connections for a port with iptables for high loading server

someone asked:

I have a server program listening on a port, and I want to limit the total number of incoming TCP connections for the port.

I tried the following rule:

-A INPUT -p tcp --syn --match multiport --dports 443,4443 -m connlimit --connlimit-above 15000 --connlimit-saddr --connlimit-mask 0 -j DROP

But it will introduce high CPU usage of “ksoftirqd”, and high packet response latency.
The normal traffic of the server is to have about totally 14000 ~ 15000 connections coming from 9000 ~ 11000 different source IPs.
Is there any other solution?

OS: Red Hat Enterprise Linux Server release 7.5.
iptables: v1.4.21

My answer:

It’s not surprising you are having a performance issue, since you’re now asking the system to spend a lot of time counting connections to determine if there are more than an arbitrary number of them, and it has to do this on every new request. (And for some reason it seems you are making lots of short-lived connections. Try to avoid that if you can.)

Ultimately if you want to keep this level of performance, you should put a separate firewall in front of the server, which can do all the necessary connection tracking and allow your server to focus on serving its application. This will introduce a slight bit of latency, though, so if your application is very latency-sensitive then you might not have any good solution at all. (Except for making the short-lived connections longer-lived.)

View the full question and any other answers on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.