Tweaking Linux network parameters
Random browsing takes you weird places, and suddenly I found myself reading up on tweaks for the Linux IPv4 TCP stack – many of them actually recommended by Sun. Your mileage may vary, and you probably should read up on every single option before you apply it. All options go in /etc/sysctl.conf.
Allow the TCP stack to reuse sockets in the TIME-WAIT state:
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
TCP Timestamps are enabled per default, and enables calculation of RTT in a more accurate way (see RFC 1323) than the retransmission timeout. IBM suggests this should be enabled for performance, but others suggest disabling it and saving 12 bytes header overhead. We’ll leave it on:
net.ipv4.tcp_timstamps = 1
Enable selective acknowledgment, which improves performance by selectively acknowledging packets received out of order (causing the sender to retransmit only the missing segments). Should be enabled for wide area network communication, but it can increase CPU utilization. Also enable Forward Acknowledgment (FACK), which operates with Selective Acknowledgment (SACK) to reduce congestion:
net.ipv4.tcp_sack = 1
net.ipv4.tcp_fack = 1
To use large packet windows (over 64Kb), you should enable window scaling as defined by RFC 1323:
net.ipv4.tcp_window_scaling = 1
Lowering the FIN Timeout value will shorten the TIME_WAIT state, freeing up resources for new connections. It is recommended when running applications that constantly create a lot of new connections – ie. a web server. The default is 60, and Sun recommends a value in the 15-30 range.
net.ipv4.tcp_fin_timeout = 15
Tweak the TCP KeepAlive values:
net.ipv4.tcp_keepalive_intvl = 15
# Number of probes before timing out (default: 9, recommended: 5):
net.ipv4.tcp_keepalive_probes = 5
The default maximum for send/receive windows is 128Kb and it’s recommended to boost this to 8Mb:
net.core.wmem_max = 8388608
# Maximum TCP Receive Window:
net.core.rmem_max = 8388608
Also tweak the IPv4 rcv/snd buffers to use a maximum of 8Mb:
net.ipv4.tcp_rmem = 4096 87380 8388608
# Memory reserved for TCP snd buffers (default: 4Kb 16Kb 4Mb):
net.ipv4.tcp_wmem = 4096 87380 8388608
If you’re using a lot of connections, you should make more local ports available. Default range gives a total of 28232 ports available. Increasing this range to 4096-65535 will give you 61439 local ports:
net.ipv4.ip_local_port_range = 4096 65536
Apart from hopefully boosting overall network performance, the tweaks above also lowered the number of sockets in the TIME_WAIT state from 300 to around 50.
Any comments and/or corrections are welcome.
Sources:
