Windows 2012 ports exhausted under heavy load

For a test case scenario, I have encountered a port depletion on Windows 2012 server which has its available ports used up quickly due to frequent queries to the database and HTTP requests to web server. Thing happened within a short period of time and nothing can ease the situation except a system reboot. Too many ports are opened but not closed properly which could be mainly due to the arriving order of incoming packets being out of sync.

Here's the extract of resolution:

On Windows platforms, the default timeout is 120 seconds, and the maximum number of ports is approximately 4,000, resulting in a maximum rate of 33 connections per second. If your index has four partitions, each search requires four ports, which provides a maximum query rate of 8.3 queries per second.

(maximum ports/timeout period)/number of partitions = maximum query rate

If this rate is exceeded, you may see failures as the supply of TCP/IP ports is exhausted. Symptoms include drops in throughput and errors indicating failed network connections. You can diagnose this problem by observing the system while it is under load, using the netstat utility provided on most operating systems.

To avoid port exhaustion and support high connection rates, reduce the TIME_WAIT value and increase the port range.

This problem does not usually appear on UNIX systems due to the higher default connection rate in those operating systems.

To set TcpTimedWaitDelay (TIME_WAIT):

    Use the regedit command to access the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\ Services\TCPIP\Parameters registry subkey.

1.     Create a new REG_DWORD value named TcpTimedWaitDelay.
2.     Set the value to 60.
3.     Stop and restart the system.

To set MaxUserPort (ephemeral port range):

    Use the regedit command to access the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\ Services\TCPIP\Parameters registry subkey.

1.     Create a new REG_DWORD value named MaxUserPort.
2.     Set this value to 32768.
3.     Stop and restart the system.

Furthermore, you may have to set another parameter StrictTimeWaitSeqCheck as well, for TcpTimedWaitDelay to be of effect:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters]
"StrictTimeWaitSeqCheck"=dword:00000001

Setting or changing these will require a reboot for the changes to be in effect.