Monday, February 28, 2011

Problems with Windows Authentication over a Site-to-Site VPN

Ever had a problem on a site-to-site VPN where Windows machines took a very long time to authenticate to a domain controller on the other site of the link? It's possible the problem is a need to fragment packets that don't want to be fragmented. The solution could remind you of trying to stuff a fat guy in a Tron suit.

Issues with maximum frame size and don't fragment are common when a IPSec tunnel is between the user's and their resources. The problem is caused by the extra headers that IPSec and GRE add to the original packet, causing the total packet size to exceed the MTU. A good way to troubleshoot this is to run the following ping command on a workstation, with the destination being the domain controller.
Ping -f -l 1472
The -f switch sets the don't fragment flag on the ICMP packet. The -l 1472 switch sends a 1472 byte packet.

With the added IPSec headers, a 1472 byte packet will fail to reach the destination, unless it can fragment the packet.

Below is an example of the results, if the packet cannot be fragmented.
c:>ping -f -l 1472

Pinging with 1472 bytes of data:

Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
The fix? Well, there are a couple of things that can be done to get this working. The first is to use Path MTU Discovery. PMTUD (Path Maximum Transmission Unit Discovery) was developed to avoid fragmentation in the path between the endpoints. It is used to dynamically determine the lowest MTU along the path from a packet's source to its destination. PMTUD tries to determine the largest packet size that can be delivered without fragmentation. When PMTUD works correctly the end stations negotiate a frame size that will not exceed the capacity of the link in the middle. Unfortunately, PMTUD doesn't always work.

There are three things that can break PMTUD, two of which are uncommon and one of which is common.

  • A router can drop a packet and not send an ICMP message. (Uncommon)

  • A router can generate and send an ICMP message but the ICMP message gets blocked by a router or firewall between this router and the sender. (Common)

  • A router can generate and send an ICMP message, but the sender ignores the message. (Uncommon)

The first and last of the three bullets above are uncommon and are usually the result of an error, but the middle bullet describes a common problem. Typically, ICMP packet filters are configured to block most, or all, ICMP message types rather than only blocking certain ICMP message types. A packet filter can block all ICMP message types except those that are "unreachable" or "time-exceeded." The success or failure of PMTUD hinges upon ICMP unreachable messages getting through to the sender of a TCP/IP packet. ICMP time-exceeded messages are important for other IP issues. So, another solution may be required to resolve this problem.

Another solution is to configure the 'crypto ipsec-df bit clear' which will clear the DF bit and allow fragmentation. This will generally solve the problem related to the larger packet sizes, by overriding any existing DF bit flags. Want to read more about this issue? See the following articles:
Enhanced by Zemanta