MTU Mismatch (Asymmetric MTU)

[visitor68]

Hi, folks 

So, imagine a server connected to a switch port whose MTU is 9000 BYTES. Then imagine the traffic has to get routed and sent out a routed interface (an interface with an IP address applied) whose MTU is set for 1500. 

I am wondering what will happen if the server sends an 8000 BYTE frame to the switch and the switch has to route it out the 1500 BYTE MTU interface. My thought is that the switch will drop the frame. Or will it fragment it and send it out over several IP datagrams? Or will something else happen? 

Thanks

[Philip D'Ath]

The switch should send back a "icmp destination unreachable" message with the "packet too big" option, which tells the server to reduce the MTU for sending to that client.

[visitor68]

Thanks...what if the uplink were a Layer 2/dot1Q trunk 

[pwwiddicombe]

The interfaces on the uplink should have the MTU set as well, or you will get fragmentation of packets.  Remember the default will be 1500, so you have to enable it on every link of the full path that you expect to pass jumbo frames.

You can test from a Windows server using the -f flag

ping <ip address> -l 8000 -f

If it doesn't work end-to-end, then do a tracert <ip address> to find out the ROUTED path to the destination, and use the above on each of these to find the network SEGMENT that it fails on (remember a single hop may have multiple switches, depending on your network size).  Then verify the config of all switches leading up to that failed jumbo ping.

A little utility called mturoute can simplify that job; but you still have to dig to find specific switches to configure.

[visitor68]

Thanks, man...can you look at the follow-up question I asked Joseph?

[Joseph W. Doherty · Accepted solution]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages wha2tsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

What the L3 switch will do, with a too large MTU packet, on a L3 interface, depends on the IP packet.  If the packet's DF (don't fragment) bit is set, switch will drop the too large packet and send a notification to the source.  If the packet's DF bit is not set, switch will fragment the packet and forward the fragments.

To your second question, if the egress port was L2, but supported a smaller MTU (a bad configuration, but if so configured) port will drop frame and not send any notification.  (Oh, and if other side of link cannot take the larger frame, it too will drop it without any notification (another bad configuration).

[visitor68]

Joseph, you wrote what I suspected. But I have a quick follow-up question: if indeed the DF bit is NOT set, we agree that the router will send the fragments out the routed interface that has the smaller 1500 BYTE MTU, BUT will the router also send an ICMP notification notifying the server (sender) that the packet had to be fragmented? Or does it only send ICMP notifications back to the server if the DF bit is set and it could not fragment because of it?

I normally wouldnt bug anyone with such questions - I would look it up. Just cant seem to find a doc that addresses that particular case. 

[Philip D'Ath · Accepted solution]

A layer 3 interface only sends an ICMP "destination unreachable" "packet to big" if it can not forward the packet.  Aka, if you get the ICMP message your packet was dropped.

[visitor68]

Thanks, Philip. Can you cite a document that says that? I havent been able to find any that soecifically addresses that scenario

[Joseph W. Doherty]

Ditto to what Philip answered.

Probably documented in some RFC.

(The likely reason the notice is only sent when the packet is dropped, is because it's dropped.  Fragmented packets will still be delivered.  So, although perhaps not optimal, it doesn't preclude the packet's delivery.  [Actually, looking for notification that packets are being dropped, yet forwarded, is an interesting idea - you can propose such as an RFC.])

[visitor68]

Joseph, when you said "Actually, looking for notification that packets are being dropped, yet forwarded, is an interesting idea - you can propose such as an RFC." did you mean a notification sent even though the packet is forwarded?

[Joseph W. Doherty]

Yep.

As it works now, sender doesn't know packets are fragmented unless it sends a too large packet with DF set and then is notified it had to be dropped.

Of course, if the sender is clever, it can send the "data" as it would with DF not set, and concurrently send test packets, of what the sender believes is max MTU with DF set, to same destination, and monitor for too large message.  If received, it can adjust "data" packets to conform to available end-to-end, MTU.  (A flaw with this approach, generally same "flow" packets are forwarded along the same path, but different "flow" packets might be directed along another path.  I.e. "data" packets, and MTU check packets, could be on different paths.)

BTW, in older MTU too large implementations, I recall that's all the sender received, i.e. it had to "experiment" to find actual max MTU.  Later implementations inform sender what max MTU is.

[Philip D'Ath]

If would be great if you could mark and rate helpful answers.  :-)

[williammanurung]

Hi .

Can you send link to me about the documention?

Thanks