Hyperflex UCS QoS System Classes MTU parameter setting

[etmarcof]

Hi,

I have installed one hypeflex cluster  4.0.2 with FI 6454  (Clean Install) and on UCS Manager  i noticed  that  on QoS System Classes   all priorities (Platinum,Gold, Silver, Bronze,Best Effort) had MTU set to 9216.  In Cisco documentation  i saw only  MTU set to 92126   on Platinum and Bronze Priority that is used by Storage vnic template  and Vmotion  vnic template.

Also On vnic Templates only show MTU 9000 in the Storages  and Vmotion templates.. 

Will this cause impact or its just an mistake done by HX Data platform installer? 

Another question  regarding Jumbo MTU, if i connect this  hyperflex cluster to upstream network switches without jumbo frames configured .In normal operation i won't notice impact correct? Only if some node or connection fails and i need to traverse upstream switches correct?

Thanks.   

[Kirk J · Accepted solution]

Those are default settings for the 6454 (i.e. best effort having 9216 set).

VMware uses 9000 MTU  max size, as you want to leave some room for overhead potential (i.e. OTV, vxlan, etc) and don't want your OS level creating frames that completely max out our networking gears capability.

Having QOS classes that 'allow' the 9216 frame size, doesn't make the traffic be jumbo in nature, it just 'allows' for it, and doesn't drop it.

Here is a potential scenario if your upstream networking does not allow jumbo frames, but you have it set/configured at Hyperflex level.

• You upgrade your UCSM. During upgrade process and FI is rebooted (lets say FI-B) where all your default storage vnics are pinned.
  • this also could happen if uplinks to FI-B are dropped, upstream switch rebooted, etc)
• All connectivity switches to FI-A immediately, and the HX jumbo frame storage data traffic continues to be locally switched on FI-A.
• FI-B comes back online.  ESXi sees the links to the 'B" side become available, so it starts 'failing BACK' the connections to side "B"  However, this does not occur simultaneously on every server.  Some storage data connections may still be on the 'A' side and some have moved back to 'B' side.  For the A<>B connectivity to occur  the jumbo storage data traffic now has to get switched upstream.  If your upstream switching gear doesn't allow jumbo frames, the traffic is doing to get dropped, and you will likely have a temporary outage, until all nodes are fully switched back to "B" side, and traffic is all locally switched at the FI level, and not being dropped upstream.

Kirk...