From time to time we receive calls into TAC about the mgmt0 interface showing as “Administratively down” from NX-OS. Since the beginning of time, this has been expected behavior due to the platform design architecture on 1st, 2nd, and 3rd Gen FI’s.
The FI switch architecture uses 2 network stacks. We use the Linux stack for eth0, a.k.a management port or mgmt0, and we use the NX-OS stack for, you guessed it, the NX-OS context. Upon FI boot, NX-OS sets the IP address for mgmt0, but it’s not used or connected to the physical interface. Because only one stack can manage the physical port, this management is done by the Linux stack. This is why we have the appearance of this “admin down” behavior.
From NX-OS, we can see that mgmt0 is “Administratively down”, and again, this is the expected behavior we’re talking about.
Weasley-B(nxos)# show int mgmt 0
mgmt0 is down (Administratively down)
Hardware: GigabitEthernet, address: ffff.ffff.ffff
Internet Address is 10.X.X.X/24
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
auto-duplex, auto-speed
EtherType is 0x0000
1 minute input rate 400 bits/sec, 0 packets/sec
1 minute output rate 0 bits/sec, 0 packets/sec
Rx
1287286 input packets 1 unicast packets 1287285 multicast packets
0 broadcast packets 120175716 bytes
Tx
0 output packets 0 unicast packets 0 multicast packets
0 broadcast packets 0 bytes
When we want to verify that our management port or mgmt0 is up and operational, we can check it one of two ways. We can check it from the local-mgmt context with the below command, and here we’re interested in “eth0” specifically.
Weasley-B(local-mgmt)# show mgmt-ip-debug | grep -i -A 7 "eth0" | head
eth0 Link encap:Ethernet HWaddr FF:FF:FF:FF:FF:FF
inet addr:10.X.X.X Bcast:10.X.X.X Mask:255.255.255.0
inet6 addr: fe80::1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2802429 errors:0 dropped:0 overruns:0 frame:0
TX packets:1904269 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:252385641 (240.6 MiB) TX bytes:968324177 (923.4 MiB)
Alternatively, we can scope to each fabric interconnect as seen below.
Weasley-B# scope fabric-interconnect a
Weasley-B /fabric-interconnect # show
Fabric Interconnect:
ID OOB IP Addr OOB Gateway OOB Netmask OOB IPv6 Address OOB IPv6 Gateway Prefix Operability
---- --------------- --------------- --------------- ---------------- ---------------- ------ -----------
A 10.X.X.X 10.X.X.X 255.255.255.0 :: :: 64 Operable
While we’re discussing the design architecture, I’ll briefly explain how CDP operates with this. Because of how these 2 stacks are implemented, we’re able to see neighbor switch information from the FI’s NX-OS context, but the FI will not share out it’s information to the neighboring management switch.
From UCS NX-OS perspective, we see what's displayed below, and notice how we’re able to see the Mgmt-SW. When running a “show cdp neighbors” from Mgmt-SW, we will not see UCS FI information, which is expected.
Weasley-B(nxos)# show cdp neighbors
Capability Codes: R - Router, T - Trans-Bridge, B - Source-Route-Bridge
S - Switch, H - Host, I - IGMP, r - Repeater,
V - VoIP-Phone, D - Remotely-Managed-Device,
s - Supports-STP-Dispute
Device-ID Local Intrfce Hldtme Capability Platform Port ID
Mgmt-SW mgmt0 171 S I WS-C2960S-48F Gig1/0/2
