Switch 2950 all ports designated

[katerina.dardoufa]

Hello community,

we have some pretty old WS-C2950G-48-EI switches in our environment which are vtp clients in the Domain. These connect with two redundant links to two 6509, which are the VTP servers in the environment. The two uplinks are either designated/root ports, since rapid-pvst is used. After performing some changes in the vlans on the VTP servers, these 2950 switches, stopped forwarding packets and lost all connectivity. Via the console we noticed that the uplinks were both designtated ports! No root port was present on the switch. In the switch log we also noticed this message, for which I cannot find an explanation:

"strata_remove_vtable_entry : vlan entry not found"

In both cases where we faced the issue, the switches were connected to port Gi1/1 of 6509 WS-X6724-SFP line cards.

Has anyone seen this behavior before? What does the message in the syslog mean?

Thank you in advance,

Katerina

[Giuseppe Larosa]

Hello Katerina,

be aware of the following limitations that apply to devices like C2950:

- they do not support more then a given number of Vlans in VTP. If the number of Vlans in VTP database is greater then their limit they revert to VTP transparent mode.

>> After performing some changes in the vlans on the VTP servers, these 2950 switches, stopped forwarding packets and lost all connectivity.

They have probably changed to VTP transparent mode and so they are not accepting following VTP updates.

If you later have deleted a Vlan in VTP servers, that Vlan can still exist in the now transparent switches and they have become root bridge for that "appended" Vlan.

- They have an even lower limit for max number of STP instances in PVST.

The two numbers can be like 128 Vlans in VTP and 64 STP instances.

So the recommendation is to manually configure the list of allowed Vlans to/from the devices with

switchport trunk allowed vlan 10,20,30

to permit only the Vlans really needed on the device so that it limits the number of STP instances running on it

(VTP pruning does not reduce the number of STP instances running on a device)

Who is the root bridge for the affected Vlan?

Is the C2950 the root bridge for that Vlan ?

use

show spanning-tree vlan X

to check on both C6500  and C2950

Hope to help

Giuseppe

[katerina.dardoufa]

Hi Giuseppe,

the switches are configured as VTP clients and already have allowed vlans on their trunk links. We faced the issue on two different 2950 switches. These switches were connected via redundant links to Gi1/1 of 6509.

The first switch is located in site A and connects to 6509-A1 and 6509-A2. The second switch is located in site B and connects to 6509-B1 and 6509-B2.

Vlan deletion on site A caused the behavior already described on the previous post for switch-A and vlan deletion on site B caused the behavior for switch-B. No other 2950 switches were affected in either case. The switches started functioning as normal after some period of time (more than STP timers!!! About 15 minutes or so).

So my understanding is that there is something going on with Gi1/1 connecting to old switches (???).

My main concern is the meaning of the message:

strata_remove_vtable_entry : vlan entry not found

Thank you in advance,

Katerina

[Giuseppe Larosa]

Hello Katerina,

you are facing a really strange issue.

However, there are not enough elements to point to the gi1/1 on the catalyst 6500 devices.

>>the switches started functioning as normal after some period of time (more than STP timers!!! About 15 minutes or so).

In your first post you have written you have lost connectivity with the affected C2950.

Now you have added the following:

- the issue started when you deleted a Vlan in your VTP servers

. you have other C2950 switches that were not affected by the change.

I would look at the affected switches and unaffected switches C2950 to find out if they are running different IOS images. The limitations for max number of Vlans in VTP and max number of STP instances can be different on different IOS images with different feature set, if I correctly remember.

Can you check that the affected C2950 are acting as VTP clients with

show vtp status.

A time of 15 minutes is more related to VTP activity then STP.

https://www.cisco.com/c/en/us/support/docs/lan-switching/vtp/10558-21.html

VTP summary advertisements are sent every 5 minutes.

>>

My main concern is the meaning of the message:

strata_remove_vtable_entry : vlan entry not found

I suppose you see this log message on the affected C2950 switches. I couldn't find an exact match. However, the message looks like to say that the device received a VTP update telling it to delete a Vlan, but that vlan couldn't be found on the local switch vlan database.

If the switches do not accept the first VTP update for any reason, but they are still VTP client, they will process the next messages from VTP server sent every 5 minutes and can detect an increased revision number.

Unfortunately, we have no details about what the switches did in those 15 minutes.

If you can post the log messages from the affected switch that are in that time window you can get better help.

Hope to help

Giuseppe

[katerina.dardoufa]

The affected switches run IOS c2950-i6q4l2-mz.121-22.EA1.bin, as do most of the 2950 still in our environment. They are VTP clients with maximum supported vlans 250. The number of existing vlans is 54.

I logged in to the switch in site A with console while it had lost connectivity and I noticed that it had no root port!!!! Ports were up.

For the switch in site B the logs are totally different!

Site B:

strata_remove_vtable_entry : vlan entry not found
strata_remove_vtable_entry : vlan entry not found
strata_remove_vtable_entry : vlan entry not found
strata_remove_vtable_entry : vlan entry not found
May 23 15:34:17.854: %SPANTREE-2-LOOPGUARD_BLOCK: Loop guard blocking port GigabitEthernet0/1 on VLAN0051.
May 23 15:34:59.074: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to down
May 23 15:35:00.082: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to down
May 23 15:38:33.170: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to up
May 23 15:38:35.206: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to up
May 23 15:38:38.006: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to down
May 23 15:38:39.014: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to down
May 23 15:39:34.118: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to up
May 23 15:39:36.154: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to up
May 23 15:40:07.606: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to down
May 23 15:40:08.618: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to down
May 23 15:40:11.274: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to up
May 23 15:40:13.306: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to up
May 23 15:42:33.338: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to down
May 23 15:42:34.350: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to down
May 23 15:44:17.422: %LINK-3-UPDOWN: Interface GigabitEthernet0/1, changed state to up
May 23 15:44:19.454: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/1, changed state to up

As can be seen from the logs a loop is created for vlan 51, but that brings the whole Gi0/1 (connection to 6509) down! The connection is up after 5 minutes, which could be the VTP update as you stated, but even then the connectivity is not restored. From the 6509 we flapped the link at 15:42 and connectivity was restored briefly after that.

The behavior is indeed weird. What surprised us it we had the same impact in two different sites with 2950 connected on ports Gi1/1 on different 6509, even though the logs and the output of show commands on the two switches is totally different. I blame old hardware!!!!

I don't think there is much more to use for investigation. I was just wondering if anyone had seen such behavior before, and what the meaning of the log message was (I could not much it to any thing I knew).

Thanks!

[Giuseppe Larosa]

Hello Katerina,

>> The affected switches run IOS c2950-i6q4l2-mz.121-22.EA1.bin, as do most of the 2950 still in our environment. They are VTP clients with maximum supported vlans 250. The number of existing vlans is 54.

It is difficult to say something more.

About the different reactions of  the two switches:

For affected switch C2950 in site A, it is probably missing the configuration of spanning-tree loop guard on its uplinks. So just after the Vlan is deleted on the VTP server, the C6500 stops to send STP BPDUs for the just deleted Vlan. the affected switch in site A still has its STP instance running and becomes root bridge for that Vlan.

From the logs of affected swich in Site B:

>> strata_remove_vtable_entry : vlan entry not found
May 23 15:34:17.854: %SPANTREE-2-LOOPGUARD_BLOCK: Loop guard blocking port GigabitEthernet0/1 on VLAN0051.

here, spanning-tree loop guard is configured and after the Vlan 51 is deleted on the VTP server, the C6500 port stops to send out STP BPDUs for that Vlan and loop guard is triggered on the access switch.

>> As can be seen from the logs a loop is created for vlan 51, but that brings the whole Gi0/1 (connection to 6509) down! The connection is up after 5 minutes, which could be the VTP update as you stated, but even then the connectivity is not restored. From the 6509 we flapped the link at 15:42 and connectivity was restored briefly after that.

I agree that the behaviour is strange.

Hope to help

Giuseppe