LACP issue (Oracle Exadata VPC Active/Active concern/related)

[Tehransoccer]

Does anyone know why (Exadata hosts) VPC/LACP becomes "I" when one of Leaf switches gets recycled? It's NOT related to LACP time-out (TAC already verified it).

whenever one of the switches get recycled/disconnected (i.e.: LAB-LEAF8), Port-Channel stays as “Individual” -> NOT_IN_SYNC:

LAB-LEAF7# show lacp interface e1/1
Interface Ethernet1/1 is up
  Channel group is 6 port channel is Po6
  PDUs sent: 31465
  PDUs rcvd: 31344
  Markers sent: 0
  Markers rcvd: 0
  Marker response sent: 0
  Marker response rcvd: 0
  Unknown packets rcvd: 0
  Illegal packets rcvd: 0
Lag Id: [ [(7f9b, 00-23-04-ee-be-04, 82ae, 8000, 101), (0, 00-1b-21-8c-93-29, d, ff, 2)] ]
Operational as aggregated link since  Mon Apr 23 05:18:54 2018

Local Port: Eth1/1   MAC Address= 38-90-a5-fa-d6-69
  System Identifier=0x8000,38-90-a5-fa-d6-69
  Port Identifier=0x8000,0x101
  Operational key=33454
  LACP_Activity=active
  LACP_Timeout=Short Timeout (1s)
  Synchronization=IN_SYNC
  Collecting=true
  Distributing=true
  Partner information refresh timeout=Short Timeout (3s)
Actor Admin State=(Ac-1:To-1:Ag-1:Sy-1:Co-1:Di-1:De-0:Ex-0)
Actor Oper State=Ac-1:To-1:Ag-1:Sy-1:Co-1:Di-1:De-0:Ex-0
Neighbor: 0x2
  MAC Address= 00-1b-21-8c-93-29
  System Identifier=0x0,00-1b-21-8c-93-29
  Port Identifier=0xff,0x2
  Operational key=13
  LACP_Activity=active
  LACP_Timeout=Short Timeout (1s)
  Synchronization=IN_SYNC
  Collecting=true
  Distributing=true
Partner Admin State=(Ac-1:To-1:Ag-1:Sy-1:Co-1:Di-1:De-0:Ex-0)
Partner Oper State=Ac-1:To-1:Ag-1:Sy-1:Co-1:Di-1:De-0:Ex-0
Aggregate or Individual(True=1)= 1
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8#
LAB-LEAF8# show lacp interface e1/1
Interface Ethernet1/1 is suspended
  Channel group is 5 port channel is Po5
  PDUs sent: 365
  PDUs rcvd: 0
  Markers sent: 0
  Markers rcvd: 0
  Marker response sent: 0
  Marker response rcvd: 0
  Unknown packets rcvd: 0
  Illegal packets rcvd: 0
Lag Id: [ [(7f9b, 00-23-04-ee-be-04, 82ae, 8000, 101), (0, 0-0-0-0-0-0, 0, 0, 0)] ]
Operational as aggregated link since  Thu Jan 01 00:00:00 1970

Local Port: Eth1/1   MAC Address= 70-70-8b-8c-76-6f
  System Identifier=0x8000,70-70-8b-8c-76-6f
  Port Identifier=0x8000,0x101
  Operational key=33454
  LACP_Activity=active
  LACP_Timeout=Long Timeout (30s)
  Synchronization=NOT_IN_SYNC
  Collecting=false
  Distributing=false
  Partner information refresh timeout=Long Timeout (90s)
Actor Admin State=(Ac-1:To-0:Ag-1:Sy-0:Co-0:Di-0:De-1:Ex-0)
Actor Oper State=Ac-1:To-0:Ag-1:Sy-0:Co-0:Di-0:De-1:Ex-0
Neighbor: 0x0
  MAC Address= 0-0-0-0-0-0
  System Identifier=0x0,0x0
  Port Identifier=0x0,0x0
  Operational key=0
  LACP_Activity=unknown
  LACP_Timeout=Long Timeout (30s)
  Synchronization=NOT_IN_SYNC
  Collecting=false
  Distributing=false
Partner Admin State=(Ac-0:To-0:Ag-0:Sy-0:Co-0:Di-0:De-0:Ex-0)
Partner Oper State=(Ac-0:To-0:Ag-0:Sy-0:Co-0:Di-0:De-0:Ex-0)
Aggregate or Individual(True=1)= 2

[mfarrenkopf]

Hi Tehransoccer,

Did you ever get a resolution to this?

We've got some Oracle equipment connected to a vPC pair of Nexus 6001 switches, running old software (NX-OS 6.0(2)N2(5)).  One of them experienced a hardware failure.  We had some Oracle interfaces dropping and coming back every 5 minutes while one switch was down.  I don't know if my server engineer has opened a case with Oracle, but there's nothing in my logs that indicates where the flapping is coming from.  We replaced the bad switch, everything is stable, LACP shows in sync on both switches as I would expect.  Also, the Oracle side shows two interfaces flapped (physical link) that I show stayed up the entire time.

So if you're still monitoring this question, I'm interested in what you may have come up with.  Thank you!