cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4228
Views
0
Helpful
7
Replies

How to recover from Vcenter Failure with Nexus 1000

patelvc7601
Level 1
Level 1

I was doing testing for vplex (EMC) on non prod Enviorment, I lost Vcenter Database. I did not have back up for vcenter. I was abel to install vcenter but I could not add the Host to DVS. On Nexus 1000 sh module shows all the host . What I am gathering is that Each ESX host have copy of the database ( /etc/vwmare/dvsdatab.db) & which includes uuid for switch (nexus 1000).

My host show up the old uuid so I tried to set the old uuid by following command :

N1K-VPLEX(config-svs-conn)# no vmware dvs
This will remove the DVS from the vCenter Server and any associated port-groups. Do you really want to proceed(yes/no)? [yes] yes

N1K-VPLEX(config-svs-conn)# vmware dvs uuid "60 be 37 50 8f 76 69 52-b3 f2 1d 00 2c 5c 8b 85" datacenter-name vPLEX
N1K-VPLEX(config-svs-conn)# END

Look at the result ???


N1K-VPLEX# sh svs con

connection VC_VPLEX:
    ip address: 10.x.x.5
    remote port: 80
    protocol: vmware-vim https
    certificate: default
    datacenter name: vPLEX
    DVS uuid: b0 30 01 50 c5 69 29 a6-25 54 ac 5d a2 92 0e c5
    config status: Enabled
    operational status: Connected
    sync status: in progress
    version: VMware vCenter Server 4.1.0 build-258902

Any help appreciated.

Sincerely,

Viral

7 Replies 7

sachthak
Level 1
Level 1

Hi Viral,

Each Host/VEM can only be part of one DVS. One thing that might have happened is that the VEM still has the old VC dataset residing on it.

I'd take a look to see if that is the case.

This faq page has some additional data on checking this:

https://communities.cisco.com/docs/DOC-14464#stale-vem

Thanks,

Sachin

Thank you Sachin.

I can see the stale data on VEM Pointint to Old Vcenter ; I do not understand the relationship ( how this data/database created).

what I


This is non prod enviornment so this should be fine but in Production enviornment this will be disaster ( Ideally I should have back up but I am not sure the vcenter database back up will solve the issue or not ). I tried to remove that dvswitch & got ioctl failed error so I will end up removing VEM & installing again.

I understand nexus 1000 extension key but never think about this dvs switch uuid ? I did not have nexus 1000 crash .

sbin # net-dvs -d -n "60 be 37 50 8f 76 69 52-b3 f2 1d 00 2c 5c 8b 85"
ioctl failed: 16
Operation failed: status = 0xbad000

Sincerely,

Viral

Hi Viral,

This data is created when the host is added to the DVS. When the host was added to the old DVS, it received the approrpiate VC data.

If the database backup was done, I believe it should solve this issues since it would match the IDs from the VC database and would avoid having to give a "new" dvs ID to the object.

I'll touch base with a few folks and loop back around with additional details.

Thanks,

Sachin

Ok ,

Let me understand this correctly, When I add Host to DVS/Nexus1000 It creates the local database cache in respective ESX/i /etc/vmware/dvsdata.db . I think this database is used in vcenter down scenario . when I bring back up the new vcenter & add dvs/nexus 1000 to it,I am assuming it's giving new UUID to dvs switch but all the ESX host have cached entry for old uuid in their database, so when I tried to add ESX into Vcenter DVS/nexus1000 , it gives me this error.

I am thinking you should be able to change UUID for DVS in vcenter mob.If not, than I will end up uninstalling & reinstalling VEM on each ESX host ( in my case it's few ,so not worry).

Thanks,

Viral

HI

Just read this post while researching the steps to building a brand new VC.

Basiclly our VC is totally stuffed and the VC DB is totally stuffed as well so need to do a new clean build of a VC. Up until now it seems to be straight forward, then I read this post and now it is more complicated.

If I understand it correctly I am not going to be able to add the Host back to the new N1K DVS once added to the new VC without reinstalling the VEMs on each on the ESXi 4.0 host we have.

So issue is that when the VEW is deinstalled then reinstalled any Guest servers running on the Host will be off the network......

Only way I can think of doing this currently is to migrate the Hosts and DVS off the old VC to a temporary VC ( I have to reuse the old VC server as the new one), then rebuild the faulty VC and then migrate the DVS and Hosts back to the Newly built VC. When I say migrate the DVS, I would Build a new N1K switch, so would have 1 N1K connected to the old VC and a new N1K connected to the tempoary VC, then move Guest of a host, discconnect Host from old VC DVS then connect it to the DVS on the Temp VC, then move Guests to the moved Hosts. after rebuild do the same thing but in reverse.

Hope that makes sence, any ideas comments other solution welcomed.......

Cheers

Tim

Hi Tim,

If I understood your proposed process correctly, this won't work unfortunately. The limitation is that a given host can only be added to a single N1k DVS.

The best solution would be to recover from a saved backup of the VC database if available.

One additional thing is that moving between VCs doesn't require a reinstall of the VEM. It would however require that the VEM is "detached" from other DVS entities. This can be checked by doing a "net-dvs -l" on the host.

Basically, removing the host from the DVS on VC1 and adding it to the DVS on VC2 would do the trick. (Assuming the N1k versions are the same on both VCs).

Thanks,

Sachin

Hi Sachin

Wish I did have a good DB backup to recover to, but unfortunatly I don't.

Yep what you advise is what I was thinking about doing, just did not explain it that well in my post.

Thanks for the info and help.

Tim

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: