Re: fresh install of APIC-EM 1.0 - grapevine_root is not ready

tomlammon · ‎11-11-2015

I just attempted to install APIC-EM 1.0 on a virtual machine with 64 GB RAM, 6 cpu cores, and 500GB disk as outlined in the requirements. The install went great until the very end when it barfed with this message:

The configuration wizard has encountered the following error:

Unable to start 'grapevine_root'. Please refer to the logs for mroe details...

after exiting the wizard, I see 20 lines of the following:

[retry=x] 'grapevine_root' is not ready. Sleeping...

has anyone else run in to this?

Tom

yawming · ‎11-12-2015

Are you still able to login to get the log ?

/var/log/grapevine/grapevine_root.log

Yawming

tomlammon · ‎11-12-2015

Yes, here's the last 50 lines:

$ tail -50 grapevine_root.log

| 2015-11-13 02:47:28,436 | DEBUG | 30583 | MainThread | 139969488332608 | grapevine.root.dao.RootDao.RootDao | RootDao.py:78 | Adding root record: {'public_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDM+2xZJgb4K+1QUmva48Hb+xjOre3drr6q4k+5kmU/8Kd9xiYx+q/36JSPOh55lOCFAOttrzLAY2+OeJ9mjSL1PAI8/1x5Dhb+jch3I5mysMIfadA9oHxCxBpWOftfcaulcMeslaZAqNM7QRuvKmQDqb80nxrRgmMPCQVdGJU5L0F6CUp5/shPejDTezeq9obtkIAt7EJT5LoHo6y28VmiJJmfj9A1h9Zn6zt19PMIrRfL6zwooMc0l0Q0sAmq0j63ZNcqXqmJCU81vm4Ox2osQ1ClMNLPIcro7dRU6GRMvPx8sWI7SgyNEbV9Fz7XQEMGGlwK1+SRW2SYMkLRvuWp 4866c95a-5d07-4bb2-95ea-59a664b2528e', 'root_id': u'6c9459bd-019f-4c5f-8438-3c141fa1e94b', 'interfaces': '[{"interface": "eth0", "ip": "10.109.71.16", "mac": "00:0c:29:8f:06:08", "netmask": "255.255.255.0"}, {"interface": "grape-br0", "ip": "169.254.0.1", "mac": "7e:91:9b:20:c4:cd", "netmask": "255.255.255.0"}]', 'hostname': 'grapevine-root-1', 'root_version': '0.3.0.2024.dev626-gcad5408', 'root_index': 0} |

Traceback (most recent call last):

File "/opt/cisco/grapevine/local/lib/python2.7/site-packages/grapevine/root/Root.py", line 93, in main_func

return root.start()

File "/opt/cisco/grapevine/local/lib/python2.7/site-packages/grapevine/root/Root.py", line 36, in start

return self._context.start()

File "/opt/cisco/grapevine/local/lib/python2.7/site-packages/grapevine/root/GrapevineContext.py", line 643, in start

self._register_root()

File "/opt/cisco/grapevine/local/lib/python2.7/site-packages/grapevine/root/GrapevineContext.py", line 696, in _register_root

controller_config_dao.update_roots(list(root_dao.get_all_roots()))

File "/opt/cisco/grapevine/local/lib/python2.7/site-packages/grapevine/root/dao/RootDao.py", line 99, in get_all_roots

yield self._unmarshall_root_record(columns)

File "/opt/cisco/grapevine/local/lib/python2.7/site-packages/grapevine/root/dao/RootDao.py", line 163, in _unmarshall_root_record

root_record[RootDao.INTERFACES_KEY] = json.loads(root_record.get(RootDao.INTERFACES_KEY, []))

File "/usr/lib/python2.7/json/__init__.py", line 338, in loads

return _default_decoder.decode(s)

File "/usr/lib/python2.7/json/decoder.py", line 366, in decode

obj, end = self.raw_decode(s, idx=_w(s, 0).end())

TypeError: expected string or buffer

(grapevine)

yawming · ‎11-13-2015

Looks like need more info for debugging

Can you please login again. From terminal run "rca" command. It runs a root cause analysis script to creates a tar file.

The tar file will be in /tmp directory. Please share this tar file.

Thanks,

Yawming

tomlammon · ‎11-13-2015

Here's the link to the file:

https://www.dropbox.com/s/m5uexc1djivl9c5/grapevine-root-10.109.71.16-rca-2015-11-14_02-30-03-AM_UTC%2B0000.tar.gz?dl=0

Please let me know when you have downloaded it.

Thank you!

Tom

yawming · ‎11-13-2015

Thanks for sharing the log, downloaded.

tomlammon · ‎11-13-2015

I thought I would give it another try, so I ran the config wizard again and this time it worked...I didn't change anything, so I don't know why it didn't work the first time.

Thanks for your help.

Tom