Installation and Deployment

If you are patching boxes manually new VMware kernel modules usually require a VEM patch. These patches are available for download from VMware.

https://www.vmware.com/mysupport/download/

Choose VEM in the pull down

Q: My ESX hosts are not showing up as Modules on the Nexus 1000V?

n1kv-beta2# show module
Mod Ports Module-Type Model Status
--- ----- -------------------------------- ------------------ ------------
1 1 Virtual Supervisor Module Nexus1000V active *

Mod Sw Hw World-Wide-Name(s) (WWN)
--- -------------- ------ --------------------------------------------------
1 4.0(1a)S1(0.82 0.0 --

Mod MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
1 00-19-07-6c-5a-a8 to 00-19-07-6c-62-a8 NA

Mod Server-IP Server-UUID Server-Name
--- --------------- ------------------------------------ --------------------
1 172.18.217.242

Usually the reason VEM modules do not show up under "show module" is that the Control VLAN is not accessible to both the VSM and VEM.

Verify the following :

1. Make sure your physical switch ports are correctly part of the Control VLAN on both the VSM and VEM

2. Make sure your Control VLAN is specified as a "system vlan" in the "uplink" port-profile
3. On the physical switches where the VSM and VEM are connected to verify that the MAC addresses of both the VSM and VEM are present in the Control VLAN

       Use "show mac-address dynamic vlan your-control-vlan-id"

4. Please follow the steps outlined in "Troubleshooting VSM-VEM connectivity issues"
5. Check the log on the VSM to see if the VSM is even attempting to add the VEM
        Use "show logging last 20" to see the last 20 entries in the log
        Below are the messages you should see when VSM and VEM can communicate

n1kv-beta2# show logging last 5
2008 Dec 11 12:17:32 n1kv-beta2 %VMS-5-CONN_CONNECT: Connection established to the VC.
2008 Dec 11 13:23:40 n1kv-beta2 %PLATFORM-5-MOD_DETECT: Module 3 detected (Serial number ) Module-Type
Virtual Ethernet Module Model
2008 Dec 11 13:23:40 n1kv-beta2 %PLATFORM-2-MOD_PWRUP: Module 3 powered up (Serial number )
2008 Dec 11 13:23:40 n1kv-beta2 %PLATFORM-5-MOD_STATUS: Module 3 current-status is MOD_STATUS_POWERED_U
P
2008 Dec 11 13:23:46 n1kv-beta2 %PLATFORM-5-MOD_STATUS: Module 3 current-status is MOD_STATUS_ONLINE/OK
2008 Dec 11 13:23:46 n1kv-beta2 %MODULE-5-MOD_OK: Module 3 is online (serial: )
2008 Dec 11 13:23:46 n1kv-beta2 %VIM-5-IF_ATTACHED: Interface Ethernet3/2 is attached to vmnic1 on rtp9
-cae-svs-2.cisco.com
2008 Dec 11 13:27:04 n1kv-beta2 %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configuring console from /dev/pts/0 /dev/
pts/0_64.102.128.128

Q: When I try to connect the VSM to VMware Virtual Center it fails
A: You may get an error saying it's trying to connect and it never does or you may get an error saying it failed

Verify that you added the Cisco Nexus 1000V extension to VMware Virtual Center.
This entails downloading an xml file from the Nexus 1000V via http and loading it as a plugin.
This is all covered in the Install guide under the "Setting up the VC Connection"

Q: Does the Cisco Nexus 1000V require a 64-bit VM?
A: Yes the Nexus 1000V requires a 64 bit VM. When building the VM choose Linux Other-64 bit.

Q: VEM modules get added but then disappear from the VSM. What might be the problem?

A: Check for loopguard on the upstream Catalyst switch

show run |include loopguard

show spanning-tree vlan "control-vlanid"

and look for ports with LOOP_inc and ports disabled.

disable loopguard or set the individual switch ports for the VSM and VEM with

spanning-tree bpdufilter enabled
spanning-tree portfast

Q: What is the mandatory spanning tree configuration on the switches where the VEM is connected to?

A: On the upstream switch(s) it is considered MANDATORY to enable below settings on the switchports the VEM is connected to::

For IOS
cat65k-1(config-if)# spanning-tree portfast trunk

For NXOS
n5k-1(config-if)# spanning-tree port type edge trunk

Q: What is the mandatory BPDU configuration on the switches where the VEM is connected?

A: on the upstream switches that the VEM is connected to it is highly recommneded to that Global BPDU Filtering and BPDU Guard be enabled.

For IOS
cat65k-1(config)# spanning-tree portfast bpdufilter
cat65k-1(config)# spanning-tree portfast bpduguard

For NXOS
n5k-1(config)# spanning-tree port type edge bpduguard default
n5k-1(config)# spanning-tree port type edge bpdufilter default

In environments where you can NOT use global modes set the following on the switchports the VEM are connected to

For IOS
cat65k-1(config-if)#spanning-tree bpdufilter
cat65k-1(config-if)#spannning-tree bpduguard

For NXOS
n5k-1(config-if)#spanning-tree bpdufilter
n5k-1(config-if)#spanning-tree bpduguard

Q: When I try to connect the VSM to vCenter I get "Extension Key was not registered before it's use"

This error means the key stored on the VSM is not matching the key that has been loaded into vCenter Server. You can check the VSM key with n1000v-testor# show vmware vc extension-key Extension ID: Cisco_Nexus_1000V_165241074 You can check the key you loaded into vCenter Server via the vSphere client Plug-in Manager

Q: What will cause a new XML extension file to get generated?

The following information is used to generate the XML xtension

A unique extension key ( auto-generated. Can be user supplied).
SubjectName of the certificate ( auto-generated. Can be user supplied )

The XML extension will be generated in the following cases.

Brand new VSM install.
change the certificate on the VSM. "install certificate" command in svs connection mode.
change the extension key using "vmware vc extension-key"

Q: How do I remove a VSM from vCenter?

A VSM can only be removed from VMware vCenter server using the "no vmware dvs " command under the svs connection. You cannot remove the DVS from within vCenter.

If you removed or destroyed the VSM without issueing the above command the old Nexus 1000V will remain in vCenter until you run the following steps.

You will need to reconnect the existing VSM or create a new VSM with the same extension key to VMware vCenter to remove it.

Create a new VSM either from ISO or OVF
We need to find the extension key tied to the old DVS and configure it on the new VSM. To find the extension key tied to a DVS follow the information in the link - http://www.cisco.com/en/US/docs/switches/datacenter/nexus1000/sw/4_0/troubleshooting/configuration/guide/trouble_3install.html#wp1206322
Once you find the extension key set it on the new VSM with the following command "n1000v(config)# vmware vc extension-key (key)
Unregister the related extension key on VMware vCenter with the instructions in the following link http://www.cisco.com/en/US/docs/switches/datacenter/nexus1000/sw/4_0/troubleshooting/configuration/guide/trouble_3install.html#wp1196708
Open a browser and browse to the VSM and download the extension.xml file. Make sure you refresh the browser page to get the current extension key.
Register the new extension.xml file in VMware vCenter
Make sure the switchname of the VSM is the same as the name of the DVS you are trying to delete
Create the svs connection to connect to the same datacenter.
Connect the VSM to vCenter with "connect" under svs connection
Once it connects to vCenter delete the DVS using the 'no vmware dvs" command under svs connection

Q: How to restore an old VSM from backup config file
A: Use the following steps to restore the old VSM and connecti it to VMware vCenter

1. Install VSM and setup basic configuration.

2. Login VSM and confirm current values which new VSM has.
          - host-id will be needed to get new license.
          # show license host-id
          - extension key will be needed to be change later.
          # show vmware vc extension-key

         n1kv# show license host-id
         License hostid: VDH=7593988201229247063
         n1kv#

         n1kv# show vmware vc extension-key
         Extension ID: Cisco_Nexus_1000V_1285300162
         n1kv#

3. copy a backuped config file to running-config
        n1kv# copy bootflash:startup-config running-config

4. Check current extension key which vCenter caches.
       https://vCenter IPaddress/mob
        content
            ExtensionManager
                extensionList["Cisco_Nexus_1000V_XXXXXXXXXX"]

        The number should be different from the number which you see step 2..
            Apply this original number to new VSM.
            Confirm the number change to original one with "show vmware vc extension-key" command.

            n1kv# vmware vc extension-key Cisco_Nexus_1000V_XXXXXXXXXX

            n1kv# show vmware vc extension-key
               Extension ID: Cisco_Nexus_1000V_XXXXXXXXXX

5. Unregister current plug-in from vCenter

         https://vCenter IPaddress/mob/?moid=ExtensionManager
           UnregisterExtension
              On "void UnregisterExtension" window,input "Cisco_Nexus_1000V_XXXXXXXXXX" in VALUE box.
                 Click [Invole Method.
                 You will see "Method Invocation Result: void"

6. Install a plugin to vCenter.
           http://VSM management IPaddress/
           save "cisco_nexus_1000v_extension.xml" at your local disk which can access to vCenter.

            On vCenter,click "plug-ins" Menu and select [Manage Plugins..].
             Another window will be opened and right-click there.
             Select [New Plugin..]
             (you may find the plugin which you have already unregistered in the previous
            step.Actually,the plug-in was deleted. you can proceed next step)

           On the other "Register Plug-in" window,Select the plugin which you save in your local disk.
           And just click [Register Plug-in].

           For a Security Warning,you can select "Ignore".
           "The Plug-in "Cisco_Nexus_1000v_XXXXXXXXXXXX" is successfully registered on
           vCenter Server IP Address" will be pop-up.

7. Connect VSM and vCenter and Check the connection status

          n1kv(config-svs-conn)# connect
          n1kv(config-svs-conn)#

          n1kv# show svs connection
             connection vc:
                  ip address: 10.xx.xx.xx
                  remote port: 80
                  protocol: vmware-vim http
                 certificate: default
                 datacenter name: nexus1000v
                 DVS uuid: 6e 80 20 50 4d 85 2f 79-e3 89 f0 f2 1b ac d7 3c
                 config status: Enabled
                 operational status: Connected
                 sync status: Complete
                 version: VMware vCenter Server 4.0.0 build-162856
                  n1kv#

8. Confirm that setting and config are restored with some commands

9.save the config
         n1kv# copy running-config startup-config
         [########################################] 100%
         n1kv#

Q: All my VSMs are generating the same extension key
A: This was a known issue with 1.3 OVF file that was generated when it was initially released. A new OVF file has been created and released on www.cisco.com. Download the new bundle and redeploy the VSM using the new OVF.

Virtual Ethernet Module (VEM)

Q: Which versions of ESX/ESXi does the 4.0(4)SV1(2) support?
A: The following builds of ESX/ESXi 4.0.0 are supported:

ESX/ESXi 164009
ESX/ESXi 175265
ESX/ESXi 181792
ESX/ESXi 193498
ESX/ESXi 208167
ESX/ESXi 219382

Q: If I’m using VUM, do I have to manually install the VEM package?
A: No, VUM will correctly install the right version of Nexus 1000V onto your host, including matching the correct versions of ESX/ESXi 4.0.0 which you are running.

Q: I’m seeing VUM show my baseline as incompatible. What should I do?
A: This incompatibility can be ignored. Please look at Vmware KB 1013068 for a more detailed description of this issue: http://kb.vmware.com/kb/1013068

Q: VUM is not remediating my hosts when I’m remediating against baselines I’ve created for builds 175265 or 181792. I’m using a proxy, could this be related?
A: There is a known issues with VUM and a proxy setup with the VSM. If your proxy can be configured to send requests to the VSM and out to the internet, this will solve your problem. This issue will be addressed with a new VUM release during the 4.0.0 Update1 release.

Q: VUM is not working with my host. I have datastores or VMs that are not accessible. Is this related?
A: Yes, VUM will fail remediating hosts if you have datastores that are not accessible. Reconnect the datastores or remove them to get VUM to proceed. If you have VMs that are inaccessible, make them accessible or remove them to get VUM to proceed.

Q: How do I remove the VEM Module from the ESX host?
A: Login to the ESX box either via ssh or console access

[root@rtp9-cae-svs-1 sbin]# ./vem-remove -s -d
watchdog-vemdpa: Terminating watchdog with PID 9892
Removing CIsco VEM VIB from COS system
[root@rtp9-cae-svs-1 sbin]#

Q: I get "Cannot complete a distribted virtual switch operation for one ore more host members" when trying to add an ESX host to the Nexus 1000v?
A: This message can occur in two situations

The first situation is if you are using VUM to install the VEM module. If you get this error it means VUM is either not installed or cannot find the correct VEM version to load on the ESX host. Verify VUM is installed, verify you have VMware VEM repository download location required via this KB article http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1013134.

With the inital 4.0 VUM code there was a bug if a VUM was configured to use a proxy. VUM would force all communication to be through the proxy and this would cause communication issues when VUM needed to talk to the VSM. This was fixed in the 4.0u1 VUM version. If you have a proxy configured and are unsure of the VUM version, you can try removing the proxy to see if the ESX host can then add to the Nexus 1000V.

The second situation is if you are manually installtng the VEM module. If you have manually installed the VEM module and you get the above error verify that the VEM module you loaded is correct for the ESX kernel version you are using. You can check the compatibility guide on the Cisco website at http://www.cisco.com/en/US/products/ps9902/products_device_support_tables_list.html

To verify the ESX kernel version the most reliable method is to

[root@cae-esx-179 ~]# cat /proc/vmware/version | grep kernel
vmkernel build: 208167, vmkcall: 45.0 driver interface: 9.0 kernel: 0.4096
vmkernelID: 0xb3d912ac
vmkernel Version Releasebuild-208167
[root@cae-esx-179 ~]#

Q: How do I clean out stale data on the VEM?
A: First why would you have stale data. One common scenario is when you take an ESX host that was part of existing Cisco Nexus 1000V and attempt to join it to a new 1000V environment without first removing it from the old environment. The old DVS information will persist on the ESX host and prevent from being added to the new 1000V until it is cleaned out.

First verify that you have stale data by running esxcfg-vswitch -l on the ESX host. If you see an existing Cisco Nexus 1000V DVS then it will need to be cleaned out.

[root@cae-esx-179 ~]# esxcfg-vswitch -l | more
.
DVS Name       Num Ports   Used Ports Configured Ports Uplinks
n1000v-MV      256         52          256               vmnic0

   DVPort ID           In Use      Client
   608                 1           vmnic0

Above you can see that an existing Nexus 1000V still exists.
The first step is to remove vmnic0 from the old DVS

[root@cae-esx-179 ~]# esxcfg-vswitch -Q vmnic0 -V 608

Next unload the VEM kernel module

[root@cae-esx-179 ~]# /usr/lib/ext/cisco/nexus/vem-v110/sbin/hotswap.sh -u

Get the VMware DVS switch id with net-dvs.

[root@cae-esx-179 ~]# /usr/lib/vmware/bin/net-dvs -l | grep cisco_nexus
switch 5b 4b 01 50 fd ae c8 79-05 1a 33 39 49 5c 5d 8b (cisco_nexus_1000v)

Remove the DVS entry using the switch id from above with the net-dvs command

[root@cae-esx-179 ~]# /usr/lib/vmware/bin/net-dvs -d -n "5b 4b 01 50 fd ae c8 79-05 1a 33 39 49 5c 5d 8b"

If the above command fails with a "bad ioctl" error then remove the VEM module completely with "vem-remove -s -r" and rerun the "net-dvs -d -n" command above

have VMs that are inaccessible, make them accessible or remove them to get VUM to proceed.

Q: How do I reserve a VEM slot number for an ESX host?
A: You need to know the UUID of the ESX host. Once you know the UUID you can reserve the slot number on the VSM with the following commands.

First get the vmware host id from the ESX server.
Via CLI with the
Login to the ESX box either via ssh or console access

[root@cae-esx-180 sbin]# ./esxcfg-info -u
33393138-3335-5553-4537-30354E375832

The host id is that long string.

Next run the following command on the VSM to make this ESX host end up as VEM module number 12
n1000v-MV# config t
n1000v-MV(config)# vem 12
n1000v-MV(config-vem-slot)# host vmware id 33393138-3335-5553-4537-30354E375832

When the ESX host with the UUID above is added to the N1KV it will come in as VEM module number 12

Upgrade

Q: Is 4.0(4)SV1(2) backward compatible with 4.0(4)SV1(1)?
A: Yes, users can upgrade their 4.0(4)SV1(1) VSMs and retain the existing configuration.

Q: Can I use VEMs running 4.0(4)SV1(1) with a VSM running 4.0(4)SV1(2)?
A: Yes, a 4.0(4)SV1(2) VSM can run in backward compatible mode and control VEM running 4.0(4)SV1(1). By default, after an upgrade, the VSM will be left in rolling-upgrade mode and it needs to be placed into normal upgrade mode (using the configuration command “upgrade mode normal”) to run in backward compatible mode.

Q: Can I use a 4.0(4)SV1(2) VSM to control a mixed set of 4.0(4)SV1(2) and 4.0(4)SV1(1) VEMs?
A: Yes, but only features supported by the 4.0(4)SV1(1) modules can be enabled in a mixed VEM environment.

Q: How can run my 4.0(4)SV1(2) VSM with 4.0(4)SV1(1) VEMs in backward compatible mode?
A: After the VSM has been upgraded to 4.0(4)SV1(2), enable normal upgrade mode using the configuration command “upgrade mode normal”.

Q: After I have upgraded my VSM to 4.0(4)SV1(2) I cannot enable new features introduced in this release.
A: New features introduced in 4.0(4)SV1(2) can only be enabled when all the VEM are running 4.0(4)SV1(2) as well. That is, the feature level is 4.0(4)SV1(2).

Q: What is the feature level? How/When can I change it?
A: The feature level is the version number that defines which features can be enabled in the switch. By default, after a switch has been upgraded, the feature level will remain at the existing version number before the upgrade (4.0(4)SV1(1)). The feature level can be updated to 4.0(4)SV1(2) only after all the connected VEM have been upgraded to 4.0(4)SV1(2).

Q: Does the feature level get automatically updated after I have upgraded all my VEM to 4.0(4)SV1(2)?
A: No, the feature level must be upgraded by the user using the “system update vem feature level” exec command.

Q: Is the feature level persistent across VSM reloads?
A: Yes

Q: Can I downgrade the feature level from 4.0(4)SV1(2) to 4.0(4)SV1(1)?
A: No, once the feature level has been moved up to 4.0(4)SV1(2), it cannot be reversed back to 4.0(4)SV1(1). At that point it is possible that some VEM in the system have already enabled some 4.0(4)SV1(2) features and reverting them back to 4.0(4)SV1(1) is not a supported operation.

Q: I have a dual VSM setup running 4.0(4)SV1(1). How should I proceed to upgrade it to 4.0(4)SV1(2)?
A: The upgraded should occur while having the primary and secondary VSM in HA mode. This allows for the proper synchronization between them.

Q: I have a dual VSM setup but I upgraded only the active VSM while the standby was down. How can upgrade the other VSM?
A: When only one of the VSMs has been upgraded the other must be upgrade in isolation or reinstalled.

Q: Can I downgrade my VSM from 4.0(4)SV1(2) to 4.0(4)SV1(1)?
A: No. VSM software downgrades are not supported in this release. Reverting back to version 4.0(4)SV1(1) is a disruptive procedure that requires backing up the configuration to external storage and reinstall of the VSM.

Q: Can I downgrade my VEM from 4.0(4)SV1(2) to 4.0(4)SV1(1)?
A: No. VEM software downgrades are not supported in this release.

Q: I'm using Jumbo MTUs is there anyting I need to do during upgrade?
A: If you are using non-default (Jumbo) MTU settings be aware of the following

Symtpom: After rebooting an ESX server the MTU settings revert to default (1500) for VMNICs (pNICs) attached to the Cisco Nexus 1000V resutlting in loss of traffic and connectivity. This can cause an issue for network based storage that is used for datastores and use jumbo MTU.

Solution: Verify MTU settings on ESX with esxcfg-nics -l command. Check for MTU settings for the corresponding pNIC. For system uplink ports, set the "system mtu" command in the system port-profile applied to the port. This ensures that MTU is preserved for uplink interfaces with system VLANS. For non-system uplinks the VSM sets the correct MTU based on the interface configuration.

Q:How do I get vCenter to display the right VSM version after upgrade?
A: That data loaded into vCenter under the summary tab only displays the information which was defined for the VSM VM as part of the .ovf template and does not depend on the version of the VSM running inside the VM. It is just like the labels we put on the servers for identification.

You can update this info using the following information
Select the VSM VM->Edit Settings->Options tab->(under) vApp Options->Advanced ->Make the changes for the "Product" and the "Version" fields.

Nexus 1000V Installer Application

Q: What is the Nexus 1000v Installer Application?
A: The Nexus 1000V Installer Application helps complete the Nexus 1000V initial VSM configuration through a wizard-style user interface. In this installer, the VSM's management IP, gateway , subnet mask, admin password, & port groups will be configured. Additionally, the Installer Application will create a svs connection to a data center within the VC after registering the VSM's extension key.

Q: How can I access Nexus 1000v Installer Application?
A: The Nexus 1000V Installer Application runs as a Java applet on the VSM's web server. After deploying the Nexus 1000V OVA image and choosing the Installer deployment option, the application can be accessed through a browser at the management IP allocated during installation. Please note, the Management Port group, the management IP address, gateway and subnet mask must be correctly set during OVA deployment to allow the VSM's web server to remain accessible.

Q: What
A: The Nexus 1000V Installer Application currently supports Internet Explorer, Mozilla Firefox and Google Chrome.

Q: What JRE versions are supported?
A: Currently, the Installer Application requires JRE 1.6 to run.

Q: Do I need to deploy the OVA before using the Installer application?
A: Yes, the Nexus 1000V OVA image must be deployed and setup with the Installer deployment option to allow use of the Installer Application.

Q: Does the Installer Application set the control/packet vlans in the upsteam switch?
A: No, all vlan configurations in the upstream switch must be configured through normal means. The Installer Application only manages the port group assignment and creation using these vlans.

Q: Does the Installer Application create the DVS?
A: Yes, the Installer Application will use the provided information to create a DVS which will boot when the VSM is started.

Q: Does the Installer Application install the VEM also?
A: No, currently the Installer Application only configures the VSM.

Q: Can I manually configure the VSM and skip the Installer Application?
A: Yes, when deploying the Nexus 1000V OVA image, please choose the Manual deployment option when prompted to configure the VSM manually.

Q: What kind of VC Crendentials do I need to use the Installer Application?
A: To run the Installer Application, the user will need to have VC credentials allowing them to edit the VM host's Network Configurations as well as VM Administrative privileges (power on VM, power off VM, reconfigure VM).

Q: Will the Installer Application restart my VSM VM?
A: Yes, to complete specific operations, the Installer Application will need to restart the VSM VM two times during the configuration process.

Q: If the Installer Application fails to complete successfully, are changes to the VM rolled back?
A: No, currently, any changes made after the Installer Application begins processing are not rolled back in failure cases.

Q: Can I name the new port group I chose to create through the Installer Application?
A: No, currently, default names are given to any new port groups created through the Installer Application. These port groups are named "Control-auto", "Management-auto", and "Packet-auto" for the Control, Management, and Packet port groups respectively.

Q: Can I make changes to the information I entered in the Installer Application prior to clicking finish?
A: Yes, the Installer Application will make no changes to your VC or VM until finish is clicked. Changes can be made by going back through the wizard and modifying the entered values.

Q: Can I use the Installer Application to configure port groups over multiple vswitches?
A: No, currently, the Nexus 1000V Installer Application only supports configuring the Control, Management, and Packet port groups on a common vswitch.

Licensing

Q: Will default licenses be available when I upgrade to 4.0(4)SV1(2) ?
A: Yes, licenses for 16 CPU sockets will be available by default.

Q: Are the default licenses available on a permanent basis ?
A: No, these are meant for evaluation only and will be available for a period of 60 days since the upgrade to 4.0(4)SV1(2).

Q: How will the default evaluation licenses be used ?
A: Default evaluation licenses will be used only if there are no permanent licenses installed.

Q: Is the expiry of the default evaluation licenses based on usage ?
A: No. The expiration date of the default evaluation licenses is absolute and fixed.

Q: Can I continue to use any existing evaluation licenses installed before upgrade ?
A: No, the evaluation licenses installed before upgrade will no longer be valid and will not be used once you upgrade to 4.0(4)SV1(2).

Q: Can I install a new evaluation license on a VSM running 4.0(4)SV1(2) ?
A: No. Since evaluation licenses are available by default, new evaluation licenses cannot be installed.

Q: Do we need to uninstall any eval license before installing a permanent one?
A: Yes if you are running 4.0(4)SV1(1) code
No if you are running 4.0(4)SV1(2) code

Q: What are overdraft licenses?
A: Overdraft licenses are a built-in mechanism in 1000V license infrastructure, to help customers plan better with their licensing needs and during server migrations.. It is not a guaranteed functionality and limits may change over time, hence customers are strongly advised not to use overdraft for their permanent license needs.

n1000v-MV# show license usage NeXUS1000V_LAN_SERVICES_PKG
----------------------------------------
Feature Usage Info
----------------------------------------
        Installed Licenses : 9
     Default Eval Licenses : 0
    Max Overdraft Licenses : 16
Installed Licenses in Use : 2
Overdraft Licenses in Use : 0
   Default Eval Lic in Use : 0
    Default Eval days left : 0
        Licenses Available : 23
           Shortest Expiry : Never

Virtual Service Domain (VSD)

Q: What is VSD ?
A: VSD is logical domain that groups set of ports on a single host. Any typical VSD configuration contains SVM Ports and VSD Member Ports.Any traffic destined to these member ports will be processed by Service VMs .Similarly any traffic that is sourced by these member ports will be processed by Service VMs first before forwarding to the original destination.

Service VM(SVM) Ports: Inside Port, Outside Port and Management Port

Inside Port: Any traffic that is destined to a particular VSD, will be redirected to the SVM through inside port. After the traffic is processed by SVM, traffic will be forwarded to VSD through outside port. If SVM is Vshield, this port is same as "Protected Mode (P0)" port of Vshield Instance.
Outside Port: Similarly, Any traffic that is originated from a particular VSD will redirected reaches the SVM through outside port, After the traffic is processed by Service VM, traffic will be forwarded to destinations through outside port. If SVM is Vshield, this port is same as "Unprotected Mode (U0)" port of Vshield Instance.
Management Port: This port is used by SVM instance to talk the SVM manager, which controls multiple SVMs that are spread across the cluster/Data Center.
Member Ports: These are group of ports on which any traffic destined/sourced to/from these ports are serviced by SVM.

Q: What is Service VM (SVM)?
A: Any VM that provides functionality such as Firewall, Monitoring capabilities. It should have separate ports for incoming and outgoing traffic. Service VM should have its own instance on each host, where the packets need to be serviced. Nexus1000V fully supports VMware's Vshield as Service VM.

Q: What should I check before powering ON the SVM Vshield Instance ?
A: Make sure that Management, Inside, Outside vmnics on Vshield Instances are configured with appropriate port-profiles . Powering ON Vshield Instance without attaching the right port-profile might lead to flooding in the network. Any change in the port-profile binding on Vshield instances requires powering OFF the Vshield Instance.

Q: How is Vshield Configured on Nexus1000V ?
A: Every Vshield Instance will have three ports 1. Management Port, 2. Inside Port , 3. Outside Port

Define the Port Profile with Service Port capability "inside" , say Inside_Profile
Define the Port Profile with Service Port capability "outside", say Outside_Profile
First VMNIC of Vshield Instance is always used to talk to Vshield Manager (should be L3 connected, i.e Vshield mgr and Vshield Instance management port should have IP addresses). This VMNIC can be in Vswitch or on N1K DVS. If it is on N1K DVS, make sure the right profile is appended to this VMNIC that provides the connectivity to Vshield Manager. For details on Configuring Vshield Manager and Management port of Vshield , follow the VMware Vshield documentation given in the link below.
Second VMNIC must always be bounded to inside port,i.e attach Inside_Profile to this VMNIC.
Third VMNIC must always be bounded to outside port, attach Outside_Profile to this VMNIC.
Power ON the Vshield Instance of the host.
Tag all the member port profiles with appropriate VSD through configuration commands.

A quick video demo on configuring Vshield Instance to Nexus1000V will be available soon.

Q: What is the significance of "default-action <drop| forward>" in the SVM Port Profile?
A: When the SVM of VSD is unreachable, the N1K switch with take the this attribute configuration into consideration,and will either forward or drop the traffic that is destined/sourced to/from members of the VSD.

Q: Can the member interface belong to multiple VSDs at the same time?
A: No, any member interface can belong to only one VSD at a time.

Q: How many Vshield Instances per host are supported?
A: 8 Vshield Instances are supported per host. Every Vshield Instance;s Inside and Outside port on host should have unique port-profile bounded to it. Once the packet enters into the host, it can be serviced by at most two vshield Instances.

Q: Which traffic is not Serviced by Vshield Instance ?
A: Any CPU bound traffic such as AIPC, CDP,LACP, IGMP Control packets are not serviced (filtered) by Vshield Instance.
Intra VSD traffic i.e any traffic between the ports that are in the same VSD with in a host will not be serviced by Vshield Instance.

Q: Is Vmotion of Vshield supported ?
A: Vmotion of Vshield Instance is not supported, as Vshield Instance is host based and needs to be installed on every host that
needs to be processed by Vshield. However the application VMs that are serviced by Vshield Instance can be Vmotioned.
If the VEM which is hosting the Vmotioned VM does not have Vshield Instance, will end up dropping the packets to / from from this
VM after being Vmotioned.

Q: Are features like ACL,ERSPAN,QoS,PortSec,DHCP,PVLAN supported on SVM Ports?
A: SVM Ports are treated as regular promiscuous ports, feature configuration is not recommended on SVM ports. However all feature configurations are supported on member ports of VSD.

For detailed Information of VMware's Vshield

http://www.vmware.com/support/pubs/vsz_pubs.html

Port-Profile Questions

Q: Increase the Max number of ports per port-profile ?
A: The default number or ports per port-profile is currently 32. To change the Max number of ports use the following command

Changing the Max number of ports per port-profile is an online change and will not cause or require an outage.

Q: What is the impact of increasing the Max number of ports per port-profile higher ?
A: It depends :-)
Nexus 1000V currently has a max of 2k active veth interfaces. That number is the active number of ports. So in theory you could have 10 port-profiles all with 1024 Max ports per-profile and the N1KV would not care unless you tried to activate more then 2k ports. So you can oversubscribe from the N1KV side.

From the VMware side it's another issue. Ports per port-profile (active or not) apply to the total number of supported ports per DVS. That number is 8k for ESX 4.0 and 20k for ESX 4.1. So lets give an example.

I have 10 port-profiles with vCenter 4.0U1 and some number of ESX4.0U1 hosts. If I increase the default number of Max ports to 1024 for each of those port-profiles I will have a problem. That would be 10k ports (again it does not matter if they are active) and ESX 4.0U1 can only allow 8k per DVS.

What will happen if you have more ports per port-profile then VMware supports?
You'll get errors on the VSM when it tries to sync with vCenter. The end result is that it will run into errors when you try to add VMs and hosts to the Nexus 1000V.

Here is a link to a VMware document that helps describe the maximums.
http://www.vmware.com/pdf/vsphere4/r41/vsp_41_config_max.pdf

Q: When do I need to use a system vlan?
A: A system vlan should be used whenever you need to pass traffic through a VEM even when it cannot connect to the VSM

A VEM is essentially a virtual line card in a virtual chassis. The VSM is the Virtual Supervisor Module. When a VEM is inserted into a virtual chassis it has no programming and cannot pass traffic until the Supervisor Module programs it. In a real chassis this is not a problem, but in a virtual distributed chassis we need the line card to be able to communicate on the network to get it's programming from the VSM. So we end up with a chicken and the egg type scenario. The VEM needs to be programmed to pass traffic but it needs to be able to communicate on the network to get programmed.

The solution is system vlans. When you use the system vlan command on port-profile that port-profile now becomes a "system port-profile". This just means that we take the information about that port-profile and store it locally on the VEM as part of the opaque data. When the VEM boots it looks at it's local opaque data and any port-profile that is stored locally automatically starts passing traffic even if the VEM has not yet communicated with the VSM.

System vlans need to be set on egress and ingress ports. This just means that you need to set the system vlan on both the eth type and veth type ports. For example lets say I want to move the Service Console of the ESX host to the VEM. I always want the Service Console to be accessible even if the VSM and VEM cannot communicate. If my Service Console is on Vlan 2 I would need to set "system vlan 2" on both the eth port-profile that is connected to the upstream switch and the veth port-profile that the Service Console connects.

The following should always have system vlan set.
Any port-profile that carries VSM-VEM control and packet traffic, Service Console, iSCSI storage, NFS, and Vmotion.

Q: Why can't I add system vlan to every port-profile?
A: Because there is a limited amount of storage space that we are allowed on the VEM
We are limited to the following number of system vlans

Release 1.1, 1.2,1.3,1.3a - 16 port-profile with system vlan set
Release 1.3b - 32 port-profiles with system vlan set

Q: Can I modify veth and eth interfaces directly?
A: Yes but we highly discourage you against it.

We only suggest changing a setting on a specific veth or eth interface if you are troubleshooting a problem. Use port-profiles over making individual changes to veth or eth interfaces.

Nexus 1010

Q:Can I use access ports with the Nexus 1010?
A: No. The Nexus 1010 expects all physical ports to be configured as a trunk on its upstream connections.

Q:Is the native VLAN supported with the Nexus 1010?
A: No. The Nexus 1010 expects all VLANs to be tagged. If a VLAN is untagged the Nexus 1010 will drop its packets

Q:Can the management VLAN of my VSMs be different than the management VLAN of the 1010?
A: No. The Nexus 1010 expects the management VLAN of the VSMs to be on the same VLAN as it's management network. There is currently no way to specifiy a different management VLAN for the VSMs installed on a 1010.

Q:Can I split a VSM HA pair so that on VSM is on a 1010 and the other is on ESX?
A: This is not a supported configuration. Nexus 1010s should be purchased and deployed in pairs.There is a live migration process for migrating a VSM from and ESX server to a 1010 but long term running of a VSM split between a Nexus 1010 and an ESX VM is not supported.

Q:Is the Nexus 1010 HA design similar to the VSM?
A: Yes it is very similar. Heartbeat messages between the 1010s are passed over the control network. Domain IDs are used to identifiy unique Nexus 1010s.
Nexus 1010 Domain IDs should not overlap with VSM Domain IDs if they are on the same control vlan It will cause problems if they are on the same ccontrol vlan and use the same domain ID. Do not overlap the Domain ID at all.

This is not a supported configuration. Nexus 1010s should be purchased and deployed in pairs.There is a live migration process for migrating a VSM from and ESX server to a 1010 but long term running of a VSM split between a Nexus 1010 and an ESX VM is not supported.

Cisco Nexus 1000v FAQ

Installation and Deployment

Virtual Ethernet Module (VEM)

Upgrade

Nexus 1000V Installer Application

Licensing

Virtual Service Domains (VSD)

Port-Profile Questions

Nexus 1010

Installation and Deployment

Virtual Ethernet Module (VEM)

Upgrade

Nexus 1000V Installer Application

Licensing

Virtual Service Domain (VSD)

Port-Profile Questions

Nexus 1010

Cisco ACI Inter VRF/Tenant Route Leaking Design – Simplified!

Connecting Physical Servers To Cisco ACI Fabric - Simplified!

VXLAN/EVPN Configuration Example (N9k / p2p)