cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
5507
Views
0
Helpful
9
Replies

VSM and VEM communication details L3 setup

rodriguezea
Level 1
Level 1

I am new to the nexus and to cisco. I have the nexus set up on my lab and everything is working but im trying to undestand some concepts and move towards the layer 3 configuration. After a week of reading cisco documents I think I know what I need to ask. (I think)

starting wtih L2.

1. All the documents state that VSM and VEM need to be on the same vlan.  I have added the packet, control and mgmt intefaces for the VSM to VLAN 10.  (my two test host are also on VLAN 10)  But what do they mean by VEM , where is the VEM interfrace. do they mean the VmwareKernelNIC, is that where the VEM endpoint would be on a host..how do I put the VEM on the same VLAN. I have it working just trying to see how is all connects.

Now on to L3

2. I have tried to switching to Layer 3  but cannot ping the the vmknics.

     a. I created VLAN 20 on my two hosts I have added a VMKNIC with IP 10.10.20.5 to host 1 and 10.10.20.6 to host 2

     b. I then Changed the SVS-DOMAIN to L3

     c. Went into the CONTROL 0 interfcae and gave it and IP address of 10.10.20.50

     d. created  a port group vethernet "Control_packetVLAN20) , its on  vlan 20 , capability l3 and assing it to the Control and Packeet interfaces on the VSM virtual machine nics on esx.

    

3. My VSM ip address for management is 192.168.1.50.  VLAN 10

I dont have a router for now. is Layer 3 mode requires a router. it migth be obvious but just need some confirmation.

I will keep reading a watching videos, but every video and document out there just puts it all on the same vlan and evertying works, which is easy. I am trying to isolate the traffic  Packet/control on one vlan and Management on another.

hope all this makes sense.

1 Accepted Solution

Accepted Solutions

Control and packet need to be on the same vlan as the vmknics. So from your example above you need to put them on VLAN 20.

port-profile type veth control-packet

switchport mode access

switchport access vlan 20

vmware port-group

no shut

system vlan 20

state enable

port-profile type veth vmkni-l3-control

switchport mode access

switchport access vlan 20

vmware port-group

no shut

capability l3control

system vlan 20

state enable

View solution in original post

9 Replies 9

lwatta
Cisco Employee
Cisco Employee

>1. All the documents state that VSM and VEM need to be on the same vlan.  I have added the packet, control and mgmt intefaces for the VSM to VLAN 10.  (my two test host are also on VLAN 10)  But what do they mean by VEM , where is the VEM interfrace. do they mean the VmwareKernelNIC, is that where the VEM endpoint would be on a host..how do I put the VEM on the same VLAN. I have it working just trying to see how is all connects.

We use internal ports in the VEM and they are not visible through vmware commands. You can use "vemcmd show ports and vemcmd show ports-old to see the internal N1KV ports. In L2 mode the port with LTL 10 is the control interface.

>Now on to L3

>

>2. I have tried to switching to Layer 3  but cannot ping the the vmknics.

>      a. I created VLAN 20 on my two hosts I have added a VMKNIC with IP 10.10.20.5 to host 1 and 10.10.20.6 to host 2

>     b. I then Changed the SVS-DOMAIN to L3

>    c. Went into the CONTROL 0 interfcae and gave it and IP address of 10.10.20.50

>     d. created  a port group vethernet "Control_packetVLAN20) , its on  vlan 20 , capability l3 and assing it to the Control and Packeet interfaces on the VSM virtual machine nics on esx.

The VMKnics you want to use on the ESXi hosts need to be in a port-profile with capability L3control, not the VSM interfaces. When you add capability L3control to a port-profile it tells the VEM that vmk interfaces assigned to that PP can be used for control traffic. The control and packet interfaces of the VSM should be in a port-profile without l3control but with system vlan set.

You only need a router if your control interface is in a different network then the vmk interfaces you want to use on the ESXi hosts for L3 control. Since everything is in the same network you don't need a router.

louis

Ok great, thatnks for the reply that cleared alot .

For the L3 I will add the VMKnic to the port-profile with capability L3 Control,

For the Control and Packet interfaces on the N1KV vm,  I will add them to a port profile without l3 control.  Any specific VLAN? like should it be able to talk to anythign specific? like the esx host? or they are not being used at all, once on Layer 3.

I will try thsi on my lab, like you mention I should be able to ping since they are on the same network. I will report the results back.

Thansk again.

Control and packet need to be on the same vlan as the vmknics. So from your example above you need to put them on VLAN 20.

port-profile type veth control-packet

switchport mode access

switchport access vlan 20

vmware port-group

no shut

system vlan 20

state enable

port-profile type veth vmkni-l3-control

switchport mode access

switchport access vlan 20

vmware port-group

no shut

capability l3control

system vlan 20

state enable

Ok I think thats what I was missing , I was creating one vether port-group and putting everything on it. I needed to have one l3 capable for the vmknics like you said, and a second one for the interfaces (control and packet) on the same vlan as teh vmknics.

I see the light!! 

Hello.

Do I understand correctly?

If we use L3 for VEM-VSM communication, must we use 3 Port-Profiles?

1. Port-Profile for ESXi host VMkernel management interface with L3 capability enabled. Let's name it Port-Profile type veth ESX-mg

2. Port-Profile for Control and Packet interfaces of the VSM without L3 capability enabled. Let's name it Port-Profile type veth Control-Packet

3. Port-Profile for Management interface of the VSM with L3 capability enabled. Let's name it Port-Profile type veth VSM-mg

Like this:

ESXi host ip:10.10.6.10/24, VLAN 521

VSM ip: 10.10.4.5/24, VLAN 1

port-profile type vethernet ESX-mg

  capability l3control

  vmware port-group

  port-binding static auto

  switchport mode access

  switchport access vlan 521

  no shutdown

  system vlan 521

  max-ports 64

  state enabled

port-profile type vethernet Control-Packet

  vmware port-group

  port-binding static auto

  switchport mode access

  switchport access vlan 1

  no shutdown

  system vlan 1

  max-ports 8

  state enabled

port-profile type vethernet VSM-mg

  capability l3control

  vmware port-group

  port-binding static auto

  switchport mode access

  switchport access vlan 1

  no shutdown

  system vlan 1

  max-ports 64

  state enabled

For my scenerio,yes I had 3

-On your example the second port-profile Control-packet needs to be on vlan 521 to talk to the vkernel nics. 

     But just clarify something, I  change the SVS-DOMAIN interface to use the CONTROL0 then I gave the ip address that is on the same vlan as the VMKNICS. 

     If you leave it using the MGMT0 interface.   I would think it would work. but havent tried that. I wanted to separate the traffic from my mgmt and vsm-vem communication.

-On the 3rd port-profile VSM-mg  it doesnt need to be capability l3 control.

The third profile for the VSM mgmt just needs to talk to vcenter. So wether is routable to Vcenter or on the same vlan will work.

All this was in my lab so I had no router that is why vsm-vem traffic is on the same vlan.

https://communities.cisco.com/docs/DOC-28631Found out this doc and after understanding the concepts this write up is very good.

https://communities.cisco.com/docs/doc-28631

I forget to mention that I use mgmt0 interface of the VSM fot L3 communication.

svs mode L3 interface mgmt0

Ok no problem, quick note in L3 mode Control and Packet traffic go over the same interface. in your case mgmt0.

So you have to make sure you have at least one VMKNIC on port-profile ESX-MG and give it an ip in that range.

You dont need your second interface CONTROL-PACKET.(since you are using mgmt0 for L3)

The 3rd inteface VSM-MG does not need to have "capability L3Control'

Once all that is setup, and you can ping your vmknic from the nexus command line you are good to go.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: