04-02-2013 06:20 AM
I am new to the nexus and to cisco. I have the nexus set up on my lab and everything is working but im trying to undestand some concepts and move towards the layer 3 configuration. After a week of reading cisco documents I think I know what I need to ask. (I think)
starting wtih L2.
1. All the documents state that VSM and VEM need to be on the same vlan. I have added the packet, control and mgmt intefaces for the VSM to VLAN 10. (my two test host are also on VLAN 10) But what do they mean by VEM , where is the VEM interfrace. do they mean the VmwareKernelNIC, is that where the VEM endpoint would be on a host..how do I put the VEM on the same VLAN. I have it working just trying to see how is all connects.
Now on to L3
2. I have tried to switching to Layer 3 but cannot ping the the vmknics.
a. I created VLAN 20 on my two hosts I have added a VMKNIC with IP 10.10.20.5 to host 1 and 10.10.20.6 to host 2
b. I then Changed the SVS-DOMAIN to L3
c. Went into the CONTROL 0 interfcae and gave it and IP address of 10.10.20.50
d. created a port group vethernet "Control_packetVLAN20) , its on vlan 20 , capability l3 and assing it to the Control and Packeet interfaces on the VSM virtual machine nics on esx.
3. My VSM ip address for management is 192.168.1.50. VLAN 10
I dont have a router for now. is Layer 3 mode requires a router. it migth be obvious but just need some confirmation.
I will keep reading a watching videos, but every video and document out there just puts it all on the same vlan and evertying works, which is easy. I am trying to isolate the traffic Packet/control on one vlan and Management on another.
hope all this makes sense.
Solved! Go to Solution.
04-02-2013 08:21 AM
Control and packet need to be on the same vlan as the vmknics. So from your example above you need to put them on VLAN 20.
port-profile type veth control-packet
switchport mode access
switchport access vlan 20
vmware port-group
no shut
system vlan 20
state enable
port-profile type veth vmkni-l3-control
switchport mode access
switchport access vlan 20
vmware port-group
no shut
capability l3control
system vlan 20
state enable
04-02-2013 07:12 AM
>1. All the documents state that VSM and VEM need to be on the same vlan. I have added the packet, control and mgmt intefaces for the VSM to VLAN 10. (my two test host are also on VLAN 10) But what do they mean by VEM , where is the VEM interfrace. do they mean the VmwareKernelNIC, is that where the VEM endpoint would be on a host..how do I put the VEM on the same VLAN. I have it working just trying to see how is all connects.
We use internal ports in the VEM and they are not visible through vmware commands. You can use "vemcmd show ports and vemcmd show ports-old to see the internal N1KV ports. In L2 mode the port with LTL 10 is the control interface.
>Now on to L3
>
>2. I have tried to switching to Layer 3 but cannot ping the the vmknics.
> a. I created VLAN 20 on my two hosts I have added a VMKNIC with IP 10.10.20.5 to host 1 and 10.10.20.6 to host 2
> b. I then Changed the SVS-DOMAIN to L3
> c. Went into the CONTROL 0 interfcae and gave it and IP address of 10.10.20.50
> d. created a port group vethernet "Control_packetVLAN20) , its on vlan 20 , capability l3 and assing it to the Control and Packeet interfaces on the VSM virtual machine nics on esx.
The VMKnics you want to use on the ESXi hosts need to be in a port-profile with capability L3control, not the VSM interfaces. When you add capability L3control to a port-profile it tells the VEM that vmk interfaces assigned to that PP can be used for control traffic. The control and packet interfaces of the VSM should be in a port-profile without l3control but with system vlan set.
You only need a router if your control interface is in a different network then the vmk interfaces you want to use on the ESXi hosts for L3 control. Since everything is in the same network you don't need a router.
louis
04-02-2013 08:17 AM
Ok great, thatnks for the reply that cleared alot .
For the L3 I will add the VMKnic to the port-profile with capability L3 Control,
For the Control and Packet interfaces on the N1KV vm, I will add them to a port profile without l3 control. Any specific VLAN? like should it be able to talk to anythign specific? like the esx host? or they are not being used at all, once on Layer 3.
I will try thsi on my lab, like you mention I should be able to ping since they are on the same network. I will report the results back.
Thansk again.
04-02-2013 08:21 AM
Control and packet need to be on the same vlan as the vmknics. So from your example above you need to put them on VLAN 20.
port-profile type veth control-packet
switchport mode access
switchport access vlan 20
vmware port-group
no shut
system vlan 20
state enable
port-profile type veth vmkni-l3-control
switchport mode access
switchport access vlan 20
vmware port-group
no shut
capability l3control
system vlan 20
state enable
04-02-2013 08:41 AM
Ok I think thats what I was missing , I was creating one vether port-group and putting everything on it. I needed to have one l3 capable for the vmknics like you said, and a second one for the interfaces (control and packet) on the same vlan as teh vmknics.
I see the light!!
04-03-2013 08:11 AM
Hello.
Do I understand correctly?
If we use L3 for VEM-VSM communication, must we use 3 Port-Profiles?
1. Port-Profile for ESXi host VMkernel management interface with L3 capability enabled. Let's name it Port-Profile type veth ESX-mg
2. Port-Profile for Control and Packet interfaces of the VSM without L3 capability enabled. Let's name it Port-Profile type veth Control-Packet
3. Port-Profile for Management interface of the VSM with L3 capability enabled. Let's name it Port-Profile type veth VSM-mg
Like this:
ESXi host ip:10.10.6.10/24, VLAN 521
VSM ip: 10.10.4.5/24, VLAN 1
port-profile type vethernet ESX-mg
capability l3control
vmware port-group
port-binding static auto
switchport mode access
switchport access vlan 521
no shutdown
system vlan 521
max-ports 64
state enabled
port-profile type vethernet Control-Packet
vmware port-group
port-binding static auto
switchport mode access
switchport access vlan 1
no shutdown
system vlan 1
max-ports 8
state enabled
port-profile type vethernet VSM-mg
capability l3control
vmware port-group
port-binding static auto
switchport mode access
switchport access vlan 1
no shutdown
system vlan 1
max-ports 64
state enabled
04-03-2013 10:01 AM
For my scenerio,yes I had 3
-On your example the second port-profile Control-packet needs to be on vlan 521 to talk to the vkernel nics.
But just clarify something, I change the SVS-DOMAIN interface to use the CONTROL0 then I gave the ip address that is on the same vlan as the VMKNICS.
If you leave it using the MGMT0 interface. I would think it would work. but havent tried that. I wanted to separate the traffic from my mgmt and vsm-vem communication.
-On the 3rd port-profile VSM-mg it doesnt need to be capability l3 control.
The third profile for the VSM mgmt just needs to talk to vcenter. So wether is routable to Vcenter or on the same vlan will work.
All this was in my lab so I had no router that is why vsm-vem traffic is on the same vlan.
04-03-2013 10:13 AM
https://communities.cisco.com/docs/DOC-28631Found out this doc and after understanding the concepts this write up is very good.
04-04-2013 12:42 PM
I forget to mention that I use mgmt0 interface of the VSM fot L3 communication.
svs mode L3 interface mgmt0
04-09-2013 11:15 AM
Ok no problem, quick note in L3 mode Control and Packet traffic go over the same interface. in your case mgmt0.
So you have to make sure you have at least one VMKNIC on port-profile ESX-MG and give it an ip in that range.
You dont need your second interface CONTROL-PACKET.(since you are using mgmt0 for L3)
The 3rd inteface VSM-MG does not need to have "capability L3Control'
Once all that is setup, and you can ping your vmknic from the nexus command line you are good to go.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: