VLANs, Port Profiles, and Limitations

[9ball]

Here's an example scenario:

1 vSphere ESX cluster containing 30 ESX hosts.

1 HA-pair of Nexus 1000v VSM.

1 VEM installed on each of the 30 ESX hosts.

600 Virtual Machines distributed throughout the ESX cluster.

300 active VLANs

According to the Cisco Nexus 1000v Getting Started Guide, Release 4.0(4)SV1(3) - Configuration Limits, one can configure up to 512 active VLANs across all VEMs in a single 1000v instance; but can only configure 256 port profiles.  Am I reading that right?

I see several ways to get around this, but none of them are as desirable as being able to configure more Port Profiles in a single 1000v instance.

One workaround is to not assign VLANs in a Port Profile and instead manually configure the VLAN on the Vethernet port after it has been created.  However, this would all but eliminate the usefulness of Port Profiles.  Furthermore, Cisco has stated in the Interface Configuration Guide "[w]e do not recommend that you override port profile  configurations by making changes to the assigned interface  configurations. Only make configuration changes to interfaces to quickly  test a change or to disabled a port."  I don't know why they don't recommend it, but I would guess that there could be a chance that a Port Profile configuration could somehow be reapplied to an interface and you would lose the custom configuration.

Another workaround is to deploy more than one 1000v instance.  Using the example above, I could put half of the VEMs on one 1000v instance, and half on another.  Neither instance could have all of the VLANs, so the 1000v would essentially force a redesign of the VMware infrastructure; affecting things such as DRS, VMotion, and potentially other VMware design considerations of which I'm not even aware (I'm not a VMware guy).

Assuming for a moment that I'm not wrong about this limitation and its effects, which I  could very well be, why does the limitation exist?  VMware supports up  to 512 port groups on a host and apparently 4094 VLANs on a vSwitch.  Why are the limits of the 1000v so much lower?  Is that just the price that has to be paid in order to enjoy the other features of the 1000v?  While the 1000v would enhance the security  and manageability of the network, it seems like it would actually limit the otherwise bountiful capacity of  an environment that utilizes a lot of VLANs.

I'd really like to hear from someone that I've got it all wrong and that there's a simple solution to my perceived problem.

~Thanks...

[Robert Burns]

9Ball,

You're read correctly.  Though some of the limitations were historically from VMware, we have "throttled" some of the limitations in our early releases.

The next release (SV1.4) which should be out in a couple months will increase the following limits:

Limits	SV1.3(a)	SV1.4
Port Profiles	256	2048
Active VLANs	256	2048
PVLANs	512	2048

I understand some of the limits "don't make sense", but our dev teams try to roll out features in a controlled manner and sometimes they don't always add up logically

Additionally, the default maximum number of ports per port-profile is currently hardcoded to 32. In VMware vSphere 4.1, the maximum number of supported ports per DVS is increasing from 8000 ports to 30000. With the vSphere 4.0 maximum, you might want to have the maximum ports per port-profile set to a lower value say four. For vSphere 4.1, the user might want to set the maximum to fifteen per port-profile. We've added the ability for the user to set the default maximum number of ports per port-profile.  To do this, we will introduce a new configuration command:

switch(config)# port-profile default max-ports <num>

Existing port-profiles using the old default value will not be affected. Only port-profiles created after the new default is set will be configured with the new default.

Hope this helps (or soon will).

Regards,

Robert

[9ball]

Thanks, Robert.  I'm glad the limitation is only a cautionary throttling and not something worse.  2048 port-profiles per DVS would definitely be sufficient for the foreseeable future.  I also see that the stats I grabbed for VMWare with respect to maximum port groups and vlans were old; vSphere 4.1 drastically increased a lot of maximums.  http://www.vmware.com/pdf/vsphere4/r41/vsp_41_config_max.pdf

Thanks for pointing out the maximum ports per DVS.  I don't think we're on 4.1 yet in production, but I have been testing the 1000v on vSphere 4.1 in the lab.  I'll make sure that vSphere 4.1 is a pre-requisite to deployment of the 1000v in production so that max-ports is not as much of a concern.  I think the VM admins have already planned for an upgrade to 4.1 soon.

Thanks again.

[admin11111]

I had faced the similar issue . But fixed by extending the individual port profile max ports.