VSAN Configuration between UCS FI and Nexus 5K

[Kshitij Purwar]

Hi Experts,

Please help me to understand VSAN deployment between FI and Nexus 5K over Traditional FC.

Should i use 2 different VSAN between FI and N5K or one VSAN on both side would be fine.

if there is any reference doc please send me link.

i would appreciate your responses.

Thanks

k

[Walter Dey]

Hi

I assume you have 2 N5k ?

In any case, one would create a dual fabric: FI-A and N5k-A resp. FI-B and N5k-B;

In each fabric you create at least one VSAN, which should be different in fabric A and B.

Zoning is done per fabric and vsan

https://supportforums.cisco.com/discussion/11912046/configure-nexus-5548-native-fc

[Kshitij Purwar]

Thanks Walter For your Reply. I read all the docs ,but it is not mention anywhere what is the benefit of using 2 vsan with infrastructure. I have 2 N5K and 2 FI and 3 Storage. i would appreciate if you could please help me to identify of advantage using 2 vsan in infrastructure.

[Walter Dey]

Hi Kshitij

Sorry for not being clear.

In the FC world, we talk about 2 separate / isolated fabrics.

This means: you create a VSAN eg. 10 on FI-A and N5k-A ; and another VSAN eg. 20 on FI-B and N5k-B.

Your server (service profile) has 2 vhba's, one in Fabric A (VSAN 10) the other one in Fabric B (VSAN 20).

Your storage is also dual homed to Fabric A (VSAN 10) resp. Fabric B (VSAN 20).

Zoning is done per VSAN !

[Kshitij Purwar]

Thanks a Lot for wonderful Explanation,but my query is why i need 2 VSAN if everything can work well if i use only 1 vsan within infrastructure.

is this a best practice to use 2 vsan or there are some benefits to use 2 vsan like fail over,load-balancing.

what i can not achieve if i use only 1 vsan.

[Walter Dey]

Hi

This FC dual fabric is best practise / recommendation since 20+ years; it's done by IBM, EMC, HP, NetApp,....

see eg. slide 28 or 55 in

http://www.alcatron.net/Cisco%20Live%202013%20Melbourne/Cisco%20Live%20Content/Data%20Centre%20And%20Virtualisation/BRKCOM-2002%20%20UCS%20Supported%20Storage%20Architectures%20and%20Best%20Practices%20with%20Storage.pdf

If you have eg only one vsan, and someone plugs a cable between a FC port connecting the 2 N5k, you merge the 2 separate fabrics, which could end in a disaster.

If however, the ports are in 2 different VSAN's, fabric merge will not happen. 

And please don't use VSAN id = 1 ; it's Special, cannot be deleted; all interfaces are basically in VSAN 1.

[Kshitij Purwar]

Thanks a Lot walter for your Time and sharing this doc.

Really a good technical presentation.

After reading this doc,now its clear -

1. I should use 2 vsan

2. i should also use san port-channel.

but i have 2 query-

1. should i use manually pin group on FI FC uplinks.

2. should i enable full zonset distribution when we have 2 different vsan on fabric.

Thanks

kshitij

[Walter Dey]

1. I should use 2 vsan

Yes, minimum is one vsan per fabric ! you could have more, e.g. for separating Windows and Linux servers

2. i should also use san port-channel.

yes ! your FI should be configured as FC end host (default setting). and then on your N5k, you have to enable

cae-sj9506(config)# feature fport-channel-trunk

see eg. https://supportforums.cisco.com/sites/default/files/legacy/9/9/2/53299-UCS_1-4-1_F-port_channel-trunk-v1.pdf

1. should i use manually pin group on FI FC uplinks.

No, this is usually not done. automating pinning is the default !

2. should i enable full zonset distribution when we have 2 different vsan on fabric.

I assume, your FC fabric consists only of one switch (N5k), therefore it doesn't matter !

Please be aware, that there is FC NO link between the 2 N5k's 

[Kshitij Purwar]

Thanks a Lot walter.

After reading your pointers and google search,i found out some learning,below is the summary -

1. i should use 2 vsan as best practice.

2. I should use device alias for zoning-

Query- As per the nexus doc we can create device alias in 2 manner,first as direct memeber of zone second create a device alias under fc-alias and add fc alias as member of zone.

Please suggest which one is good.

3. I should use F-port channel between UCS and nexus 5K

- As per the doc i should use seperate vsan on each FI and N5k and active mode on F-port channel.

on FI A side and nexus A side,i will enable vsan 100 on F port channel and on FI B side and nexus B side,i will enable vsan 200 on F port channel.

but reffering to below link,person has created 2 vsan on each Nexus and manually enabling  vsan on each uplink.

https://virtuallymikebrown.com/2013/05/23/cisco-nexus-fibre-channel-configuration-template/

it is still confusing,how will it work.

please suggest.should i follow him.

[Walter Dey]

2. I should use device alias for zoning-

yes ! there is also a fc-alias which works differently ! see e.g.

http://blog.scottlowe.org/2010/12/08/using-device-aliases-on-a-cisco-mds/

http://etctc.net/cisco/san/2015/08/26/Cisco-fcalias-or-device-alias/

http://lejyphilip.blogspot.ch/2012/08/dfference-between-fcalias-device-alias.html

https://supportforums.cisco.com/discussion/10425736/what-difference-between-fcalias-device-alias

3. I should use F-port channel between UCS and nexus 5K

yes !

on FI A side and nexus A side,i will enable vsan 100 on F port channel and on FI B side and nexus B side,i will enable vsan 200 on F port channel.

yes ! correct !

The reference https://virtuallymikebrown.com/2013/05/23/cisco-nexus-fibre-channel-conf... is totally misleading resp. wrong !

It shows a ISL link between the 2 MDS, trunking the 2 VSAN's of fabric A resp B. This would merge the 2 fabrics into one !

If you read the text carefully

.......So in short, this ISL is not needed. I’ll be removing it the next chance I get.......

[Kshitij Purwar]

Thanks Walter.

what is the use case of ISL.where should i use it.

As i know this is also use in ethernet to configure a trunk port to pass different vlans,but how it is related in san switch,where we can active only one vsan per switch.

confused.....

[Walter Dey]

Hi Kshitij

In case of N5k

no FC ISL between switches ! this is the guarantee for a dual fabric !

yes Ethernet ISL/trunk between switches , vPC is the recommendation here !

Walter.