best design to create default route for ACI tenants

[Bojan Zivancevic]

I am in the process of reading the docs related to routing principles inside ACI fabric. While I am doing this I would like a high level recommendation on possible options on this topic. Emphasizing "high level", I do not want step by step instructions, just guidance.

So, the starting point is a clean ACI fabric with two tenants. For simplicity sake, these two tenants have just one EPG each, it's a vlan/subnet. I need to provide the default route for both tenants to make their way to the internet. Physical router is the gateway for ACI tenants, it will be connected to one of the leaf ports.This means I need to "share" this link somehow with each tenant.

- I noticed there is a "common" tenant. Is this the way to go, to somehow implement this default routing inside a common tenant?

- Should I make a new tenant just for this default routing? But then I must allow the traffic to pass between the tenants.

- Third idea I am having is to somehow associate the default route with some new EPG, then make contracts to allow the traffic to be passed - but this won't work AFAIK because EPG's can talk to each other only if they belong to the same tenant.

Many thanks

[dpita]

Hello

Yes this is possible. Its a variation on "Shared L3 out" sometimes known as "shared internet" whereby the common tenant would have a 0.0.0.0/0 route and it would be shared or leaked into the other VRFs. you don't necessarily need to make a new tenant. Common can provide or any user tenant can provide it. Inter-tenant communication between EPGs is possible, just needs to special contracts and its known as inter-tenant route leaking. 

take a look at this article for more information on shared L3 out.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/aci-fundamentals/b_ACI-Fundamentals/b_ACI-Fundamentals_chapter_010100.html

[Bojan Zivancevic]

Thanks for the suggestion.

By the way, does ACI support 802.1q trunks on the leafs, connecting to external physical switches? If it does, can I use these vlans for different EPGs or tenants?

[dpita]

Absolutely! we support 802.1q. if you have one trunk interface going out to an external switch. its best to use different EPGs per VLAN you want to extend, that way you can continue to use the same port/path. Doesn't matter if its same or different tenant. any tenant can use the same interface/path with different VLANs. 

[Bojan Zivancevic]

Right, this should mean that I can change my initial setup above (with physical router connected to the leaf port) and use just one physical connection (dot1q) to the external switch, and then use for instance several vlans for my "customer" EPGs/tenants and one vlan for my external shared L3 connection to the outside world. Basically I would not connect the router directly this time, it would be the switch over dot1q link.

Am I right?

[dpita]

You can do both. 

If you want to flood all traffic to the L2 switch shared by all tenants and then to the Router, thats doable but probably not as optimized as connected the router to the leaf and creating and L3 out as well as connecting the switch for other purposes?

Not too sure i understand the design but the basics are: if you can do it with traditional network equipment you can do it on ACI. Just need to think about the policies and goal. 

[Bojan Zivancevic]

No no, I want to use shared L3 out. It's just the vlan-based setup, not physical port-based (instead of having two physical ports I would like to use just one with two SVI interfaces). So, I would like to have sort of SVI L3 interfaces on ACI side. One SVI for internal VMs and the other one for the connection to the internet router.

ACI would serve as a default gateway for VMs (this is "VM vlan" SVI) and ACI would also have a SVI inside "internet vlan" that would be used for talking to the internet router.

The other option is to use two physical ports of course. The function would be the same, it's just the number of ports used.

[Bojan Zivancevic]

I have been through some more documents trying to figure out how this L3-out is working. Let me rephrase the question according to my latest findings.

It seems that connections to outside world can be done in three ways. Classic L3 interface, routed interface with subinterfaces (usual stuff done on physical routers), or SVI interfaces. They have one thing in common in the documentation - they are all supposed to be used for external communication.

This is just fine because the topic (in the docs) is external connectivity. And it seems I can use dot1q subinterfaces and all. Great.

What I really want to know is: Can I use that same dot1q trunk to mix tenant vlans and external connections? Let me put it in the most simpler way: I have two vlans on that trunk port. One is supposed to be connected to some virtual machines (they belong to the tenant), and the other vlan is my L3-out connection to outside world.

This is not covered in any document I have found. They all talk about having multiple vlans on the same interface, but all for L2/L3 out purposes.

[dpita]

VMs through L2 switch to ACI should work. But to be honest I'm not sure on the Internet router through the L2 switch. I've only ever seen it as directly connected P2P connection or a vPC design. while I'm sure the L2 connectivity would work not sure how the peering would work. 

I just configured this in the lab (Sharing an interface between an L3 out SVI and an EPG static path), while the configuration does get accepted it may or may not be supported or work for the L3 peering, i can't say. 

What i can comment on is that the access policies become quite a mess. You would need a single VLAN pool in order tied to two domains. one for under the EPG and one for the L3 out. 

I suggest you test it in the lab and see if it works for your needs. Hope that helps!

[Bojan Zivancevic]

Not sure I want to investigate further! :)

So, you are saying that I would be better of to use just one physical port for L3out connection to the router, and the second physical port to connect my vmware infrastructure? This is how I understand your suggestion. If this is true, I can do it and will do it this way. No need to overcomplicate things.