good resilient replacement for AT&T AVPN (MPLS) circuit

[hmc2500]

In our critical sites we typically have 1 internet circuit and a AT&T AVPN (MPLS) private circuit used more for failover. 

What type of circuit would be a good replacement for MPLS and still have good resiliency in our sites?  MPLS is expensive.  

[Joseph W. Doherty]

Possibly another ISP.

If your current ISP is acceptable for primary WAN, likely you can find another ISP for backup, or perhaps use both concurrently.

As to resiliency, that might be addressed by information your ISPs provide, i e. how does traffic get from point A to point B.

Decades ago, I worked at a company that wanted no shared points of possible failure for our WAN links.  Often the hardest part was insuring single points of failure of links and POPs weren't shared between our providers.

[hmc2500]

- Have to ask, are there any benefits having MPLS over other types of circuit (I can only think of better end to end QOS and having a private circuit using private ip addresses). 

It's so expensive compared to an internet circuit, might not give us valuable return of investment.

- I've also heard that some ISP's now offer a solution with 4G wireless as backup for an internet circuit. Would you know if that is reliable enough?

We have HA and other redundancy on the internal network. My experience is that most outages were caused by either a problem with the ISP or local access provide (MPLS) and not anything internal. 

In some cases I've learned that the ISP sometimes turns out to also be the local access provider in some areas (for us Comcast for example) and a fiber cut in the area can affect both circuits.

- What do other company's typically have as backup?

[Joseph W. Doherty]

"Have to ask, are there any benefits having MPLS over other types of circuit"

Yes there are.  One you've mentioned is some QoS support.

Perhaps the biggest benefit of any non-Internet path, is usually some "guarantee" of some amount of bandwidth.

"It's so expensive compared to an internet circuit, might not give us valuable return of investment."

Yep, MPLS, or other non-Internet bandwidth, is often much more expensive than Internet bandwidth.  Again, though, the former will often guarantee bandwidth, the latter doesn't.

The question is, is the cost difference worthwhile to your business needs?

Of course, do keep in mind, if you're willing to route different kinds of traffic over different paths, you can leverage both approaches (such routing, though, incurs its own additional costs).

I cannot comment on 4G wireless, beyond available bandwidth might be an issue.  I've been retired from IT about 5 years now, so cannot comment on what's typical backup network WAN technology.

Interestingly, the company I mentioned, in my prior post, had the same issue.  I.e. they had dual private WAN links, everywhere (they were also fairly large international company), and the WAN costs were a killer.  Trying to decrease costs they went from p2p leased lines, then frame-relay, then ATM and then MPLS.  Finally, they decided to use MPLS with Internet VPN as a backup and/or unimportant traffic path.

I found, we obtained WAN performance, across the Internet, almost the same as using a private WAN link, especially 3rd world.

Generally, Internet providers attempt to keep customers happy by minimizing any congestion with on-going bandwidth upgrades.

You can also use QoS across the Internet.  The "trick" is using QoS upon egress to the Internet, and don't oversubscribe a Internet path or share site-to-site Internet usage with general Internet usage.

For example, you could have a HQ site with a gigE with 500 CIR working with a mixture of branches whose aggregate will not send more than 500 Mbps to the HQ (and the HQ will not send more than any branch's particular CIR).  Effectively, you've made your own private WAN.  The only issue is, will the Internet, itself, congest when you push 500 Mbps?  It may, but as noted above, most ISPs do on-going work to insure they don't congest, but, yes, there's no guarantee of that.

Also again, being clever in how to use WAN bandwidth can reduce WAN costs, but "clever" has its own costs, which need to be evaluated too.  (At the time, "clever" cost about 1/10 of reduced savings cost.  Great ROI, but cannot say same ratio might be achieved today.)

[hmc2500]

Your response is much appreciated.

Yep, MPLS, or other non-Internet bandwidth, is often much more expensive than Internet bandwidth. Again, though, the former will often guarantee bandwidth, the latter doesn't.

There is such a thing as dedicated internet lines (typically fiber) now.  With that their sla guarantees 99% uptime (not sure if this is really true). However its again expensive but less expensive than MPLS for the same amount of bandwidth.

https://www.business.att.com/products/att-dedicated-internet.html?WT.srch=1&source=ECPS0000000PSM00P&wtpdsrchprg=AT%2526T%2520ABS&wtpdsrchgp=ABS_SEARCH&wtPaidSearchTerm=dedicated%20fiber%20line&wtpdsrchpcmt=dedicated%20fiber%20line&kid=kwd-3466163871...

I thought with SDWAN equipment you can implement QOS and prioritize critical traffic while using all WAN links in HA active/active mode (improved quality of experience?). 

My thought was having SDWAN + 2 dedicated internet lines might be a good way to closely match the reliability and resiliency of MPLS.  

Was just wondering what WAN links businesses typically use nowadays.

[Joseph W. Doherty]

Alas, both those dedicated Internet links, and SD-WAN, are, more-or-less, since I retired.

From the link you provided on the dedicated Internet links, I doubt they're actual worth their premium.  I don't doubt they will deliver what they promise, but what do they really promise?

I take them much like if I provided your host a gig link to my edge switch, for which I guarantee an uptime and bandwidth on that switch port to your host link.  But what about the rest of the network, like my host wants to intercommunicate with another host not on the same edge switch?

Regarding SDWAN, the little I understand of it, it might actually be awesome, much like a PfRv5 or so.  On the other hand, I've never been much impressed by AutoQoS.  I.e. it might work, but not nearly as well as it should.  Again, I just don't know.

Unfortunately, sellers of products will tout their value, keep in mind caveat emptor.

[Kanan Huseynli]

Hi,

check SDCI solutions like from megaport or equinix. They provide SLA based site-to-site access between their POPs. I never deployed such network, just check these type of providers and what they can provide.

[Joseph W. Doherty]

Ah, something new to me.  An interesting "product".

Basically, as with many other things, which is better, doing something yourself or paying someone else to do it?  Many criteria go into making such decisions.  Generally, costs increase to have anyone else do something for you, yet that may actually be a better choice because of factors such as if your time is worth much more doing your thing rather than something else that you'll be charged for.

In your case, yes MPLS bandwidth is more expensive than Internet bandwidth, but there's more to providing a well working network that the standalone cost of one component of your network.  Your MPLS might actually be, overall, the best possible approach or conversely the worst possible approach.  You really need to understand your network requirements and how different network technologies might be used to meet them, and their costs (both CapEx and OpEx), to determine whether there's actually a better option (which might also change over time).

Be very careful about "costs"; who is offering improvements.

As an example, earlier I mentioned a migration from frame-relay to ATM.  WAN vendor suggested, why we can provide you the same bandwidth for 2/3s the cost.  Management liked that!

Initial cutover was a fiasco!!!  Well, except TelCo costs did initially decrease by 1/3 for same bandwidth links.

What got overlooked was things like, by default, on FR, CIR wasn't enforced on our routers, but based on WAN provider suggestions, ATM SCR (CIR) was!  (Rather noticeable when CIR/SCR were from 1/10 to 1/4 of link's physical bandwidth.)

Or, BTW, our router interface cards all need to be replaced with ATM interface cards.

Or, BTW, the ATM interface cards required an IOS feature upgrade.

Or, BTW, the IOS feature upgrade incurred an increase in service contracts cost.

Or, BTW, unlike FR, ATM loses effective bandwidth due to cell overhead and last cell padding.  (Example, you can send 96 bytes in two 53 bytes ATM cells, you [only] lose 9.4% of your bandwidth to ATM cell overhead.  But if you send 97 byte, you need three 53 bytes cells, so overhead is now about 39%.)

Or, BTW, if the whole L2 frame, makes it to destination, missing a single cell (dropped in ATM network), whole L2 frame needs to be retransmitted.  Basically, similar situation as a corrupted frame from host's edge device to host.  Problem is, wasted bandwidth from ATM cloud to your site, because issue only discovered at end host, unlike a corrupted frame or packet which will be dropped before being forwarded to next L2/L3 device.

Were WAN costs reduced?  Probably, at least before we had to switch some branches to IMUX ATM (possibly another interface upgrade) to make up for the bandwidth lost to ATM.

Management, though, for some reason, felt there was no need to analyze the overall cost impact.  After all, our WAN bills did (initially) decrease.