Questions about L2 VPN ping

[morris0117]

Greetings,

On an international L2 VPN service, the customer is complaining about latency.
Here are several ping tests on the 2 PE routers:

RouterA#ping mpls pseudowire b.b.b.b vcid source a.a.a.a repeat n
Success rate is 100 percent (100/100), round-trip min/avg/max = 15/15/19 ms
Success rate is 100 percent (500/500), round-trip min/avg/max = 15/15/23 m
Success rate is 100 percent (1000/1000), round-trip min/avg/max = 15/15/116 ms
Success rate is 100 percent (1000/1000), round-trip min/avg/max = 15/15/29 ms
Success rate is 100 percent (1000/1000), round-trip min/avg/max = 15/15/28 ms

RouterB#ping mpls pseudowire a.a.a.a vcid source b.b.b.b repeat n
Success rate is 100 percent (500/500), round-trip min/avg/max = 16/16/113 ms
Success rate is 100 percent (500/500), round-trip min/avg/max = 16/16/27 ms
Success rate is 100 percent (1000/1000), round-trip min/avg/max = 15/16/22 ms
Success rate is 100 percent (1000/1000), round-trip min/avg/max = 15/16/32 ms
Success rate is 100 percent (1000/1000), round-trip min/avg/max = 16/16/22 ms

It looks low in most tests, but there are still 116 ms and 113 ms cases.
Q1: Is it normal to have those rare high rtt? If not, what should we do next? 
Q2: How can avg rtt equal to min when max is not the same? How does it calculate the avg?

Thanks a lot!

[Kumaresan Ravichandran]

Hello @morris0117 

It's not normal to have large spikes RTT on a stable network connection. MPLS configuration or unexpected routing changes can also lead to latency spikes. Verify that your MPLS setup and routing tables are configured correctly and that there are no unexpected routing changes during the high latency times.

Are you tested at different times of the day or under varying network loads to see if there is a pattern to the latency spikes ?

>_<

[MHM Cisco World]

Do traceroute mpls 

Share output 

MHM

[morris0117]

@Kumaresan Ravichandran ,

Thanks for replying. I'm not sure what you mean by "under varying network loads" but I just simply did the ping test on the same routers several times within an hour this morning. BTW, a senior colleague just told me it's normal in our network bc the router sometimes need to process some prioritized tasks so it responds to icmp late and it causes latency.

Now there's still one question: How can avg rtt equal to min rtt when max rtt is not the same? How does avg rtt be calculated?

@MHM Cisco World , 

Thank you for actively replying as always. I checked the traceroute and it's fine, without high latency.
Pls help to answer the 2nd question if you're interested. Thanks.

[Kumaresan Ravichandran · Accepted solution]

@morris0117 

Here some relatable answer for second one.

The average RTT is calculated by summing all the RTT values for a set of ping responses and then dividing by the total number of responses.

Consider a set of ping responses with a large number of low RTTs and a few high RTTs. If you have 990 responses with an RTT of 15ms and 10 responses with higher RTTs (say, ranging from 19ms to 116ms), the average will be much closer to 15ms, even though the maximum RTT is much higher.

If you calculate the average with the above example, it is clear that the majority of low RTTs will keep the average close to the minimum: Average RTT=(990×15)+19+23+29+116/1000=15.18ms

>_<

Rate it, If its helpful to you.

[Joseph W. Doherty]

BTW, what @Kumaresan Ravichandran shows for computing an average is correct, variants would cause the same results.  I.e. most of the values are near the min result but with very few higher values, even just one high peak value will show that kind of result.

[morris0117]

@Kumaresan Ravichandran 

Thanks for giving that calculation example, that's clear! So it seems that the decimal places of the output are omitted.

[MHM Cisco World]

Use ping repeat two 

And check 

Min/average/Max

MHM

[Joseph W. Doherty]

"BTW, a senior colleague just told me it's normal in our network bc the router sometimes need to process some prioritized tasks so it responds to icmp late and it causes latency."

What your senior told you is correct concerning any pinged host might be slow to respond.  I.e. ping latency is the combination of transit delay, in both directions, and host response times.

For the latter, Cisco supports SLA pings where the receiving host negates its response delay by noting how long it took to process the ping reply.  This feature might not be provided for MPLS.

However, another cause of occasional ping times can be transient queuing delays.

How to tell host delay from network delay?  Either use a dedicated responder only for this purpose, a responder which can negate its own delay, or ping different responders, concurrently, and see if they get similar results (at the same times).

[Georg Pauwen]

Hello,

--> On an international L2 VPN service, the customer is complaining about latency.

Where does this latency manifest itself ? Is it related to specific applications ? What kind of QoS do you have implemented on the CE/PE routers ?

Can you post the output of:

show mpls forwarding-table

?

Sorry for asking all these question, but troubleshooting latency in MPLS L2VPNs can be a bit cumbersome...