Pseudowire over XRd not working?

[SA2]

Hi, We´ve been using XRd as a way to do integration testing with our autoprovisioning platform using gnmi.
This works great.

However I'm currently trying to do end to end testing of the whole solution with actual traffic forwarding across the finished service.
(A simple ping test from the CPE to a loopback on a remote SE).

Routing/LDP is up and running and all the XRd nodes have reachability and signal the PWHE correctly, it is UP/UP.
But trying to forward traffic through it fails.

Ive simplified the setup to be more minimal for testing. but still seeing the same issue

The configuration on AGG1:

Configuration on AGG2 is mirrored.

CPE1 has a ip interface with 1.1.1.1/24 trying to ping 1.1.1.2 on CPE2.

The packets are received on AGG1 and sent across mpls to AGG2.
As can be seen on wireshark:

AGG2 however does not seem to recognise that these should be sent out to CPE2. and reports nothing received or sent in the show l2vpn group g xc p details output.

Is this something that simply does not work on the XRd-control plane version?
Or should it work but with restrictions to performance.
We would just like to do simple verification of the configured service in this virtual environment.

-Anders

[Harold Ritter]

Hi @SA2 ,

I am glad you got it working. BTW, what version of XRd are you using?

Regards,

[SA2]

This is on 7.9.2 I still can't get pseudowire headend to work, or even tagged pseudowire when using different vlans on the different egress LSRs, but progress is being made.

[MHM Cisco World]

Different vlan this new issue?

MHM

[SA2]

Given the following topology:

The three node pairings at the bottom of the diagram are the endpoints of the three pseudowires configured in the lab.

After disabling excplicit null labelling on the AGG nodes the simplest PW between ACC1 and ACC2 is working.
Here both AGG1 and AGG2 have the same VLAN configured on the access-circuit:

AGG1:
interface GigabitEthernet0/0/0/2.100 l2transport
 encapsulation dot1q 100
 rewrite ingress tag pop 1 symmetric
!
l2vpn
 xconnect group g
  p2p p
   interface GigabitEthernet0/0/0/2.100
   neighbor ipv4 10.0.0.22 pw-id 100
   !
  !
!

AGG2:
interface GigabitEthernet0/0/0/2.100 l2transport
 encapsulation dot1q 100
 rewrite ingress tag pop 1 symmetric
!
l2vpn
 xconnect group g
  p2p p
   interface GigabitEthernet0/0/0/2.100
   neighbor ipv4 10.0.0.21 pw-id 100
   !
  !
!

ICMP ping between ACC1 and ACC2 using this PW works correctly.

On the second PW the VLANS differ on the AGGs, but this should be taken care of by the rewrite and this does indeed work in our production network.

AGG1:
interface GigabitEthernet0/0/0/2.666 l2transport
 ! Different VLAN on other end
 encapsulation dot1q 666
 rewrite ingress tag pop 1 symmetric
!
l2vpn
 xconnect group g
  p2p p_notworking
   ! Does not work (different vlan on either side)
   interface GigabitEthernet0/0/0/2.666
   neighbor ipv4 10.0.0.22 pw-id 666
   !
  !
!

AGG2:
interface GigabitEthernet0/0/0/2.666 l2transport
 ! Different VLAN on other end
 encapsulation dot1q 333
 rewrite ingress tag pop 1 symmetric
!
l2vpn
 xconnect group g
  p2p p_notworking
   ! Does not work (different vlan on either side)
   interface GigabitEthernet0/0/0/2.666
   neighbor ipv4 10.0.0.21 pw-id 666
   !
  !
 !
!

This does not work. ARP fails to resolve the remote address.

In the third scenario, there is a pseudowire headend node on the SE1 device.

AGG1:
interface GigabitEthernet0/0/0/2.2 l2transport
 encapsulation dot1q 2
 rewrite ingress tag pop 1 symmetric
!
l2vpn
 xconnect group g
  p2p p_pwhe_notworking
   interface GigabitEthernet0/0/0/2.2
   neighbor ipv4 10.0.0.11 pw-id 100100
   !
  !
 !
!

SE1:
interface PW-Ether100
 ipv4 address 2.2.2.1 255.255.255.0
 attach generic-interface-list ALL
!
l2vpn
 xconnect group g
  p2p p_pwhe_notworking
   interface PW-Ether100
   neighbor ipv4 10.0.0.21 pw-id 100100
  !
 !
!

 Here ICMP is not working, however ARP resolves correctly on the SE node, but not on the ACC node.
Indicating that the packet is handled correctly going from the ACC to the SE but not the other way around.

Full configurations for reference:

AGG1:
hostname AGG1
!
grpc
 port 57400
 address-family dual
!
address-family ipv4 unicast
!
address-family ipv6 unicast
!
line default
 transport input ssh
!
call-home
 service active
 contact smart-licensing
 profile CiscoTAC-1
  active
  destination transport-method email disable
  destination transport-method http
 !
!
netconf-yang agent
 ssh
!
!
interface Loopback0
 description system interface 
 ipv4 address 10.0.0.21 255.255.255.255
!
interface MgmtEth0/RP0/CPU0/0
 ipv4 address 172.100.100.21 255.255.255.0
 ipv6 address 2001:172:100:100::21/80
!
interface GigabitEthernet0/0/0/0
 description AGG1->SE1
 mtu 9000
 ipv4 address 10.0.11.2 255.255.255.0
 dampening
!
interface GigabitEthernet0/0/0/1
 description AGG1->AGG2
 mtu 9000
 ipv4 address 10.0.21.1 255.255.255.0
 dampening
!
interface GigabitEthernet0/0/0/2
 description AGG1->ACC1
!
interface GigabitEthernet0/0/0/2.2 l2transport
 encapsulation dot1q 2
 rewrite ingress tag pop 1 symmetric
!
interface GigabitEthernet0/0/0/2.100 l2transport
 encapsulation dot1q 100
 rewrite ingress tag pop 1 symmetric
!
interface GigabitEthernet0/0/0/2.666 l2transport
 ! Different VLAN on other end
 encapsulation dot1q 666
 rewrite ingress tag pop 1 symmetric
!
router isis 1337
 is-type level-2-only
 net 49.0010.0000.0000.0021.00
 address-family ipv4 unicast
  metric-style wide
  mpls traffic-eng level-2-only
  mpls traffic-eng router-id Loopback0
 !
 interface Loopback0
  passive
  address-family ipv4 unicast
  !
 !
 interface GigabitEthernet0/0/0/0
  circuit-type level-2-only
  point-to-point
  address-family ipv4 unicast
   fast-reroute per-prefix
   metric 10000 level 2
   mpls ldp sync
  !
 !
 interface GigabitEthernet0/0/0/1
  circuit-type level-2-only
  point-to-point
  address-family ipv4 unicast
   fast-reroute per-prefix
   metric 10000 level 2
   mpls ldp sync
  !
 !
!
l2vpn
 xconnect group g
  p2p p
   interface GigabitEthernet0/0/0/2.100
   neighbor ipv4 10.0.0.22 pw-id 100
   !
  !
  p2p p_notworking
   ! Does not work (different vlan on either side)
   interface GigabitEthernet0/0/0/2.666
   neighbor ipv4 10.0.0.22 pw-id 666
   !
  !
  p2p p_pwhe_notworking
   interface GigabitEthernet0/0/0/2.2
   neighbor ipv4 10.0.0.11 pw-id 100100
   !
  !
 !
!
mpls ldp
 router-id 10.0.0.21
 address-family ipv4
 !
 interface GigabitEthernet0/0/0/0
 !
 interface GigabitEthernet0/0/0/1
 !
!
mpls ip-ttl-propagate disable forwarded
ssh server v2
ssh server netconf vrf default
end

AGG2:
hostname AGG2
!
grpc
 port 57400
 address-family dual
!
address-family ipv4 unicast
!
address-family ipv6 unicast
!
line default
 transport input ssh
!
call-home
 service active
 contact smart-licensing
 profile CiscoTAC-1
  active
  destination transport-method email disable
  destination transport-method http
 !
!
netconf-yang agent
 ssh
!
! 
interface Loopback0
 description system interface 
 ipv4 address 10.0.0.22 255.255.255.255
!
interface MgmtEth0/RP0/CPU0/0
 ipv4 address 172.100.100.22 255.255.255.0
 ipv6 address 2001:172:100:100::22/80
!
interface GigabitEthernet0/0/0/0
 description AGG2->SE1
 mtu 9000
 ipv4 address 10.0.12.2 255.255.255.0
 dampening
!
interface GigabitEthernet0/0/0/1
 description AGG2->AGG1
 mtu 9000
 ipv4 address 10.0.21.2 255.255.255.0
 dampening
!
interface GigabitEthernet0/0/0/2
 description AGG2->ACC2
!
interface GigabitEthernet0/0/0/2.100 l2transport
 encapsulation dot1q 100
 rewrite ingress tag pop 1 symmetric
!
interface GigabitEthernet0/0/0/2.666 l2transport
 ! Different VLAN on other end
 encapsulation dot1q 333
 rewrite ingress tag pop 1 symmetric
!
router isis 1337
 is-type level-2-only
 net 49.0010.0000.0000.0022.00
 address-family ipv4 unicast
  metric-style wide
  mpls traffic-eng level-2-only
  mpls traffic-eng router-id Loopback0
 !
 interface Loopback0
  passive
  address-family ipv4 unicast
  !
 !
 interface GigabitEthernet0/0/0/0
  circuit-type level-2-only
  point-to-point
  address-family ipv4 unicast
   fast-reroute per-prefix
   metric 10000 level 2
   mpls ldp sync
  !
 !
 interface GigabitEthernet0/0/0/1
  circuit-type level-2-only
  point-to-point
  address-family ipv4 unicast
   fast-reroute per-prefix
   metric 10000 level 2
   mpls ldp sync
  !
 !
!
l2vpn
 xconnect group g
  p2p p
   interface GigabitEthernet0/0/0/2.100
   neighbor ipv4 10.0.0.21 pw-id 100
   !
  !
  p2p p_notworking
   ! Does not work (different vlan on either side)
   interface GigabitEthernet0/0/0/2.666
   neighbor ipv4 10.0.0.21 pw-id 666
   !
  !
 !
!
mpls ldp
 router-id 10.0.0.22
 address-family ipv4
 !
 interface GigabitEthernet0/0/0/0
 !
 interface GigabitEthernet0/0/0/1
 !
!
mpls ip-ttl-propagate disable forwarded
ssh server v2
ssh server netconf vrf default
end

SE1:
hostname SE1
!
grpc
 port 57400
 address-family dual
!
vrf ISP
 address-family ipv4 unicast
  import route-target
   1337:10046
  !
  export route-target
   1337:10046
  !
 !
 address-family ipv6 unicast
  import route-target
   1337:10046
  !
  export route-target
   1337:10046
  !
 !
!
vrf Customer-Test-VRF
 rd 10.0.0.11:666
 address-family ipv4 unicast
  import route-target
   1337:666
  !
  export route-target
   1337:666
  !
  maximum prefix 100 80
 !
 address-family ipv6 unicast
  import route-target
   1337:666
  !
  export route-target
   1337:666
  !
 !
!
address-family ipv4 unicast
!
address-family ipv6 unicast
!
line default
 transport input ssh
!
dhcp ipv4
 profile ISP-DHCP-PROXY proxy
  helper-address vrf ISP 198.51.100.67 giaddr 0.0.0.0
  helper-address vrf default 172.100.200.201 giaddr 0.0.0.0
  relay information option vpn
  relay information option
  relay information option vpn-mode rfc
  relay information option remote-id LAB-PE-1
  relay information option allow-untrusted
  relay information option remote-id enable
 !        
 interface PW-Ether666 proxy information option format-type circuit-id IN-666
 interface PW-Ether666 proxy profile ISP-DHCP-PROXY
!
call-home
 service active
 contact smart-licensing
 profile CiscoTAC-1
  active
  destination transport-method email disable
  destination transport-method http
 !
!
netconf-yang agent
 ssh
!
!
policy-map test
 class class-default
  set cos 5
 ! 
 end-policy-map
! 
interface Loopback0
 description system interface 
 ipv4 address 10.0.0.11 255.255.255.255
!
interface MgmtEth0/RP0/CPU0/0
 ipv4 address 172.100.100.11 255.255.255.0
 ipv6 address 2001:172:100:100::11/80
!
interface GigabitEthernet0/0/0/0
 description SE1->AGG1
 mtu 9000
 ipv4 address 10.0.11.1 255.255.255.0
 dampening
!
interface GigabitEthernet0/0/0/1
 description SE1->AGG2
 mtu 9000
 ipv4 address 10.0.12.1 255.255.255.0
 dampening
!
interface GigabitEthernet0/0/0/2
 shutdown
!
interface GigabitEthernet0/0/0/3
 shutdown
!
interface PW-Ether100
 ipv4 address 2.2.2.1 255.255.255.0
 attach generic-interface-list ALL
!
route-policy PASS-ALL
  pass
end-policy
!
!
!
router isis 1337
 is-type level-2-only
 net 49.0010.0000.0000.0011.00
 address-family ipv4 unicast
  metric-style wide
  mpls traffic-eng level-2-only
  mpls traffic-eng router-id Loopback0
 !
 interface Loopback0
  passive
  address-family ipv4 unicast
  !
 !
 interface GigabitEthernet0/0/0/0
  circuit-type level-2-only
  point-to-point
  address-family ipv4 unicast
   fast-reroute per-prefix
   metric 10000 level 2
   mpls ldp sync
  !
 !
 interface GigabitEthernet0/0/0/1
  circuit-type level-2-only
  point-to-point
  address-family ipv4 unicast
   fast-reroute per-prefix
   metric 10000 level 2
   mpls ldp sync
  !
 !
!
l2vpn
 xconnect group g
  p2p p_pwhe_notworking
   interface PW-Ether100
   neighbor ipv4 10.0.0.21 pw-id 100100
  !
 !
!
generic-interface-list ALL
 interface GigabitEthernet0/0/0/0
 interface GigabitEthernet0/0/0/1
!
mpls ldp
 router-id 10.0.0.11
 address-family ipv4
 !
 interface GigabitEthernet0/0/0/0
 !
 interface GigabitEthernet0/0/0/1
 !
!
mpls ip-ttl-propagate disable forwarded
ssh server v2
ssh server netconf vrf default
end

ACC1:
hostname ACC1
!
grpc
 port 57400
 address-family dual
!
address-family ipv4 unicast
!
address-family ipv6 unicast
!
line default
 transport input ssh
!
call-home
 service active
 contact smart-licensing
 profile CiscoTAC-1
  active
  destination transport-method email disable
  destination transport-method http
 !
!
netconf-yang agent
 ssh
!
interface MgmtEth0/RP0/CPU0/0
 ipv4 address 172.100.100.31 255.255.255.0
 ipv6 address 2001:172:100:100::31/80
!
interface GigabitEthernet0/0/0/0
 description ACC1->AGG1
!
interface GigabitEthernet0/0/0/0.2
 description PWHE
 ipv4 address 2.2.2.2 255.255.255.0
 encapsulation dot1q 2
!
interface GigabitEthernet0/0/0/0.100
 description PW -> ACC 2 working
 ipv4 address 1.1.1.1 255.255.255.0
 encapsulation dot1q 100
!
interface GigabitEthernet0/0/0/0.666
 description PW -> ACC 2 not working
 ipv4 address 3.3.3.1 255.255.255.0
 encapsulation dot1q 666
!
ssh server v2
ssh server netconf vrf default
end

ACC2:
hostname ACC2
!
grpc
 port 57400
 address-family dual
!
address-family ipv4 unicast
!
address-family ipv6 unicast
!
line default
 transport input ssh
!
call-home
 service active
 contact smart-licensing
 profile CiscoTAC-1
  active
  destination transport-method email disable
  destination transport-method http
 !
!
netconf-yang agent
 ssh
!
interface MgmtEth0/RP0/CPU0/0
 ipv4 address 172.100.100.32 255.255.255.0
 ipv6 address 2001:172:100:100::32/80
!
interface GigabitEthernet0/0/0/0
 description ACC1->AGG1
!
interface GigabitEthernet0/0/0/0.100
 description PW to ACC 1 working
 ipv4 address 1.1.1.2 255.255.255.0
 encapsulation dot1q 100
!
interface GigabitEthernet0/0/0/0.666
 description PW to ACC 1 not working
 ipv4 address 3.3.3.2 255.255.255.0
 encapsulation dot1q 333
!
ssh server v2
ssh server netconf vrf default
end

[Harold Ritter]

Hi @SA2 ,

Thanks for the additional information. Here's what I can see so far.

Scenario 1: (same vlan on both sides)

Glad that it worked. I had never tested it and there are probably many limitations (such as explicit-null not working)

Scenario 2: (different vlans on either side)

It appears that the "rewrite" command has no effect whatsoever. This is most probably related to the XRd platform, as it definitely works on XRv9k.

Scenario 3:

This is probably just a side effect of the rewrite command not working.

Regards,

[MHM Cisco World]

Last time I was suspect about label0 and dont inform you' 

Now I hope I will guide you to correct answer 

You need 

Rewrite ingress tag translate 1-to-1 dot1q x

this command we use if vlan is different' and use internetwork vlan

MHM

[SA2]

@MHM Cisco World Hi we're trying to test in a lab environment that the services that we are configuring are working as intended.
The current usecase needs this to be identical to our production configuration, so changing this to the suggested configuration is not an option for us currently.

However, I just tested our setup but using XRv9K for the AGGs while all the other nodes are XRD, and then the setup works.
Atleast as far as Ive managed to test as of now.

[SA2]

@Harold RitterThank you, that seems to be what is happening yes.
I tried using the Xrv9K as the ServiceEdge but it did not support PW-Ether interfaces.
Tried both the VRPE and VRR versions.

Do you know of Is this simply an issue of not installing the correct modules? or is this not supported?
With the qemu image I didn't find a way to install additional software modules.

[Ramblin Tech]

XRv9K supports VLAN-tagged EoMPLS PWs. In CML, I have modeled a BNG use-case by virtually looping back the VLAN-tagged PW attachment circuit (via an unmanaged switch) into a second XRv9K interface with L3 subinterfaces to emulate the basic functionality of a PWHE (IPoE subscriber management applied to subinterfaces; the far-end of the PW attaching to subscriber CPE). 

This might be similar to what you want to test with your provisioning system. 

[SA2]

@Ramblin Tech That's an interesting suggestion, however for our current usecase we need it to mirror our production configuration, so doing this isn't the best option currently.

However I just tested our setup with XRv9K as the AGG nodes while all the others remain XRD and then it works as far as I've managed to test it as of now.

[Harold Ritter]

Hi @SA2 ,

I am glad to hear that it now works with a mix of XRv9k and XRd. As gas ads I know, the PWHE feature is not available on the XRv9k platform.

Regards,

[PremanAsia]

Same Here,,Still looking for solution
Regards PremanAsia