cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
869
Views
4
Helpful
31
Replies

IPSec VPN Tunnel Lan-to-Lan decrypt count zero

El Rondo
Level 1
Level 1

Hi, 

I have one issue with IPSec tunnel Lan-to-Lan between ASA 5525x (v9.8) and ASA FPR 2110 (v9.16). My Tunnel is up but ping between each client was not successful. Both peer status sh cry isakmp sa in "MM_ACTIVE".
I ran packet-tracer icmp between peer and result shows ALLOW for every phase 1 and 2. Debug command sh cry ipsec sa shows packets encrypt is non zero but decrypt is zero for both peer.

Spoiler
#pkts encaps: 80, #pkts encrypt: 80, #pkts digest: 80
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0

I suspected NAT was the issue and until now I haven't found the root cause. Hopefully someone who had the solution could help me to fix the issue.

31 Replies 31

balaji.bandi
Hall of Fame
Hall of Fame

This means it one way communication,. when you run the debug other side do you see the packets ? (its long config need to look what is wrong, i will look and suggest if i find any obvious on the config?)

May be if you think NAT issue, then avoid NAT source and Destination from NAT config :

example :

https://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-firewalls/215884-configure-a-site-to-site-vpn-tunnel-with.html

https://www.packetswitch.co.uk/cisco-asa-site-to-site-vpn/

command troubleshooting tips :

https://www.cisco.com/c/en/us/support/docs/security/asa-5500-x-series-next-generation-firewalls/81824-common-ipsec-trouble.html

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

Indeed it NAT issue

Do

Nat (inside'outside) source static object-local-lan object-local-lan destiantion object-remote-lan object-remote-lan

Do this in both fw

MHM

NAT exemption were already in the config.

SITE HQ

object network SERVER
subnet 172.16.4.0 255.255.255.0

object network PT_SERVER
subnet 172.31.1.0 255.255.255.0

nat (LAN,WAN) source static SERVER SERVER destination static PT_SERVER PT_SERVER no-proxy-arp route-lookup

SITE PT

object network SERVER
subnet 172.31.1.0 255.255.255.0

object network HQ_SERVER
subnet 172.16.4.0 255.255.255.0

nat (LAN,WAN) source static SERVER SERVER destination static HQ_SERVER HQ_SERVER no-proxy-arp route-lookup

Can you do packet-tracer from HQ to PT?

MHM

packet-tracer input LAN icmp 172.16.4.16 8 0 172.31.1.3 detailed 
 
Phase: 1
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
nat (LAN,WAN) source static SERVER SERVER destination static PT_SERVER PT_SERVER
Additional Information:
NAT divert to egress interface WAN
Untranslate 172.31.1.3/0 to 172.31.1.3/0
 
Phase: 2
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group INSIDE_ACCESS_IN in interface LAN
access-list INSIDE_ACCESS_IN extended permit object-group IP_Allow object-group ALL_INSIDE_LAN any log 
object-group protocol IP_Allow
 protocol-object ip
 protocol-object pim
 protocol-object gre
 protocol-object esp
 protocol-object ah
 protocol-object ospf
 protocol-object nos
object-group network ALL_INSIDE_LAN
 description: # All vlan from inside interface
 network-object object LEVEL_G
 network-object object LEVEL_2
 network-object object LEVEL_3
 network-object object LEVEL_4
 network-object object LEVEL_5
 network-object object LEVEL_6
 network-object object WIFI
 network-object host 172.17.9.1
 network-object object SERVER
 network-object object WIFI_B
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x2aaac8f84a30, priority=13, domain=permit, deny=false
hits=4293110, user_data=0x2aaabdbd79c0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=LAN, output_ifc=any
 
Phase: 3
Type: CONN-SETTINGS
Subtype: 
Result: ALLOW 
Config:
class-map class-default
 match any
policy-map global_policy
 class class-default
  set connection decrement-ttl
service-policy global_policy global
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x2aaacf306280, priority=7, domain=conn-set, deny=false
hits=16782303, user_data=0x2aaacf3037c0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=LAN, output_ifc=any
 
Phase: 4
Type: NAT
Subtype: 
Result: ALLOW
Config:
nat (LAN,WAN) source static SERVER SERVER destination static PT_SERVER PT_SERVER
Additional Information:
Static translate 172.16.4.16/0 to 172.16.4.16/0
 Forward Flow based lookup yields rule:
 in  id=0x7f4f8e3504d0, priority=6, domain=nat, deny=false
hits=68, user_data=0x7f4f8e34ff50, cs_id=0x0, flags=0x0, protocol=0
src ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=172.31.1.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0
input_ifc=LAN, output_ifc=WAN
 
Phase: 5
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x2aaac7eb3e40, priority=0, domain=nat-per-session, deny=true
hits=175400579, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=any
 
Phase: 6
Type: IP-OPTIONS
Subtype: 
Result: ALLOW
Config:       
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x2aaac8bc2cb0, priority=0, domain=inspect-ip-options, deny=true
hits=16887969, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=LAN, output_ifc=any
 
Phase: 7
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x2aaac8bc24c0, priority=66, domain=inspect-icmp-error, deny=false
hits=262174, user_data=0x2aaac8bc1a30, cs_id=0x0, use_real_addr, flags=0x0, protocol=1
src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=any, dscp=0x0
input_ifc=LAN, output_ifc=any
 
Phase: 8
Type: VPN
Subtype: encrypt
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 out id=0x7f4f8eb30410, priority=70, domain=encrypt, deny=false
hits=11, user_data=0x8cad1c, cs_id=0x2aaac99988d0, reverse, flags=0x0, protocol=0
src ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=172.31.1.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=WAN
 
Phase: 9
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (LAN,WAN) source static SERVER SERVER destination static PT_SERVER PT_SERVER
Additional Information:
 Forward Flow based lookup yields rule:
 out id=0x7f4f8e352e90, priority=6, domain=nat-reverse, deny=false
hits=68, user_data=0x7f4f8e350050, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=172.31.1.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0
input_ifc=LAN, output_ifc=WAN
              
Phase: 10
Type: FLOW-CREATION
Subtype: 
Result: ALLOW
Config:
Additional Information:
New flow created with id 171302415, packet dispatched to next module
Module information for forward flow ...
snp_fp_tracer_drop
snp_fp_inspect_ip_options
snp_fp_inspect_icmp
snp_fp_translate
snp_fp_adjacency
snp_fp_encrypt
snp_fp_fragment
snp_ifc_stat
 
Module information for reverse flow ...
 
Result:
input-interface: LAN
input-status: up
input-line-status: up
output-interface: WAN
output-status: up
output-line-status: up
Action: allow

 

packet-tracer input WAN icmp  172.31.1.3 8 0  172.16.4.16 detailed

Do this packet tracer also 

MHM

do you mean run this at site HQ ?

 

Yes' HQ and PT

And swapping the IP when you run it in PT

MHM

SITE HQ

packet-tracer input WAN icmp  172.31.1.3 8 0 172.16.4.16 detailed 
 
Phase: 1
Type: ACCESS-LIST
Subtype: 
Result: ALLOW
Config:
Implicit Rule
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x2aaac8b59700, priority=1, domain=permit, deny=false
hits=927861548, user_data=0x0, cs_id=0x0, l3_type=0x8
src mac=0000.0000.0000, mask=0000.0000.0000
dst mac=0000.0000.0000, mask=0100.0000.0000
input_ifc=WAN, output_ifc=any
 
Phase: 2
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
nat (LAN,WAN) source static SERVER SERVER destination static PT_SERVER PT_SERVER
Additional Information:
NAT divert to egress interface LAN
Untranslate 172.16.4.16/0 to 172.16.4.16/0
 
Phase: 3      
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group OUTSIDE_ACCESS_IN in interface WAN
access-list OUTSIDE_ACCESS_IN extended permit icmp any object-group ALL_INSIDE_LAN object-group ICMP_Allow 
object-group network ALL_INSIDE_LAN
 description: # All vlan from inside interface
 network-object object LEVEL_G
 network-object object LEVEL_2
 network-object object LEVEL_3
 network-object object LEVEL_4
 network-object object LEVEL_5
 network-object object LEVEL_6
 network-object object WIFI
 network-object host 172.17.9.1
 network-object object SERVER
 network-object object WIFI_B
object-group icmp-type ICMP_Allow
 icmp-object echo
 icmp-object echo-reply
 icmp-object time-exceeded
 icmp-object traceroute
 icmp-object unreachable
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x2aaac8e651b0, priority=13, domain=permit, deny=false
hits=85, user_data=0x2aaabdc046c0, cs_id=0x0, use_real_addr, flags=0x0, protocol=1
src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=8, tag=any
dst ip/id=172.16.4.0, mask=255.255.255.0, icmp-code=0, tag=any, dscp=0x0
input_ifc=WAN, output_ifc=any
 
Phase: 4
Type: CONN-SETTINGS
Subtype: 
Result: ALLOW
Config:
class-map class-default
 match any
policy-map global_policy
 class class-default
  set connection decrement-ttl
service-policy global_policy global
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x2aaacf3052a0, priority=7, domain=conn-set, deny=false
hits=163773516, user_data=0x2aaacf3037c0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
        src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=WAN, output_ifc=any
 
Phase: 5
Type: NAT
Subtype: 
Result: ALLOW
Config:
nat (LAN,WAN) source static SERVER SERVER destination static PT_SERVER PT_SERVER
Additional Information:
Static translate 172.31.1.3/0 to 172.31.1.3/0
 Forward Flow based lookup yields rule:
 in  id=0x7f4f8e3526d0, priority=6, domain=nat, deny=false
hits=1, user_data=0x7f4f8e350050, cs_id=0x0, flags=0x0, protocol=0
src ip/id=172.31.1.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0
input_ifc=WAN, output_ifc=LAN
 
Phase: 6
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x2aaac7eb3e40, priority=0, domain=nat-per-session, deny=true
hits=176642530, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=any, output_ifc=any
 
Phase: 7
Type: IP-OPTIONS
Subtype: 
Result: ALLOW
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x2aaac8b61bc0, priority=0, domain=inspect-ip-options, deny=true
hits=172928612, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0
input_ifc=WAN, output_ifc=any
 
Phase: 8
Type: VPN
Subtype: ipsec-tunnel-flow
Result: DROP  
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7f4f9b2d4710, priority=70, domain=ipsec-tunnel-flow, deny=false
hits=50, user_data=0x8f943c, cs_id=0x2aaac99988d0, reverse, flags=0x0, protocol=0
src ip/id=172.31.1.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0
input_ifc=WAN, output_ifc=any
 
Result:
input-interface: WAN
input-status: up
input-line-status: up
output-interface: LAN
output-status: up
output-line-status: up
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule
 
 
SITE PT

packet-tracer input WAN icmp 172.16.4.16 8 0 172.31.1.3 detailed

Phase: 1
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
Implicit Rule
Additional Information:
Forward Flow based lookup yields rule:
in id=0x5571b43600, priority=1, domain=permit, deny=false
hits=1327993280, user_data=0x0, cs_id=0x0, l3_type=0x8
src mac=0000.0000.0000, mask=0000.0000.0000
dst mac=0000.0000.0000, mask=0100.0000.0000
input_ifc=WAN, output_ifc=any

Phase: 2
Type: UN-NAT
Subtype: static
Result: ALLOW
Config:
nat (LAN,WAN) source static SERVER SERVER destination static HQ_SERVER HQ_SERVER
Additional Information:
NAT divert to egress interface LAN
Untranslate 172.31.1.3/0 to 172.31.1.3/0

Phase: 3
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Config:
access-group OUTSIDE_ACCESS_IN in interface WAN
access-list OUTSIDE_ACCESS_IN extended permit icmp any object-group ALL_INSIDE_LAN object-group ICMP_Allow
object-group network ALL_INSIDE_LAN
network-object object WIFI_BENGKEL
network-object object WIFI_JTK
network-object object WIFI_JKE_A
network-object object WIFI_JKE_B
network-object object WIFI_HEP
network-object object WIFI_HOSTEL_LELAKI
network-object object WIFI_HOSTEL_PEREMPUAN
network-object object JTP
network-object object JTM
network-object object HEP
network-object object BENGKEL
network-object object CISCO
network-object object JTK
network-object object JKE_A
network-object object JKE_B
network-object object SERVER
network-object object WIFI_JTP
network-object object WIFI_JTM
network-object object WIFI_DEWAN_A
network-object object WIFI_DEWAN_B
network-object object JTA
network-object object JKP
network-object object WIFI_JTA_JKP
object-group icmp-type ICMP_Allow
icmp-object echo
icmp-object echo-reply
icmp-object time-exceeded
icmp-object traceroute
icmp-object unreachable
Additional Information:
Forward Flow based lookup yields rule:
in id=0x5576ba40d0, priority=13, domain=permit, deny=false
hits=286215, user_data=0x55601f8480, cs_id=0x0, use_real_addr, flags=0x0, protocol=1
src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=8, tag=any
dst ip/id=172.31.1.0, mask=255.255.255.0, icmp-code=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=WAN, output_ifc=any

Phase: 4
Type: NAT
Subtype:
Result: ALLOW
Config:
nat (LAN,WAN) source static SERVER SERVER destination static HQ_SERVER HQ_SERVER
Additional Information:
Static translate 172.16.4.16/0 to 172.16.4.16/0
Forward Flow based lookup yields rule:
in id=0x557a8eff30, priority=6, domain=nat, deny=false
hits=0, user_data=0x5573700210, cs_id=0x0, flags=0x0, protocol=0
src ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=172.31.1.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=WAN, output_ifc=LAN

Phase: 5
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x556e727770, priority=0, domain=nat-per-session, deny=true
hits=19111215, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=any

Phase: 6
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x5571b4bfe0, priority=0, domain=inspect-ip-options, deny=true
hits=13372219, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=WAN, output_ifc=any

Phase: 7
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x55736fdb20, priority=70, domain=ipsec-tunnel-flow, deny=false
hits=1, user_data=0x11603d4, cs_id=0x5574abf0f0, reverse, flags=0x0, protocol=0
src ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=172.31.1.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=WAN, output_ifc=any

Phase: 8
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
class-map inspection_default
match default-inspection-traffic
policy-map global_policy
class inspection_default
inspect icmp
service-policy global_policy global
Additional Information:
Forward Flow based lookup yields rule:
in id=0x557518c6a0, priority=70, domain=inspect-icmp, deny=false
hits=3785419, user_data=0x557518bb80, cs_id=0x0, use_real_addr, flags=0x0, protocol=1
src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=WAN, output_ifc=any

Phase: 9
Type: INSPECT
Subtype: np-inspect
Result: ALLOW
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x5571b4b7f0, priority=66, domain=inspect-icmp-error, deny=false
hits=3785419, user_data=0x5571b4ae40, cs_id=0x0, use_real_addr, flags=0x0, protocol=1
src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=WAN, output_ifc=any

Phase: 10
Type: NAT
Subtype: rpf-check
Result: ALLOW
Config:
nat (LAN,WAN) source static SERVER SERVER destination static HQ_SERVER HQ_SERVER
Additional Information:
Forward Flow based lookup yields rule:
out id=0x557726a4c0, priority=6, domain=nat-reverse, deny=false
hits=1, user_data=0x55784abc90, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=172.31.1.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=WAN, output_ifc=LAN

Phase: 11
Type: NAT
Subtype: per-session
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x556e727770, priority=0, domain=nat-per-session, deny=true
hits=19111217, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=any

Phase: 12
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x5572863330, priority=0, domain=inspect-ip-options, deny=true
hits=20358197, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=LAN, output_ifc=any

Phase: 13
Type: VPN
Subtype: encrypt
Result: ALLOW
Config:
Additional Information:
Reverse Flow based lookup yields rule:
out id=0x5577dfb4f0, priority=70, domain=encrypt, deny=false
hits=1, user_data=0x115f8b4, cs_id=0x5574abf0f0, reverse, flags=0x0, protocol=0
src ip/id=172.31.1.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=WAN

Result:
input-interface: WAN
input-status: up
input-line-status: up
output-interface: LAN
output-status: up
output-line-status: up
Action: drop
Drop-reason: (ipsec-spoof) IPSEC Spoof detected, Drop-location: frame 0x000000aaacef4928 flow (NA)/NA

It clear now issue in HQ 

And I found this 

policy-route route-map MYGOVNET

So the traffic not use defualt route but use pbr' I see it to any' but can you exclude traffic from server to server form pass via pbr

MHM

Both HQ site and PT site have multiple ISP.

ISP 1 Mygovnet (public IP) - specifically for Wired and server subnet
ISP 2 Unifi (Broadband) - specifically for Wifi subnet

I used PBR to force specific traffic (wired and server subnet) routed through mygovnet ISP. If I exclude server subnet from the PBR it wont pass through mygovnet isp but instead use unifi isp.

 

HQ

 
Phase: 8
Type: VPN
Subtype: ipsec-tunnel-flow
Result: DROP  <<- drop
Config:
Additional Information:
 Forward Flow based lookup yields rule:
 in  id=0x7f4f9b2d4710, priority=70, domain=ipsec-tunnel-flow, deny=false
hits=50, user_data=0x8f943c, cs_id=0x2aaac99988d0, reverse, flags=0x0, protocol=0
src ip/id=172.31.1.0, mask=255.255.255.0, port=0, tag=any
dst ip/id=172.16.4.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0
input_ifc=WAN, output_ifc=any
 
Check 
SHOW ASP TABLE VPN-CONTEXT DETAIL | begin 0x8f943c
see if this VPN is for same peer or not ?
 
PT

Drop-reason: (ipsec-spoof) IPSEC Spoof detected, Drop-location: frame 0x000000aaacef4928 flow (NA)/NA

this error is appear if the FW receive un-encrypt traffic, 
this can be is HQ can reach PT vis Second ISP, the one that not config for IPsec
to make sure 
capture CAP interface <second ISP> match <HQ LAN subnet> <PT LAN subnet>
then ping see if capture show anything 
MHM

 

HQ

SHOW ASP TABLE VPN-CONTEXT DETAIL | begin 0x8f943c

no results

This command run without 0x8f943c

SHOW ASP TABLE VPN-CONTEXT DETAIL

VPN CTX = 0x00A18CFC

Peer IP = 172.31.1.0
Pointer = 0x71B2D210
State = UP
Flags = DECR+ESP
SA = 0x02879833
SPI = 0xF6E97E01
Group = 1
Pkts = 0
Bad Pkts = 0
Bad SPI = 0
Spoof = 0
Bad Crypt0 = 0
Rekey Pkt = 1
Rekey Call = 1
VPN Filter = <none>

VPN CTX = 0x00A17FE4

Peer IP = 172.31.1.0
Pointer = 0x71B2BF80
State = UP
Flags = ENCR+ESP
SA = 0x02883983
SPI = 0xB7DCDB8A
Group = 1
Pkts = 422
Bad Pkts = 0
Bad SPI = 0
Spoof = 0
Bad Crypt0 = 0
Rekey Pkt = 1
Rekey Call = 1
VPN Filter = <none>

PT

 capture CAP interface UNIFI match ip 172.16.4.0 255.255.255.0 172.31.1.0 255.255.255.0

I initiate ping request from PT subnet to HQ subnet and vice versa HQ subnet ping to PT subnet. it show capturing 0 bytes.

result:

capture CAP type raw-data interface UNIFI [Capturing - 0 bytes]
match ip 172.16.4.0 255.255.255.0 172.31.1.0 255.255.255.0

I don't think this packet tracer would work as from the WAN interface perspective it won't see the VPN traffic with the endpoints private IP addresses, it would instead see it coming from the remote peer "public" IP.