Re: troubleshooting ikev2 site to site vpn

Fartingdragon · ‎04-20-2023

I will drop configs of both ASAs soon. Or at least what I believe my peer has theirs set to.

Right now, I have tried to troubleshoot it by using show crypto and debug.

show crypto ikev2 sa
there are no ikev2 Sas

debug crypto condition peer WAN Address

debug crypto ikev2 protocol 127
debug crypto ikev2 platform 127

Both debug shows no output

I suspect my peer vpn site, gave me the wrong WAN address. Aside from the configs of both peers, is there anything else that could be helpful?

Terminal monitor was turned on when I did the above. I have enabled ikev2 on my interface I want the traffic to go through.

Since then I have did terminal no monitor and no debug all in case I forget. But, it shows nothing.

MHM Cisco World · ‎04-20-2023

Did you try initate traffic?

Fartingdragon · ‎04-20-2023

I tried pinging their wan and all packets received?

MHM Cisco World · ‎04-20-2023

You must ping behind Asa, using source local lan and destination remote lan of acl of vpn.

Fartingdragon · ‎04-26-2023

After trying to ping them with ICMP packets, it looks like the VPN was up and active. It said

"ASA-1/act# show crypto ikev2 sa

IKEv2 SAs:

Session-id:1, Status:UP-ACTIVE, IKE count:1, CHILD count:1

Tunnel-id Local Remote Status Role

4098985 50.50.50.50/500 51.51.51.51/500 READY INITIATOR

Encr: AES-CBC, keysize: 256, Hash: SHA96, DH Grp:5, Auth sign: PSK, Auth verify: PSK

Life/Active Time: 86400/533 sec

Child sa: local selector 10.121.0.0/0 - 10.121.255.255/65535

remote selector 10.1.21.0/0 - 10.1.21.255/65535

ESP spi in/out: 0x695dfc7f/0xb7116de0 "

However, any other kind of packet, tells me the packets are being dropped by an implicit ACL rule. After trying a few things, I can no longer get the vpn active again.

This is the config so far on our ASA where our WAN is for example 50.50.50.50 and peer local network is 10.221.0.0 /16 while their WAN is 51.51.51.51 and remote peer network is 10.1.21.0 /24:

name 50.50.50.50 fw_1_ext
!
interface GigabitEthernet0/1
 nameif ISP_2
 security-level 0
 ip address fw_1_ext 255.255.255.240 
!
object network fw_1_ext
 host 50.50.50.50
 
nat (inside,outside) source dynamic IntAllSeg interface
nat (inside,ISP_2) source dynamic IntAllSeg interface

object network RemoteBrotherCoNetwork
 subnet 10.1.21.0 255.255.255.0
 description RemoteBrotherCoNetwork
 
object network LocalBrotherCoNetwork
 subnet 10.221.0.0 255.255.0.0
 
access-list ISP_2_Interface_cryptomap extended permit ip object LocalBrotherCoNetwork object RemoteBrotherCoNetwork 
 
nat (inside,ISP_2_Interface) source static LocalBrotherCoNetwork LocalBrotherCoNetwork destination static RemoteBrotherCoNetwork RemoteBrotherCoNetwork no-proxy-arp route-lookup
 
route ISP_2_Interface 10.1.21.0 255.255.255.0 51.51.51.51 1
 
crypto ipsec ikev2 ipsec-proposal AES256
 protocol esp encryption aes-256
 protocol esp integrity sha-1 md5
 
crypto map ISP_2_Interface_map 3 match address ISP_2_Interface_cryptomap
crypto map ISP_2_Interface_map 3 set peer 51.51.51.51
crypto map ISP_2_Interface_map 3 set ikev2 ipsec-proposal AES256
crypto map ISP_2_Interface_map interface ISP_2_Interface

crypto ikev2 policy 1
 encryption aes-256
 integrity sha
 group 5 2
 prf sha
 lifetime seconds 86400
crypto ikev2 enable ISP_2_Interface

group-policy GroupPolicy_51.51.51.51 internal
group-policy GroupPolicy_51.51.51.51 attributes
 vpn-tunnel-protocol ikev2 
 
tunnel-group 51.51.51.51 type ipsec-l2l
tunnel-group 51.51.51.51 general-attributes
 default-group-policy GroupPolicy_51.51.51.51
tunnel-group 51.51.51.51 ipsec-attributes
 ikev2 remote-authentication pre-shared-key password
 ikev2 local-authentication pre-shared-key password

MHM Cisco World · ‎04-27-2023

Form first view there is no issue with config'

I need to see packet tracer for this issue

Thanks

MHM

Fartingdragon · ‎04-27-2023

I found out I was missing a line for my crypto proposal is why it didn't work again. However, it only works with icmp traffic since it is implicitly blocked by something in the acl. I had to explicitly create a global access list for traffic to work. Will you need the acl list?

MHM Cisco World · ‎04-27-2023

The acl apply to inside share here

Thanks

Also show run nat i want to check something

Fartingdragon · ‎04-30-2023

access-list global_access extended permit tcp object DMZseg any object-group HTTP_HTTPS_FTP 
access-list global_access extended permit object-group Server_Access_Ports object-group Server_Access any 
access-list global_access extended permit ip object-group InternalDataSegments object-group ABCResources 
access-list global_access extended permit object-group dmz_29 object DMZ_SMTP_GW object-group DM_INLINE_NETWORK_1 
access-list global_access extended permit icmp any any object-group DM_INLINE_ICMP_1 
access-list global_access extended permit icmp any any
access-list global_access extended permit ip object cctv any 
access-list inside_access_in extended permit ip any any 
access-list outside_access_in extended permit udp any any gt 30000 
access-list outside_access_in extended permit object-group Traceroute_Allow any any 
access-list outside_access_in extended permit tcp any object CRL object-group HTTP_HTTPS 
access-list outside_access_in extended permit object-group exch_01 any host 10.x.x.x
access-list outside_access_in extended permit tcp any object cctv1_32 eq https 
access-list outside_access_in extended permit tcp any object cctv2_32 eq https 
access-list outside_access_in extended permit tcp any object cctv3_32 eq https 
access-list outside_access_in extended permit tcp any object cctv4_32 eq 2021 
access-list outside_access_in extended permit tcp any object cctv5_32 eq 2020 
access-list outside_access_in extended permit udp any object cctv6_32 eq 61542 
access-list Servers extended permit ip object-group Server_OWA any 
access-list ISP_2_cryptomap extended permit ip object LocalBrotherCoNetwork object RemoteBrotherCoNetwork
access-list ISP_2_access_in extended permit tcp any object CRLPAK object-group HTTP_HTTPS 
access-list ISP_2_access_in extended permit object-group DM_INLINE_SERVICE_1 any any 
access-list dmz2_access_in extended permit ip any any 
access-list sfr_redirect extended deny ip 192.168.100.0 255.255.255.0 10.100.0.0 255.255.0.0 
access-list sfr_redirect extended deny ip any object-group SFR_Deny_DMZ 
access-list sfr_redirect extended permit ip any any 
access-list dmz1_access_in extended permit object-group DM_INLINE_SERVICE_2 any any 
access-list AnyConnect-ACL extended permit tcp 192.168.100.0 255.255.255.0 any4 eq 3389 
access-list AnyConnect-ACL extended permit udp 192.168.100.0 255.255.255.0 any4 eq 3389 
access-list AnyConnect-ACL extended permit tcp 192.168.100.0 255.255.255.0 any4 eq 445 
access-list AnyConnect-ACL extended permit udp 192.168.100.0 255.255.255.0 any4 eq 445 
access-list AnyConnect-ACL extended permit udp 192.168.100.0 255.255.255.0 any4 eq domain 
access-list AnyConnect-ACL extended permit tcp 192.168.100.0 255.255.255.0 any4 eq 5585 
access-list AnyConnect-ACL extended permit icmp 192.168.100.0 255.255.255.0 any4 
access-list guest_access_in extended permit tcp any host 10.250.1.5 eq 8880 
access-list guest_access_in extended permit tcp any host 10.250.1.5 eq 8843 
access-list guest_access_in extended permit ip any 172.x.x.0 255.255.255.0 
access-list guest_access_in extended deny ip any 10.0.0.0 255.0.0.0 
access-list guest_access_in extended deny ip any 172.16.0.0 255.240.0.0 
access-list guest_access_in extended deny ip any 192.168.0.0 255.255.0.0 
access-list guest_access_in extended permit ip any any 
access-list phones_access_in extended permit ip any 172.x.x.0 255.255.255.0 
access-list phones_access_in extended deny ip any 10.0.0.0 255.0.0.0 
access-list phones_access_in extended deny ip any 172.16.0.0 255.240.0.0 
access-list phones_access_in extended deny ip any 192.168.0.0 255.255.0.0 
access-list phones_access_in extended permit ip any any 
access-list City_VPN extended permit ip host 172.x.x.12 object-group lock_track 
access-list City_VPN extended permit ip host 172.x.x.12 host 10.8.101.214 
access-list City_VPN extended permit ip host 172.x.x.11 object-group s2s-dmz2 
access-list dmz4_access_in extended permit tcp object 172.x.x.10_32 any eq https 
access-list dmz4_access_in extended permit ip object 172.x.x.10_32 any 
access-list dmz4_access_in extended permit ip object 172.x.x.11_32 any 
access-list dmz4_access_in extended permit tcp object 172.x.x.11_32 any eq https 
access-list dmz4_access_in extended permit ip object 172.x.x.12_32 any 
access-list dmz4_access_out extended deny ip 10.0.0.0 255.0.0.0 any 
access-list dmz4_access_out extended permit tcp any host 172.x.x.10 eq https 
access-list dmz4_access_out extended permit tcp any host 172.x.x.11 eq https 
access-list dmz4_access_out extended permit tcp any host 172.x.x.12 eq https 
access-list cctv extended permit ip object DataSeg12 172.x.x.0 255.255.255.0 
access-list cctv extended permit ip object DataSeg05 172.x.x.0 255.255.255.0 
access-list cctv extended deny ip 192.168.0.0 255.255.0.0 172.x.x.0 255.255.255.0 
access-list cctv extended deny ip 172.16.0.0 255.240.0.0 172.x.x.0 255.255.255.0 
access-list cctv extended deny ip 10.0.0.0 255.0.0.0 172.x.x.0 255.255.255.0 
access-list cctv extended permit ip any 172.x.x.0 255.255.255.0 
access-list cctv_in extended permit ip 172.x.x.0 255.255.255.0 object DataSeg12 
access-list cctv_in extended permit ip 172.x.x.0 255.255.255.0 object DataSeg05 
access-list cctv_in extended permit ip 172.x.x.0 255.255.255.0 172.x.x.0 255.255.255.0 
access-list cctv_in extended deny ip 172.x.x.0 255.255.255.0 192.168.0.0 255.255.0.0 
access-list cctv_in extended deny ip 172.x.x.0 255.255.255.0 172.16.0.0 255.240.0.0 
access-list cctv_in extended deny ip 172.x.x.0 255.255.255.0 10.0.0.0 255.0.0.0 
access-list cctv_in extended permit ip 172.x.x.0 255.255.255.0 any 
access-list SPLIT-TUNNEL standard permit 10.100.0.0 255.255.0.0 
access-list SPLIT-TUNNEL standard permit 10.99.0.0 255.255.0.0 
pager lines 24

Fartingdragon · ‎04-30-2023

NAT
nat (inside,outside) source static any any destination static Pool_SSL_VPN Pool_SSL_VPN no-proxy-arp route-lookup
nat (inside,outside) source static DataSeg00 DataSeg00 destination static NJIO_SUBNET NJIO_SUBNET no-proxy-arp
nat (inside,GTT_2) source static ManagementSegAll ManagementSegAll destination static NJIO_SUBNET NJIO_SUBNET no-proxy-arp
nat (inside,GTT_2) source static DataSeg00 DataSeg00 destination static NJIO_SUBNET NJIO_SUBNET no-proxy-arp
nat (CCTV,outside) source static 172.x.x.150_32 51.51.51.51_32 service udp_61542 udp_61542
nat (CCTV,outside) source static 172.x.x.222_32 51.51.51.51_32 service tcp_2020 tcp_2020
nat (CCTV,outside) source static 172.x.x.223_32 51.51.51.51_32 service tcp_2021 tcp_2021
nat (inside,CCTV) source static InternalDataSegments InternalDataSegments destination static cctv cctv no-proxy-arp
nat (guest,CCTV) source static guest guest destination static cctv cctv no-proxy-arp
nat (phones,CCTV) source static phones phones destination static cctv cctv no-proxy-arp
nat (DMZ_4,outside) source static 172.x.x.12_32 8.225.194.152_32
nat (DMZ_4,outside) source static 172.x.x.11_32 8.225.194.151_32
nat (inside,outside) source static InternalDataSegments Hide_Address_CityNet destination static s2s-dmz2 s2s-dmz2
nat (DMZ_4,outside) source static 172.x.x.10_32 8.225.194.150_32
nat (inside,outside) source static InternalDataSegments Def_local destination static l_track l_track
nat (inside,outside) source static any any destination static NETWORK_OBJ_192.168.100.0_26 NETWORK_OBJ_192.168.100.0_26 no-proxy-arp route-lookup
nat (inside,outside) source static exch_01 exch_01_nat
nat (inside,GTT_2) source static CRL CRL_Ext
nat (inside,GTT_2) source static DMZ_SMTP_GW DMZ_SMTP_GW_Ext
nat (inside,dmz2) source dynamic InternalDataSegments Hide_Address_CityNet destination static CityNetR CityNetR
nat (inside,outside) source dynamic InternalDataSegments interface destination static Vlink Vlink
nat (inside,GTT_2) source dynamic InternalDataSegments Hide_Address_Internet destination static Vlink Vlink
nat (inside,dmz1) source dynamic InternalDataSegments Hide_Address_A destination static A2 A2
nat (inside,outside) source dynamic DMZseg interface
nat (inside,GTT_2) source dynamic DMZseg interface
nat (inside,outside) source dynamic InternalAllSegments interface
nat (inside,GTT_2) source dynamic InternalAllSegments interface
nat (inside,dmz1) source dynamic InternalDataSegments Hide_Address_A destination static A A
nat (guest,outside) source dynamic OBJ-VLAN303-GUEST interface
nat (phones,outside) source dynamic OBJ-VLAN302-PHONES interface
nat (CCTV,outside) source dynamic cctv interface
nat (inside,GTT_2) source static LocalBrotherCoNetwork LocalBrotherCoNetwork destination static RemoteBrotherCoNetwork RemoteBrotherCoNetwork no-proxy-arp route-lookup

Fartingdragon · ‎05-04-2023

any luck?

MHM Cisco World · ‎05-07-2023

there are many ACL

packet-tracer input INSIDE tcp <any IP from local LAN> 1234 <any IP from remote LAN> 80 detail <<- please share output here

Fartingdragon · ‎05-11-2023

I already added the global acl. Do you want that done without it?

Fartingdragon · ‎05-13-2023

Result of the command: "packet-tracer input inside tcp 10.113.10.5 1234 10.1.10.5 80 detail"

Phase: 1

Type: ACCESS-LIST

Subtype:

Result: ALLOW

Config:

Implicit Rule

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f8647e50c50, priority=1, domain=permit, deny=false

hits=151091255281, user_data=0x0, cs_id=0x0, l3_type=0x8

src mac=0000.0000.0000, mask=0000.0000.0000

dst mac=0000.0000.0000, mask=0100.0000.0000

input_ifc=inside, output_ifc=any

Phase: 2

Type: ROUTE-LOOKUP

Subtype: Resolve Egress Interface

Result: ALLOW

Config:

Additional Information:

found next-hop REMOTE_WAN_IP using egress ifc ISP_2

Phase: 3

Type: UN-NAT

Subtype: static

Result: ALLOW

Config:

nat (inside,ISP_2) source static LocalBrotherCoNetwork LocalBrotherCoNetwork destination static RemoteBrotherCoNetworkRemoteBrotherCoNetworkno-proxy-arp route-lookup

Additional Information:

NAT divert to egress interface ISP_2

Untranslate 10.1.10.5/80 to 10.1.10.5/80

Phase: 4

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Config:

access-group inside_access_in in interface inside

access-list inside_access_in extended permit ip any any

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f864710f860, priority=13, domain=permit, deny=false

hits=1342295588, user_data=0x7f863c149000, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=inside, output_ifc=any

Phase: 5

Type: CONN-SETTINGS

Subtype:

Result: ALLOW

Config:

class-map class-default

match any

policy-map global_policy

class class-default

set connection decrement-ttl

service-policy global_policy global

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f864bed52a0, priority=7, domain=conn-set, deny=false

hits=1198823948, user_data=0x7f8648e7b3a0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=inside, output_ifc=any

Phase: 6

Type: NAT

Subtype:

Result: ALLOW

Config:

nat (inside,ISP_2) source static LocalBrotherCoNetwork LocalBrotherCoNetwork destination static RemoteBrotherCoNetworkRemoteBrotherCoNetworkno-proxy-arp route-lookup

Additional Information:

Static translate 10.113.10.5/1234 to 10.113.10.5/1234

Forward Flow based lookup yields rule:

in id=0x7f86548a2390, priority=6, domain=nat, deny=false

hits=6, user_data=0x7f86511d3250, cs_id=0x0, flags=0x0, protocol=0

src ip/id=LocalBrotherCoNetwork, mask=255.255.0.0, port=0, tag=any

dst ip/id=10.1.10.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0

input_ifc=inside, output_ifc=ISP_2

Phase: 7

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f8645cc6270, priority=0, domain=nat-per-session, deny=false

hits=2413011680, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=any, output_ifc=any

Phase: 8

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f8646c50910, priority=0, domain=inspect-ip-options, deny=true

hits=1716334922, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=inside, output_ifc=any

Phase: 9

Type: SFR

Subtype:

Result: ALLOW

Config:

class-map sfr

match access-list sfr_redirect

policy-map global_policy

class sfr

sfr fail-open monitor-only

service-policy global_policy global

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f864877ee80, priority=71, domain=sfr, deny=false

hits=1611696227, user_data=0x7f86471d0990, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=inside, output_ifc=any

Phase: 10

Type: FOVER

Subtype: standby-update

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0x7f8646d1a720, priority=20, domain=lu, deny=false

hits=1059657358, user_data=0x0, cs_id=0x0, flags=0x0, protocol=6

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=inside, output_ifc=any

Phase: 11

Type: VPN

Subtype: encrypt

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

out id=0x7f8654737390, priority=70, domain=encrypt, deny=false

hits=2, user_data=0xcb61244c, cs_id=0x7f8652ab65b0, reverse, flags=0x0, protocol=0

src ip/id=LocalBrotherCoNetwork, mask=255.255.0.0, port=0, tag=any

dst ip/id=10.1.10.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0

input_ifc=any, output_ifc=ISP_2

Phase: 12

Type: NAT

Subtype: rpf-check

Result: ALLOW

Config:

nat (inside,ISP_2) source static LocalBrotherCoNetwork LocalBrotherCoNetwork destination static RemoteBrotherCoNetworkRemoteBrotherCoNetworkno-proxy-arp route-lookup

Additional Information:

Forward Flow based lookup yields rule:

out id=0x7f8654967f40, priority=6, domain=nat-reverse, deny=false

hits=7, user_data=0x7f8654a3da80, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip/id=LocalBrotherCoNetwork, mask=255.255.0.0, port=0, tag=any

dst ip/id=10.1.10.0, mask=255.255.255.0, port=0, tag=any, dscp=0x0

input_ifc=inside, output_ifc=ISP_2

Phase: 13

Type: VPN

Subtype: ipsec-tunnel-flow

Result: ALLOW

Config:

Additional Information:

Reverse Flow based lookup yields rule:

in id=0x7f86536701b0, priority=70, domain=ipsec-tunnel-flow, deny=false

hits=2, user_data=0xcb61438c, cs_id=0x7f8652ab65b0, reverse, flags=0x0, protocol=0

src ip/id=10.1.10.0, mask=255.255.255.0, port=0, tag=any

dst ip/id=LocalBrotherCoNetwork, mask=255.255.0.0, port=0, tag=any, dscp=0x0

input_ifc=ISP_2, output_ifc=any

Phase: 14

Type: NAT

Subtype: per-session

Result: ALLOW

Config:

Additional Information:

Reverse Flow based lookup yields rule:

in id=0x7f8645cc6270, priority=0, domain=nat-per-session, deny=false

hits=2413011682, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=any, output_ifc=any

Phase: 15

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Reverse Flow based lookup yields rule:

in id=0x7f8646b091a0, priority=0, domain=inspect-ip-options, deny=true

hits=259659095, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0

input_ifc=ISP_2, output_ifc=any

Phase: 16

Type: FLOW-CREATION

Subtype:

Result: ALLOW

Config:

Additional Information:

New flow created with id 1819480064, packet dispatched to next module

Module information for forward flow ...

snp_fp_tracer_drop

snp_fp_inspect_ip_options

snp_fp_tcp_normalizer

snp_sfr

snp_fp_translate

snp_fp_tcp_normalizer

snp_fp_adjacency

snp_fp_encrypt

snp_fp_fragment

snp_ifc_stat

Module information for reverse flow ...

snp_fp_tracer_drop

snp_fp_inspect_ip_options

snp_fp_ipsec_tunnel_flow

snp_fp_tcp_normalizer

snp_fp_translate

snp_sfr

snp_fp_tcp_normalizer

snp_fp_adjacency

snp_fp_fragment

snp_ifc_stat

Result:

input-interface: inside

input-status: up

input-line-status: up

output-interface: ISP_2

output-status: up

output-line-status: up

Action: allow

When I do a packet tracer from Inside tcp <any IP from local LAN> 21 <any IP from remote LAN> 443

The ACL that allowed it is

this ACL

config
access-group ISP_2_access_in in interface ISP_2 
access-list ISP_2_access_in extended permit object-group DM_INLINE_SERVICE_1 any any 
object-group service DM_INLINE_SERVICE_1 
service-object object SSL 
service-object tcp destination eq https 
service-object tcp destination eq imap4 
service-object tcp destination eq pop3 
service-object tcp destination eq smtp 
service-object icmp echo-reply

MHM Cisco World · ‎05-18-2023

that explain issue if I am right.
you must permit the L2L VPN UDP port 500/4500
I know it used by control ACL but I think also is effect by interface ACL.
I will run lab and check this point.