VPN site-to-site (Amazon Web Services) : NAT Exempt

[helpmeplss]

Dear Everyone,

Cisco Firepower Extensible Operating System (FX-OS) v2.12.0 (build 499)
Cisco Firepower 2130 Threat Defense v7.2.4 (build 165)
Model : Cisco Firepower 2130 Threat Defense (77) Version 7.2.4 (Build 165)

I need help configuring an IPsec VPN between my network and the AWS site-to-site. As shown in the attached photo, the connection protocol IKEv2 IPsecOverNatT is enabled, and both ends agree on this protocol. Initially, there are no problems with the connection, and I can ping and access the EC2 instances at the AWS remote site.

However, the problem arises when I notice that the connection between this remote site becomes intermittent, with ping requests timing out every 30 seconds. This is unacceptable. I also have VPN connections set up as site-to-site, using a different protocol, IKEv1.

I suspect that this could be due to a NATting issue since my protocol is using IPsecOverNatT, which shouldn't have NAT enabled.

My question is, how do I configure NAT exemption for this? It's worth noting that I have other VPN connections set up as site-to-site using a different protocol, IKEv1, and they are also using the same ISP line. Additionally, I have several existing NAT rules. I've attached a screenshot of the NAT rules, with the highlighted rule being for AWS NATting.

1. gnet_aws: 10.16.0.13 - This is the private IP for the remote site.
2. gnet_lmt_to_aws: 172.21.0.0/16 - This is the internal private IP range.

It's important to mention that the AWS documentation states that the MAIN SA Selector should be CIDR 0.0.0.0, else it will encounter a problem with TS_UNACCEPTABLE during Phase 2 rekeying. I have managed to solve this issue, but I am now facing problems with NAT exemption.

Again, how do I configure NAT exemption for this? The CIDR is too broad, and the existing rule might cause conflicts. Also, how do I add an exemption? Should I specify only the remote side address (EC2 Private IP) in another group object?

Options: A. Should I create another group object and specify only the remote side address (EC2 Private IP)?

Let me know if you need more information. Your help is greatly appreciated

[MHM Cisco World]

can you share the 
packet-tracer for traffic from LAN to LAN 
share it here 

MHM

[helpmeplss]

Hi @MHM Cisco World  Thank you for reply here is Internal IP to Remote Site. 

> packet-tracer input vlan20-fwmgt tcp (172.21.11.26) (1024) 10.17.0.159 www detailed

Phase: 1
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Elapsed time: 29855 ns
Config:
Implicit Rule
Additional Information:
Forward Flow based lookup yields rule:
in id=0x559c8dc2d0, priority=1, domain=permit, deny=false
hits=14255095, user_data=0x0, cs_id=0x0, l3_type=0x8
src mac=0000.0000.0000, mask=0000.0000.0000
dst mac=0000.0000.0000, mask=0100.0000.0000
input_ifc=vlan20-fwmgt, output_ifc=any

Phase: 2
Type: UN-NAT
Subtype: static
Result: ALLOW
Elapsed time: 20472 ns
Config:
nat (TM_DOME1,any) source static gnet_aws gnet_aws destination static gnet_lmt_to_aws gnet_lmt_to_aws no-proxy-arp
Additional Information:
NAT divert to egress interface TM_DOME1(vrfid:0)
Untranslate 10.17.0.159/80 to 10.17.0.159/80

Phase: 3
Type: ACCESS-LIST
Subtype: log
Result: ALLOW
Elapsed time: 9894 ns
Config:
access-group CSM_FW_ACL_ global
access-list CSM_FW_ACL_ advanced permit ipinip any any rule-id 268444685
access-list CSM_FW_ACL_ remark rule-id 268444685: PREFILTER POLICY: No Prefilter
access-list CSM_FW_ACL_ remark rule-id 268444685: RULE: DEFAULT TUNNEL ACTION RULE
Additional Information:
This packet will be sent to snort for additional processing where a verdict will be reached
Forward Flow based lookup yields rule:
in id=0x55a4d7ee10, priority=12, domain=permit, deny=false
hits=179600, user_data=0x5579de1800, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, ifc=vlan20-fwmgt(vrfid:0)
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, ifc=TM_DOME1(vrfid:0), vlan=0, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=any

Phase: 4
Type: CONN-SETTINGS
Subtype:
Result: ALLOW
Elapsed time: 9894 ns
Config:
class-map class_map_traceroute
match access-list traceroute
policy-map global_policy
class class_map_traceroute
set connection decrement-ttl
service-policy global_policy global
Additional Information:
Forward Flow based lookup yields rule:
in id=0x559e9524d0, priority=7, domain=conn-set, deny=false
hits=952432, user_data=0x55a39812f0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=vlan20-fwmgt(vrfid:0), output_ifc=any

Phase: 5
Type: NAT
Subtype:
Result: ALLOW
Elapsed time: 9894 ns
Config:
nat (TM_DOME1,any) source static gnet_aws gnet_aws destination static gnet_lmt_to_aws gnet_lmt_to_aws no-proxy-arp
Additional Information:
Static translate 172.21.11.26/1024 to 172.21.11.26/1024
Forward Flow based lookup yields rule:
in id=0x559c4775b0, priority=6, domain=nat, deny=false
hits=141945, user_data=0x559e915e40, cs_id=0x0, flags=0x0, protocol=0
src ip/id=172.21.0.0, mask=255.255.0.0, port=0, tag=any
dst ip/id=10.16.0.0, mask=255.248.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=TM_DOME1(vrfid:0)

Phase: 6
Type: NAT
Subtype: per-session
Result: ALLOW
Elapsed time: 9894 ns
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x558ff99220, priority=0, domain=nat-per-session, deny=false
hits=37992418, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=any

Phase: 7
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Elapsed time: 9894 ns
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x559c3cb100, priority=0, domain=inspect-ip-options, deny=true
hits=1361909, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=vlan20-fwmgt(vrfid:0), output_ifc=any

Phase: 8
Type: FOVER
Subtype: standby-update
Result: ALLOW
Elapsed time: 43503 ns
Config:
Additional Information:
Forward Flow based lookup yields rule:
in id=0x55a3cdf6e0, priority=20, domain=lu, deny=false
hits=291315, user_data=0x0, cs_id=0x0, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=vlan20-fwmgt(vrfid:0), output_ifc=any

Phase: 9
Type: VPN
Subtype: encrypt
Result: ALLOW
Elapsed time: 13648 ns
Config:
Additional Information:
Forward Flow based lookup yields rule:
out id=0x559c153c30, priority=70, domain=encrypt, deny=false
hits=3787, user_data=0x123ff7d4, cs_id=0x559bff4ef0, reverse, flags=0x0, protocol=0
src ip/id=172.21.0.0, mask=255.255.0.0, port=0, tag=any
dst ip/id=10.16.0.0, mask=255.248.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any(vrfid:65535), output_ifc=TM_DOME1

Phase: 10
Type: NAT
Subtype: rpf-check
Result: ALLOW
Elapsed time: 5118 ns
Config:
nat (TM_DOME1,any) source static gnet_aws gnet_aws destination static gnet_lmt_to_aws gnet_lmt_to_aws no-proxy-arp
Additional Information:
Forward Flow based lookup yields rule:
out id=0x559ecce310, priority=6, domain=nat-reverse, deny=false
hits=124788, user_data=0x559ef51bc0, cs_id=0x0, use_real_addr, flags=0x0, protocol=0
src ip/id=172.21.0.0, mask=255.255.0.0, port=0, tag=any
dst ip/id=10.16.0.0, mask=255.248.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any(vrfid:65535), output_ifc=TM_DOME1

Phase: 11
Type: VPN
Subtype: ipsec-tunnel-flow
Result: ALLOW
Elapsed time: 49474 ns
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x55a59e01e0, priority=70, domain=ipsec-tunnel-flow, deny=false
hits=3787, user_data=0x12401e2c, cs_id=0x559bff4ef0, reverse, flags=0x0, protocol=0
src ip/id=10.16.0.0, mask=255.248.0.0, port=0, tag=any
dst ip/id=172.21.0.0, mask=255.255.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=TM_DOME1(vrfid:0), output_ifc=any

Phase: 12
Type: NAT
Subtype: per-session
Result: ALLOW
Elapsed time: 4265 ns
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x558ff99220, priority=0, domain=nat-per-session, deny=false
hits=37992420, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=6
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=any, output_ifc=any

Phase: 13
Type: IP-OPTIONS
Subtype:
Result: ALLOW
Elapsed time: 1706 ns
Config:
Additional Information:
Reverse Flow based lookup yields rule:
in id=0x559617b120, priority=0, domain=inspect-ip-options, deny=true
hits=824938, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0
src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any
dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any, dscp=0x0, nsg_id=none
input_ifc=TM_DOME1(vrfid:0), output_ifc=any

Phase: 14
Type: FLOW-CREATION
Subtype:
Result: ALLOW
Elapsed time: 93830 ns
Config:
Additional Information:
New flow created with id 47385277, packet dispatched to next module
Module information for forward flow ...
snp_fp_inspect_ip_options
snp_fp_tcp_normalizer
snp_fp_snort
snp_fp_translate
snp_fp_tcp_normalizer
snp_fp_adjacency
snp_fp_encrypt
snp_fp_fragment
snp_fp_tracer_drop
snp_ifc_stat

Module information for reverse flow ...
snp_fp_inspect_ip_options
snp_fp_ipsec_tunnel_flow
snp_fp_tcp_normalizer
snp_fp_translate
snp_fp_snort
snp_fp_tcp_normalizer
snp_fp_adjacency
snp_fp_fragment
snp_fp_tracer_drop
snp_ifc_stat

Phase: 15
Type: EXTERNAL-INSPECT
Subtype:
Result: ALLOW
Elapsed time: 26443 ns
Config:
Additional Information:
Application: 'SNORT Inspect'

Phase: 16
Type: SNORT
Subtype: appid
Result: ALLOW
Elapsed time: 14824 ns
Config:
Additional Information:
service: (0), client: (0), payload: (0), misc: (0)

Phase: 17
Type: SNORT
Subtype: firewall
Result: ALLOW
Elapsed time: 180260 ns
Config:
Network 0, Inspection 0, Detection 2, Rule ID 268459029
Additional Information:
Starting rule matching, zone 1 -> 11, geo 0 -> 0, vlan 0, src sgt: 0, src sgt type: unknown, dst sgt: 0, dst sgt type: unknown, user 9999999, no url or host, no xff
Matched rule ids 268459029 - Allow

Result:
input-interface: vlan20-fwmgt(vrfid:0)
input-status: up
input-line-status: up
output-interface: TM_DOME1(vrfid:0)
output-status: up
output-line-status: up
Action: allow
Time Taken: 532868 ns

[MHM Cisco World]

Hi friend 

Sorry for late reply I search look for your post in security vpn, until I found it here in routing. 

Anyway 

This nat with any as egress interface is suboptimal, change it and specify correct interface. 

Also I don't see lookup did you config routing for remote Lan (AWS prefix)? 

nat (TM_DOME1,any) source static gnet_aws gnet_aws destination static gnet_lmt_to_aws gnet_lmt_to_aws no-proxy-arp
Additional Information:
NAT divert to egress interface TM_DOME1(vrfid:0)
Untranslate 10.17.0.159/80 to 10.17.0.159/80

[helpmeplss]

Hi,

You're right that the 'ANY' rout should be specified. Yes, I've added the AWS prefix 10.16.0.0/13. There are two possible changes I need to make: following AWS's recommendation to use the CIDR 0.0.0.0/0 and removing the manually added static route for the AWS VPN

[Georg Pauwen]

Hello,

stupid question maybe, but other than the ping timouts, does the VPN actually experience user traffic problems as well ?

Either way, on the Cisco, I think the command:

no crypto ikev2 ipsec-over-nat-t

disables the NatT.