Internet Redundancy with ASA SLA and IPSec

I’ve seen a lot of examples of redundant Internet connections that use SLA to track a primary connection. The logic is that the primary Internet connection is constantly being validated by pinging something on that ISP’s network and routing floats over to a secondary service provider in the event of a failure. I was recently challenged with how this interacted with IPSec. As a result I built out this configuration and performed some fairly extensive testing.

It is worth noting that this is not a substitute for a properly multi-homed Internet connection that utilizes BGP. It is, however, a method for overcoming the challenges often found in the SMB environments where connections are mostly outbound or can alternatively be handled without completely depending on either of the service provider owned address spaces.

In this article, we will start out with a typical ASA redundant Internet connection using IP SLA. Then we will overlay a IPSec Site to Site configuration and test the failover process.

ASA_IPSec_Redundant

The base configuration for this lab is as follows.

ASA1

//this is an ASA5505 therefore
//the policy is assigned to VLANs

hostname ASA1
!
interface Ethernet0/0
 description ASA1 to Primary SP
 switchport access vlan 2
!
interface Ethernet0/1
 description ASA1 to Inside
 switchport access vlan 1
!
interface Ethernet0/2
 description ASA1 to Secondary SP
 switchport access vlan 3
!
interface Vlan1
 nameif inside
 security-level 100
 ip address 10.1.1.1 255.255.255.0
!
interface Vlan2
 nameif outside
 security-level 0
 ip address 1.1.1.100 255.255.255.0
!
interface Vlan3
 nameif outside2
 security-level 0
 ip address 2.2.2.100 255.255.255.0
!
object network obj_any
 subnet 0.0.0.0 0.0.0.0
object network obj_any2
 subnet 0.0.0.0 0.0.0.0
!
object network obj_any
 nat (inside,outside) dynamic interface
object network obj_any2
 nat (inside,outside2) dynamic interface
!
route outside 0.0.0.0 0.0.0.0 1.1.1.1 1 track 1
route outside2 0.0.0.0 0.0.0.0 2.2.2.2 100
!
sla monitor 123
 type echo protocol ipIcmpEcho 1.1.1.1 interface outside
 num-packets 3
 frequency 10
!
sla monitor schedule 123 life forever start-time now
track 1 rtr 123 reachability

Internet

interface FastEthernet0/0
 description SP1 to Site1
 ip address 1.1.1.1 255.255.255.0
!
interface FastEthernet0/1
 description SP2 to Site1
 ip address 2.2.2.2 255.255.255.0
!
interface FastEthernet1/0
 description SP to Site2
 ip address 3.3.3.3 255.255.255.0

ASA2

//this is an ASA running pre 8.3 code
//so the NAT syntax follow the old model

hostname ASA2
!
interface Ethernet0/0
 description ASA2 to Service Provider
 switchport access vlan 2
!
interface Ethernet0/1
 description ASA2 to Inside
 switchport access vlan 1
!
interface Vlan1
 nameif inside
 security-level 100
 ip address 10.2.2.1 255.255.255.0
!
interface Vlan2
 nameif outside
 security-level 0
 ip address 3.3.3.100 255.255.255.0
!
nat (inside) 1 0.0.0.0 0.0.0.0
global (outside) 1 interface
!
route outside 0.0.0.0 0.0.0.0 3.3.3.3 1

Host1 (simulated by router)

hostname Host1
!
no ip routing
interface FastEthernet0/0
 description To ASA1 LAN
 ip address 10.1.1.111 255.255.255.0
ip default-gateway 10.1.1.1

Host2 (simulated by router)

hostname Host2
!
no ip routing
interface FastEthernet0/0
 description To ASA2 LAN
 ip address 10.2.2.222 255.255.255.0
ip default-gateway 10.2.2.1

Now Let’s Test the Initial Configuration prior to adding VPN.

Host1

//testing Host1 to Internet over Primary Connection
Host1#ping 11.11.11.11
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 11.11.11.11, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

Internet

//force a failover
ip access-list extended DENY
 deny ip any any
interface FastEthernet0/0
 ip access-group DENY in

Host1

Host1#ping 11.11.11.11
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 11.11.11.11, timeout is 2 seconds:
...!!
Success rate is 40 percent (2/5), round-trip min/avg/max = 1/3/4 ms

As can be seen in this example, the failover does work properly. There is an outage of 20 seconds or so while we wait on SLA to deem the primary link unusable, invalidate the connection and form a new connection over the secondary service provide. This is good to know because applications will be impacted as the connection moves to a different public IP address.

At this point, we can overlay the VPN configuration and do some additional testing.

ASA_VPN_Pri_Sec

ASA1

//configure the nat exemption objects and rule
object network obj-10.1.1.0
 subnet 10.1.1.0 255.255.255.0
object network obj-10.2.2.0
 subnet 10.2.2.0 255.255.255.0
nat (inside,any) source static obj-10.1.1.0 obj-10.1.1.0 destination static obj-10.2.2.0 obj-10.2.2.0

//create the phase 1 policy
crypto ikev1 policy 10
 authentication pre-share
 encryption aes
 hash sha
 group 2
 lifetime 86400

//enable isakmp on both outside interfaces
crypto ikev1 enable outside
crypto ikev1 enable outside2

//create the tunnel group and psk
tunnel-group 3.3.3.100 type ipsec-l2l
tunnel-group 3.3.3.100 ipsec-attributes
 ikev1 pre-shared-key cisco

//create the crypto ACL
access-list L2LAccessList extended permit ip 10.1.1.0 255.255.255.0 10.2.2.0 255.255.255.0

//create transform set
crypto ipsec ikev1 transform-set myset esp-aes esp-sha-hmac

//create the crypto map
crypto map mymap 10 match address L2LAccessList
crypto map mymap 10 set peer 3.3.3.100
crypto map mymap 10 set ikev1 transform-set myset

//apply the crypto map to both outside interfaces
crypto map mymap interface outside
crypto map mymap interface outside2

ASA2

//pre 8.3 configuration
//create the nat exemption acl and rule
access-list NONAT extended permit ip 10.2.2.0 255.255.255.0 10.1.1.0 255.255.255.0
nat (inside) 0 access-list NONAT

//create the phase 1 policy
crypto isakmp policy 10
 authentication pre-share
 encryption aes
 hash sha
 group 2
 lifetime 86400

//enable isakmp on the outside interface
crypto isakmp enable outside

//create the TWO tunnel groups (one for each potential tunnel source/destination on ASA1)
tunnel-group 2.2.2.100 type ipsec-l2l
tunnel-group 2.2.2.100 ipsec-attributes
 pre-shared-key cisco
tunnel-group 1.1.1.100 type ipsec-l2l
tunnel-group 1.1.1.100 ipsec-attributes
 pre-shared-key cisco

//create the crypto acl
access-list L2LAccessList extended permit ip 10.2.2.0 255.255.255.0 10.1.1.0 255.255.255.0

//create the transform set
crypto ipsec transform-set myset esp-aes esp-sha-hmac

//create the transform set (notice the primary and secondary tunnel destination)
crypto map mymap 10 match address L2LAccessList
crypto map mymap 10 set peer 1.1.1.100 2.2.2.100
crypto map mymap 10 set transform-set myset
crypto map mymap 10 set reverse-route

//apply the crypto map to the outside interface
crypto map mymap interface outside

Now let’s confirm the VPN functionality during normal operation.

H1

Host1#ping 10.2.2.222 repeat 20
Type escape sequence to abort.
Sending 20, 100-byte ICMP Echos to 10.2.2.222, timeout is 2 seconds:
.!!!!!!!!!!!!!!!!!!!
Success rate is 95 percent (19/20), round-trip min/avg/max = 1/2/4 ms

ASA2

//we can see that the connection that landed on ASA2 is from the primary IP address
ASA2# show crypto isakmp sa

   Active SA: 1
    Rekey SA: 0 (A tunnel will report 1 Active and 1 Rekey SA during rekey)
Total IKE SA: 1

1   IKE Peer: 1.1.1.100
    Type    : L2L             Role    : responder
    Rekey   : no              State   : MM_ACTIVE
!
ASA2# show crypto ipsec sa | inc peer
      current_peer: 1.1.1.100

So let’s see what happens when the primary internet goes down during an extended ping. To do this, I will start a ping from Host 1 and add the ACL on the internet router.

Host1

Host1#ping 10.2.2.222 repeat 300
Type escape sequence to abort.
Sending 300, 100-byte ICMP Echos to 10.2.2.222, timeout is 2 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!...
.......!!!!!!!!!!!!!
Success rate is 96 percent (290/300), round-trip min/avg/max = 1/2/4 ms

We can also look and see what happened at ASA2

ASA2

//notice that ASA2 is now receiving IPSec from 2.2.2.100
ASA2# show crypto isakmp sa

   Active SA: 1
    Rekey SA: 0 (A tunnel will report 1 Active and 1 Rekey SA during rekey)
Total IKE SA: 1

1   IKE Peer: 2.2.2.100
    Type    : L2L             Role    : responder
    Rekey   : no              State   : MM_ACTIVE
!
ASA2# show crypto ipsec sa | inc peer
      current_peer: 2.2.2.100

So we can see from the example that we have a short outage as the ICMP traffic from Host1 to Host2 is moved from one path to the other. Let’s see what happens with a telnet session traversing the path as it fails. For this test, I will reset the primary path by removing the ACL from the Internet router. Then I will telnet from Host1 to Host2. This will simulate a generic TCP connection through the tunnel.

Host1

//notice that the connection fails as the tunnel is moved from the primary to secondary path
Host1#telnet 10.2.2.222
Trying 10.2.2.222 ... Open


User Access Verification

Username: admin
Password:
Host2#
Host2#
[Connection to 10.2.2.222 closed by foreign host]
Host1#

//also notice that the connection can be immediately re-established
Host1#telnet 10.2.2.222
Trying 10.2.2.222 ... Open


User Access Verification

Username: admin
Password:
Host2#

It is worth noting that the same failure is seen as the tunnel is moved back to the primary path. Basically the ASA has to have the “conn” table in agreement with the routing table and IPSec peering. As the routing moves the traffic off of one interface to another, the ASA resets the connection.

Conclusion

From this exercise, I we see that the ASA’s SLA feature worked as advertised. It also demonstrates that the routing change interacted with the IPSec operation exactly as one would’ve expected. The one thing that sort of caught me off guard is the fact that a TCP connection is reset even though it seems unnecessary. My recommendations to anyone working in an environment like this would be to test thoroughly and to only build the minimal complexity required to achieve the required technological objectives. Obviously this is only an example to demonstrate the features. The configuration in a given environment should be hardened and tested as required by the organization.

I’d love to hear from you, so share your experiences by commenting below.

Disclaimer: This article includes the independent thoughts, opinions, commentary or technical detail of Paul Stewart. This may or may does not reflect the position of past, present or future employers.

No related content found.

About Paul Stewart, CCIE 26009 (Security)

Paul is a Network and Security Engineer, Trainer and Blogger who enjoys understanding how things really work. With over 15 years of experience in the technology industry, Paul has helped many organizations build, maintain and secure their networks and systems.
This entry was posted in How-To and tagged . Bookmark the permalink.

6 Responses to Internet Redundancy with ASA SLA and IPSec

  1. Would TCP connection be preserved if we use sysopt to preserve VPN flows ? I’m curious to see the connection state table and associated log files during failover. If VPN dropped I would expect to see some Deny No TCP connection in state table messages. Will try to replicate it in spare time…

  2. Kamil says:

    Never explored the traffic-zones in the real life scenario but it seems very interesting, especially idea around ECMP for VPN flows load-balancing, However from what I read, the same, i.e. TCP connection never flows over i.e. two interfaces in the same zone and is based on hashing algorithm. Upon failover (lost of one of the interfaces within the zone) the traffic re-establishes since a new TCP handshake is required over a newly established VPN tunnel.

    Regarding TCP bypass I think when VPN “flaps” and with failover not being statefull, the TCP connection will reset anyway and cause the Telnet session to re-establish. The TCP bypass does not preserve initial TCP handshake. TCP bypass is purely to pass the security check for the state table “converting” the TCP traffic into UDP traffic for asymmetric routing scenarios.

    The only scenario where I see it working with TCP bypass is for the end server to re-try re-establish the TCP connection by a combination of the end hosts re-transmits happening before the TCP keep-alive expires declaring the session down and sending out the RST – on assumption VPN tunnel managed to re-establish to the redundant peer prior to this action. this is only an assumption, as I am almost certain that ASA will send the RST flag to the end host when the VPN flaps. I managed to preserve the VPN flow in the similar scenario with the help of the “sysopt” and it worked perfectly.

    Thanks for the inspiring article and comments.

  3. Nice write up Paul, it gives me some testing ideas to increase our resiliency!

  4. BELHADJ says:

    Hello Paul,

    I hope you are fine.

    Please a question for you the Expert 🙂

    I’m preparing for my CCIE Security written exam and I’m planning to pass it in June 2016.
    I’m concentrating now on the “Cisco ASA All-in-One Next Generation Firewall, IPS and VPN Services” Third Edition and others.

    What you recommend me for this exam: most theory and less labs or 50% for each?

    Thank you for your time to help others.

    Best regards.

Comments are closed.