Hairpinning Internet and VPN Traffic in Cisco IOS with NAT

This week I wanted to address a concept that comes up occasionally. This is the concept of hair-pinning Internet traffic through a VPN. For this particular case study, we will use an IOS based Cisco router to terminate both ends of the VPN. Additionally, we will use crypto maps to create a traditional policy based IPSec tunnel. I want to warn that this is the difficult way to solve this problem. In a future article, I’ll demonstrate a simpler way to do this with route based VPNs.

The Scenario–

You are the administrator of the network shown below. On the left is a branch office that has an IPSec tunnel built to the LAN for the Headquarters (represented by Loopback0). As is typically the case, you have no control over routers or hosts that are outside the organization on the Internet. Your boss decides he (or she) desires that all branch Internet traffic use the 192.0.2.131 from R3 at Headquarters. All traffic between Branch and HQ must be encrypted using a traditional crypto map configuration.

Prior to implementing the changes requested by your boss, the following configuration exists.

PC (emulated by a Cisco Router)

hostname PC
!
no ip routing
!
interface FastEthernet0/1
 ip address 192.168.0.2 255.255.255.0
!
ip default-gateway 192.168.0.1

WWW_Server (emulated by a Cisco Router)–Not under your control

hostname WWW_Server
!
no ip routing
!
interface FastEthernet0/0
 ip address 192.0.2.3 255.255.255.128
!
ip default-gateway 192.0.2.2

R2–Not under your control

hostname R2
!
interface FastEthernet0/0
 ip address 192.0.2.2 255.255.255.128
!
interface Serial0/0
 ip address 192.0.2.130 255.255.255.128

The above device configurations are shown for informational purposes only. No changes will be necessary. The following devices configure the current NAT and VPN configuration for their respective location.

R1–Branch Office

hostname R1
!
crypto isakmp policy 10
 encr 3des
 hash md5 
 authentication pre-share
crypto isakmp key cisco address 192.0.2.131
!         
crypto ipsec transform-set MYSET esp-3des esp-md5-hmac 
!         
crypto map MYMAP 10 ipsec-isakmp 
 set peer 192.0.2.131
 set transform-set MYSET 
 match address CRYPTO
!         
interface FastEthernet0/0
 ip address 192.0.2.1 255.255.255.128
 ip nat outside
 crypto map MYMAP
!         
interface FastEthernet0/1
 ip address 192.168.0.1 255.255.255.0
 ip nat inside
!         
ip route 0.0.0.0 0.0.0.0 192.0.2.2
!
ip nat inside source list NAT interface FastEthernet0/0 overload
!         
ip access-list extended CRYPTO
 permit ip 192.168.0.0 0.0.0.255 192.168.1.0 0.0.0.255
ip access-list extended NAT
 deny   ip 192.168.0.0 0.0.0.255 192.168.1.0 0.0.0.255
 permit ip 192.168.0.0 0.0.0.255 any
!

R3–Headquarters

hostname R3
!
crypto isakmp policy 10
 encr 3des
 hash md5 
 authentication pre-share
crypto isakmp key cisco address 192.0.2.1
!         
crypto ipsec transform-set MYSET esp-3des esp-md5-hmac 
!         
crypto map MYMAP 10 ipsec-isakmp 
 set peer 192.0.2.1
 set transform-set MYSET 
 match address CRYPTO
!  
interface Loopback0
 ip address 192.168.1.1 255.255.255.0
!
interface Serial0/0
 ip address 192.0.2.131 255.255.255.128
 ip nat outside
 crypto map MYMAP
!
ip route 0.0.0.0 0.0.0.0 192.0.2.130
!
ip nat inside source list NAT interface Serial0/0 overload
!         
ip access-list extended CRYPTO
 permit ip 192.168.1.0 0.0.0.255 192.168.0.0 0.0.0.255
ip access-list extended NAT
 deny   ip 192.168.1.0 0.0.0.255 192.168.0.0 0.0.0.255
 permit ip 192.168.1.0 0.0.0.255 any
!

To verify our base configuration, we can attempt to ping the Loopback0 on R3 and WWW_Server. We’ll source all of our traffic from “PC”. We’ll also run a “debug ip nat” on R1, to make sure that it is currently translating the traffic to the Internet.

Enable “debug ip nat”

R1#debug ip nat         
IP NAT debugging is on

Testing Connectivity from PC

//192.168.1.1 is the HQ LAN
PC#ping 192.168.1.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 40/41/44 ms

//192.0.2.3 is WWW_Server
PC#ping 192.0.2.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.0.2.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 16/35/44 ms
PC#

Debug Output

R1#
*Mar  1 01:54:48.179: NAT*: s=192.168.0.2->192.0.2.1, d=192.0.2.3 [45]
*Mar  1 01:54:48.199: NAT*: s=192.0.2.3, d=192.0.2.1->192.168.0.2 [45]
<--snip (repeats 5 times)-->

So to achieve our objectives, we need to start asking ourselves some questions. How do we get all Internet bound traffic from R1 to HQ? Do we need to perform any NAT at all on R1? This is actually the simple part, so let’s go ahead and address these. Since all Internet bound traffic should go through HQ, R1 no longer needs to NAT. Additionally, to tunnel all of this traffic, we need to change our crypto acl’s so the phase 2 associations will include everything from R1’s LAN.

R1 Reconfiguration

//remove the NAT Configuration
R1(config)#interface FastEthernet0/0
R1(config-if)#no ip nat outside
R1(config-if)#interface FastEthernet0/1
R1(config-if)#no ip nat inside
R1(config)#$no ip nat inside source list NAT interface FastEthernet0/0 overload       
R1(config)#no ip access-list extended NAT

//Change the CRYPTO ACL
R1(config)#do show access-list CRYPTO
Extended IP access list CRYPTO
    10 permit ip 192.168.0.0 0.0.0.255 192.168.1.0 0.0.0.255 (55 matches)
R1(config)#ip access-list extended CRYPTO
R1(config-ext-nacl)#no 10
R1(config-ext-nacl)#10 permit ip 192.168.0.0 0.0.0.255 any
R1(config-ext-nacl)#do show access-list CRYPTO
Extended IP access list CRYPTO
    10 permit ip 192.168.0.0 0.0.0.255 any

Since we changed the CRYPTO ACL on R1, we need to mirror those changes on R3.

//CRYPTO ACL changes on R3
R3(config)#do show access-list CRYPTO
Extended IP access list CRYPTO
    10 permit ip 192.168.1.0 0.0.0.255 192.168.0.0 0.0.0.255 (50 matches)
R3(config)#ip access-list extended CRYPTO
R3(config-ext-nacl)#no 10
R3(config-ext-nacl)#10 permit ip any 192.168.0.0 0.0.0.255
R3(config-ext-nacl)#do show access-list CRYPTO
Extended IP access list CRYPTO
    10 permit ip any 192.168.0.0 0.0.0.25

//we should also clear the SA's to make sure 
//the new policy is applied to the traffic.

R3#clear crypto sa

So let’s see what we have by testing again.

Testing from PC

//Test to HQ LAN
PC#ping 192.168.1.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.1, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 16/37/48 ms

//Test to WWW_Server
PC#ping 192.0.2.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.0.2.3, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
PC#

Obviously, our second test still failed. Let’s start tracking the issues down.

Did all of the traffic leave R1 encrypted?

R1#show crypto ipsec sa

interface: FastEthernet0/0
    Crypto map tag: MYMAP, local addr 192.0.2.1

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (192.168.0.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   current_peer 192.0.2.131 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 9, #pkts encrypt: 9, #pkts digest: 9
    #pkts decaps: 4, #pkts decrypt: 4, #pkts verify: 4

We can see that there are 9 encaps and 4 decaps. I actually sent 10 total echo requests, but the tunnel had to build. So 9 makes sense. What is important to note here is that I have 5 less decaps than encaps, so I probably didn’t get anything back from my second ping attempt (to WWW_Server).

Let’s look at R3 now

R3#show crypto ipsec sa

interface: Serial0/0
    Crypto map tag: MYMAP, local addr 192.0.2.131

   protected vrf: (none)
   local  ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/0/0)
   remote ident (addr/mask/prot/port): (192.168.0.0/255.255.255.0/0/0)
   current_peer 192.0.2.1 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 4, #pkts encrypt: 4, #pkts digest: 4
    #pkts decaps: 9, #pkts decrypt: 9, #pkts verify: 9

Well that is a mirror of what we have over on R1. R3 received 5 more packets than it responded with, but why? We typically don’t have access to third party WWW_Servers, but let’s pretend we do and see what it is seeing.

WWW_Server

WWW_Server#debug ip packet 
IP packet debugging is on

PC Testing

PC#ping 192.0.2.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.0.2.3, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

WWW_Server Results

WWW_Server#
*Mar  1 02:15:49.563: IP: s=192.168.0.2 (FastEthernet0/0), d=192.0.2.3, len 100, rcvd 1
*Mar  1 02:15:49.563: IP: tableid=0, s=192.0.2.3 (local), d=192.168.0.2 (FastEthernet0/0), routed via RIB
*Mar  1 02:15:49.563: IP: s=192.0.2.3 (local), d=192.168.0.2 (FastEthernet0/0), len 100, sending
*Mar  1 02:15:49.583: IP: s=192.0.2.2 (FastEthernet0/0), d=192.0.2.3, len 56, rcvd 1

Okay, now we see something interesting. The packets are coming from the actual IP address of “PC”. Well that sort of makes sense since we didn’t make any NAT changes on R3. Let’s make those changes now.

R3

R3#show run | inc ip nat inside source
ip nat inside source list NAT interface Serial0/0 overload
R3#show access-list NAT   
Extended IP access list NAT
    10 deny ip 192.168.1.0 0.0.0.255 192.168.0.0 0.0.0.255 (29 matches)
    20 permit ip 192.168.1.0 0.0.0.255 any

//changes required to have the R1 LAN dynamically create NAT entries
//note that we still don't want to nat for anything going to R1 LAN

R3(config)#ip access-list extended NAT
R3(config-ext-nacl)#deny ip any 192.168.0.0 0.0.0.255
R3(config-ext-nacl)#permit ip 192.168.0.0 0.0.0.255 any
R3(config-ext-nacl)#do show access-list NAT
Extended IP access list NAT
    10 deny ip 192.168.1.0 0.0.0.255 192.168.0.0 0.0.0.255 (29 matches)
    20 permit ip 192.168.1.0 0.0.0.255 any
    30 deny ip any 192.168.0.0 0.0.0.255
    40 permit ip 192.168.0.0 0.0.0.255 any

Now let’s try our test again

PC1 Testing

PC#ping 192.0.2.3

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.0.2.3, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

WWW_Server Results

WWW_Server#
*Mar  1 02:18:49.563: IP: s=192.168.0.2 (FastEthernet0/0), d=192.0.2.3, len 100, rcvd 1
*Mar  1 02:18:49.563: IP: tableid=0, s=192.0.2.3 (local), d=192.168.0.2 (FastEthernet0/0), routed via RIB
*Mar  1 02:18:49.563: IP: s=192.0.2.3 (local), d=192.168.0.2 (FastEthernet0/0), len 100, sending
*Mar  1 02:18:49.583: IP: s=192.0.2.2 (FastEthernet0/0), d=192.0.2.3, len 56, rcvd 1

Unfortunately our results are the same. This is where things get interesting. If you recall from your experience with NAT, the rules are evaluated when packets flow between an “ip nat inside” and “ip nat outside” interface. Alternatively, if you are using “ip nat enable”, the rules are evaluated when the packets flow between any two interfaces with NAT enabled. Since NVI doesn’t solve this alone, let’s look at the solution using the traditional nat configuration that we started with.

Key Concept–NAT rules are evaluated when packets pass between opposing NAT enabled interfaces. For example, “ip nat inside” to “ip nat outside”.

So how can we force traffic that lands on our outside interface to another interface? Remember, this traffic is supposed to go back out that same interface. That is the dilemma. The only solution I know to defy the routing table is policy based routing. Let’s create a loopback network to temporarily steer the traffic to. Then we’ll create a route-map to match the Internet bound traffic from R1.

R3

//create a loopback network

R3(config)#interface Loopback10
R3(config-if)# ip address 10.10.10.1 255.255.255.0
R3(config-if)# ip nat inside

//create an acl and route-map
//match only internet bound traffic received from R1's LAN

R3(config-if)#ip access-list extended ROUTEMAP
R3(config-ext-nacl)# deny   ip 192.168.0.0 0.0.0.255 192.168.1.0 0.0.0.255
R3(config-ext-nacl)# permit ip 192.168.0.0 0.0.0.255 any

//send this traffic out our loopback (10.10.10.2 is on 10.10.10.0/24)
R3(config-ext-nacl)#route-map ROUTEMAP permit 10
R3(config-route-map)# match ip address ROUTEMAP
R3(config-route-map)# set ip next-hop 10.10.10.2

//apply the policy routing to our outside interface

R3(config)#interface s0/0
R3(config-if)#ip policy route-map ROUTEMAP

Now let’s test our configuration once again.

WWW_Server

WWW_Server#debug ip packet
IP packet debugging is on

PC

PC#ping 192.0.2.3  

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.0.2.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 52/68/88 ms

WWW_Server

WWW_Server#
*Mar  1 02:47:07.015: IP: s=192.0.2.131 (FastEthernet0/0), d=192.0.2.3, len 100, rcvd 1
*Mar  1 02:47:07.019: IP: tableid=0, s=192.0.2.3 (local), d=192.0.2.131 (FastEthernet0/0), routed via RIB

So this seems to work and our boss should be happy. The trick was that we had to policy route the traffic that initiates the dynamic nat entries to an inside interface. It is actually the act of routing them back out to the outside interface (from the inside interface–Loopback 0 in our example) that creates the translations. The return traffic is properly translated without doing anything fancy. Keep in mind that there are easier ways to do this. Next week, we’ll look at this same example, but solve the problem using a route based VPN with virtual interfaces.

If you have other methods or scenarios, I’d love to hear about them. Feel free to reach out via Facebook, Twitter, or the “Requests” link above. Also, I love to hear comments and feedback about the weekly articles.

About Paul Stewart, CCIE 26009 (Security)

Paul is a Network and Security Engineer, Trainer and Blogger who enjoys understanding how things really work. With over 15 years of experience in the technology industry, Paul has helped many organizations build, maintain and secure their networks and systems.
This entry was posted in How-To. Bookmark the permalink.

21 Responses to Hairpinning Internet and VPN Traffic in Cisco IOS with NAT

  1. Pingback: Cisco ASA 8.4 VPN — Dealing with Internet Hairpin Traffic | PacketU

  2. I stumbled on this exact same solution from some obscure forum post years ago. Good explanation.
    I’m curious about whether NVI fixes the issue as I have not had occasion to deploy this scenario since.

  3. albert says:

    I did not see 10.10.10.2 anywhere in the diagram . should it be 10.10.10.1 ?

    • Setting a next hop of 10.10.10.2 pushes the packet through the logic of the loopback interface connected to 10.10.10.0/24 (via 10.10.10.1). It’s not a real address, but it works the magic of inserting that loopback interface into the flow.

      • albert says:

        great! thanks for the response. Now I need to understand this concept and apply it to my real case scenario where I have IPSEC +GRE tunnels. Any quick help on how to do this ?

      • George says:

        Paul, great article. I just don’t understand why if I set next-hop pointing to 10.10.10.2 it works perfect but fails when it’s set to 10.10.10.1 (loopback address).

      • The logic of the router is that 10.10.10.1 would be handled locally. A next hop of 10.10.10.2 forces it to route the packet (but it never really actually sends the packet out loop10). It just simulates it going out the loopback interface and associates the packet with the nat rule. This is an older method, definitely a hack. Now I would probably look at using route-maps attached to my static translations (but this will probably still work on current IOS).

      • George says:

        Thanks Paul!. I understand. As you say, when pointing the next-hop to the (local) loopback interface IP address it’s seems logical to no hit the nat rule because i’m not “comming” from the nat outside. But.. it’s normal to don’t see either the packet re-routed to Internet without being translated with nat?? I did sniffing and debug ip packet and debug ip policy and the packet hits the policy, gets forwarded to the loopback interface and then nothing, it’s no forwarded to any other interface nor going back to the tunnel (show crypto ipsec sa).
        I did another simple test with two routers and policy routing the traffic setting the next-hop to the local loopback IP address interface does not get the packet re-routed/forwarded to another interface, is that normal?

  4. Nam Pham says:

    Thanks for your excellent post !!!
    Can this method still work if R3 is not connected directly to Internet and placed behind another NAT Router ???

    • Paul Stewart, CCIE 26009 (Security) says:

      I haven’t tested that, but I see no reason why it wouldn’t work. The NAT device would probably need translation(s) to support ESP and ISAKMP.

      • Paul Stewart, CCIE 26009 (Security) says:

        It would also be necessary to modify the NAT ACL to include the remote subnet that would be hairpinned to the internet.

  5. Doru says:

    can you tell me if is possible or how to do, to push the 0.0.0.0 by cryptomap on branch on gw that is on HQ different that R3

    • Paul Stewart, CCIE 26009 (Security) says:

      I’m not sure I fully appreciate the question. The source/dest pairs should be adjusted for each branch deployment. If there’s much complexity, GRE/IPSec might make it much easier to deploy and understand.

  6. albert says:

    well, this U turn on the Loopback trick is not working on my Setup. Beside the IPSEC/GRE tunnels between the routers. The NAT inside on the loopback is not doing his job. I am seeing the packets getting back out to the Internet but it is not Natting.

    • If you are using GRE/IPSec, you might look at the following article–

      https://packetu.com/2012/07/03/using-route-based-vpns-to-make-hairpinning-more-logical/

      There is still a loopback on R3, but it is not required for the core function being demonstrated. It is simply a place to simulate an internal IP for the example. This type of configuration is easier to accomplish with GRE. This is true because you configure the tunnel interface as “ip nat outside”, get the routing right and you should be good.

      • albert says:

        As I don’t have control of R1 this approach (tunnel protection ipsec) wasn’t an option. But I was able to make it work with just an ip nat inside within the Tunnel interface. No loopback 0 and no IP Policy map required. I have an ip nat inside on my R3 f0/0 and ip nat outside on R3 s0/0.

  7. Pépé says:

    The loopback trick doesn’t seem to work, I know I am close but can’t get it right :

    I have a similar scenario involving multiple outgoing interfaces on R3 :
    – WAN1 for VPN
    – WAN 2 for Internet

    In this case I believe the policy should be applied not on the outside interface (WAN2) but the incoming interface where traffic comes from (WAN1).
    I can see traffic going through the ROUTEMAP access-list but not through the NAT of WAN2.

    Any suggestions perhaps ?

  8. Philip S. says:

    So I spent the past couple of weeks trying to reproduce this configuration between two routers over the internet and failed over and over again. I figured I was making a syntax or fat fingering an IP address. Well I finally figured out my problem and I hope this will help your readers.

    Watch your IOS level! I was running advance security 15.1-4.M10 on a 2800 series and it would not work! As soon as I downgraded to version 12.4(24)T8 the config above worked just fine!

  9. Nick Appletech says:

    I came across this fine article and I thank you for your valuable analysis.

    A question: In your last R3 configuration snippet, shouldn’t we also add a NAT definition:

    ip nat inside source list ROUTEMAP interface Serial0/0 overload

    …??

    How else would NAT translations be done? I think that “ip nat inside” alone, on Loopback0 interface wouldn’t be enough.

    Can you please clarify?

    Thanks in advance!

    Nick

  10. chummers36 says:

    Can I just get the full config for HQ site please?

  11. Kyler says:

    I wrote a post about how to allow a guest subnet to use public IPs for your DMZ hosts, might be interesting:

    http://systems-co.blogspot.com/2016/06/cisco-routers-easy-hair-pin-nat-for.html

Comments are closed.