Be Careful with TCP Syslog and the ASA

I wanted to take just a moment to share a little gotcha that could take you by surprise. To demonstrate, I have a simple topology with an ASA in the middle. I am inspecting ICMP so ping traffic is stateful and flows properly.

TCP Syslog
To confirm connectivity,  I can ping from csr1000v-2 from csr1000v-1

csr1000v-1#ping 10.0.0.10 repeat 5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/6/16 ms

Now for the ASA change that can catch an administrator off guard

asav-1(config)#logging on
asav-1(config)#logging trap informational
asav-1(config)#logging host inside 1.1.1.1 tcp/1025

//clear the connection just to make sure the next connection will be new
asav-1(config)#clear conn

Now the connectivity from csr1000v-1 to csr1000v-2 is broken

csr1000v-1#ping 10.0.0.10 repeat 5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.10, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

So what is wrong? Let’s take one quick look at the logging configuration.

asav-1(config)# show run logging
logging enable
logging trap informational
logging host inside 1.1.1.1 6/1025
asav-1(config)#

Aside from the translation from “TCP” to the protocol number “6”, the configuration is as expected. So what does “show logging” tell us?

asav-1(config)# show logging
Syslog logging: enabled
    Facility: 20
    Timestamp logging: disabled
    Hide Username logging: enabled
    Standby logging: disabled
    Debug-trace logging: disabled
    Console logging: disabled
    Monitor logging: disabled
    Buffer logging: disabled
    Trap logging: level informational, facility 20, 164 messages logged
        Logging to inside 1.1.1.1 tcp/1025 Not connected since Tue, 28 Jun 2016 21:45:09 UTC
    Permit-hostdown logging: disabled
    History logging: disabled
    Device ID: disabled
    Mail logging: disabled
    ASDM logging: disabled

Notice the bold text in the output. Let’s enable logging to the console to see if the ASA will tell us more about why the traffic is being blocked.

asav-1(config)# logging console inform

Now sending traffic from csr1000v-1 to csr1000v-2 yields some messages in the console log of the ASA.

%ASA-3-201008: Disallowing new connections.
%ASA-3-201008: Disallowing new connections.

The Cisco website displays all of the syslog messages and gives us the answer to %ASA-3-201008. Since we are doing TCP based logging, the ASA can determine the status of the syslog server (or that it doesn’t exist). Since the connection cannot be established and log the activity, it defaults to disallowing new connections for transit traffic. This probably makes sense in some highly secure installations, but can create problems if it is unexpected.

There are a couple of ways to solve this. The first method is to only use UDP based logging. The other way to solve this is with the following command.

asav-1(config)# logging permit-hostdown

Conclusion

While this is a simple thing when you can quickly correlate that issue to the change, there are cases when it could be somewhat elusive. Think about when other issues may be going on at the same time. Also, existing connections will continue to allow traffic to flow. The biggest gotcha is that if you are only logging to the broken syslog server, you will not get any log output indicating the issue. So if this feature is leveraged in an environment, care must be taken to ensure the uptime requirements are met.

Disclaimer: This article includes the independent thoughts, opinions, commentary or technical detail of Paul Stewart. This may or may does not reflect the position of past, present or future employers.

No related content found.

About Paul Stewart, CCIE 26009 (Security)

Paul is a Network and Security Engineer, Trainer and Blogger who enjoys understanding how things really work. With over 15 years of experience in the technology industry, Paul has helped many organizations build, maintain and secure their networks and systems.
This entry was posted in Design and tagged . Bookmark the permalink.

2 Responses to Be Careful with TCP Syslog and the ASA

  1. Shuja says:

    Hey brother

    Good post. I am unclear about “Since the connection cannot be established and log the activity, it defaults to disallowing new connections for transit traffic. This probably makes sense in some highly secure installations, but can create problems if it is unexpected.” Can you elaborate further if possible.

    • Good question. Simply put, there are probably a few installations that have a very strict logging policy. If that policy states not to allow traffic if the traffic cannot be logged, TCP syslogging might be used. So with TCP (connection oriented) logging, unlike UDP (connectionless and unacknowledged), the ASA can determine when it cannot reach the syslog server or that service has stopped. In that case when the syslog server appears to be down, traffic that would otherwise go through the firewall is dropped. If UDP logging is used, the ASA typically wouldn’t have a way to realize the logging server or service is down.

Comments are closed.