Syslog | Logging

Interestingly, when discussing system logging, tired metaphors seem most apt. System log information “is a goldmine” of useful information, but searching through these data is “like trying to find a needle in a haystack.” Tired metaphor or not, the preceding statement is true. Time-stamped logs generated by servers, gateways, firewalls, proxies, routers, and switches often contain invaluable security-related inforrmation, but system administrators are normally so overwhelmed with other maintenance and configuration chores that analyses of these logs is disregarded. The key to successful log analysis is to adopt the proper tools for your environment to automatically parse, visualize, and report summarized log data. For example, many organizations utilize MRTG (Multi Router Traffic Grapher— http://people.ee.ethz.ch/~oetiker/webtools/mrtg/) to visualize router and switch SNMP network data.

Syslog

In its most simplistic terms, the syslog protocol provides a transport to allow a machine to send event notification messages across IP networks to event message collectors—also known as syslog servers. Syslog is an odd protocol in that it was implemented on many platforms before the protocol was ratified by the IEEE in RFC3164. Rather than begin by defining the protocol, RFC3164 starts with “This document describes the observed behavior of the syslog protocol”

Syslog messages use UDP/514 for transport, increasing the possibility of losing packets and never noticing, and also making it easy for anybody to forge fake packets, either to insert log events, or to flood the server. Syslog, at this time, does not provide for encryption, so the messages are sent in the clear and can be sniffed by anyone on the wire. Recently, a proposed draft has been submitted that describes a mechanism to add origin authentication, message integrity, replay-resistance, message sequencing, and detection of missing syslog niessages, but this is not commonly implemented. Several of the popular syslogd replacements (including syslog-ng) can use TCP for reliable delivery, and some add a checksum and/or cryptographic signature to each log event.

Syslog is native on most UNIX platforms, but is not available natively on Microsoft Windows. The most common Windows syslog daemon is Kiwi Syslog (www.kiwisyslog.com).

Syslog messages (ASCII-based) may be sent to local logs, a local console, a remote syslog server, or a remote syslog relay. The syslog facility collects messages and records them normally in log files in /var/log. A facility is defined as a subsystem which generates messages. What is recorded and where it is placed depends on the facility configuration file (syslog.conf). Syslog also uses severity (or priority to some) to classify log messages by importance. The severity levels, from least to most important, are:

• 0 Emergency: system is unusable
• 1 Alert: action must be taken immediately
• 2 Critical: critical conditions
• 3 Error: error conditions
• 4 Warning: warning conditions
• 5 Notice: normal but significant condition
• 6 Informational: informational messages
• 7 Debug: debug-level messages

There is also a severity of none. A severity of none indicates that all messages should be discarded. Entries in syslog.conf indicate how messages from each facility at the various severity levels should be handled.

Here is a small section of a BSD syslog file:

Jan 9 14:46:50 nsl /kernel: usbl:  on
uhcil
Jan 9 14:46:50 nsl /kernel: usbl: USB revision 1.0
Jan 9 14:46:50 nsl /kernel: uhubl: VIA UHCI root hub, class 9/0, rev
1.00/1.00, addr 1
Jan 9 14:46:50 nsl /kernel: uhubl: 2 ports with 2 removable, self
powered
Jan 9 14:47:00 nsl login: ROOT LOGIN (root) ON ttyv0
Jan 9 14:47:18 nsl /kernel: Connection attempt to UDP
192.168.20.20:162 from 192.168.20.1:24343
Jan 9 14:47:19 nsl /kernel: Connection attempt to TCP
127.0.0.1:16001 from 127.0.0.1:1024

Note that the messages are ASCII based and are composed of three major space-delimited fields: the time and date stamp, the hostname and facility, and a text-based message.

Configuration of syslog and syslog remote logging is trivial. Much more difficult than generating appropriate syslog messages is defining processes that determine how the logs are parsed, who is responsible for parsing, and what type of log entries result in actionable alerts.

In a VoIP environment, IP phones may generate syslog messages and servers almost certainly will. These messages should be sent to a centralized server where they are automatically parsed, and where reports are generated at least on a daily basis. These logs are a valuable and often ignored source of both intrusion detection events and system performance messages. For example, syslog can be configured to report failed logon attempts, sudo (a command that attempts to change a restricted user’s permissions to system level or root privilege) attempts, or any action that interacts with the PAM subsystem (Pluggable Authentication Module—an authentication framework). Any message that refers to one of these events may indicate that an intrusion has occurred.