NIDS Limitations

NIDSs that rely upon signatures must constantly update the signature database. Obviously, pure signature matching NIDSs will not alert on attacks for which they have no signature. If signature definitions are too specific, signature-based IDSs may miss variations on known attacks. (A common technique for creating new attacks is to modify existing attacks.) Signature-based NIDSs can also impose noticeable performance problems on systems when numerous attack signatures are matched concurrently. Additionally, signature-based NIDS inspection can be evaded. Secure Networks showed in 1998 that attacks which exploit funda-mental TCP/IP problems—insertion, evasion, and Denial-of-Service attacks—are able to elude NIDS detection. Dan Kaminsky recently showed he could send a series of fragmented packets to a NIDS that, based on the time and the operating system platform that they arrive at, reassemble into an attack for that platform that is not recognized by the NIDS.

Honeypots and Honeynets

A honeypot is a computer system that is shielded from the Internet by a router or firewall that is transparent to an attacker. The honeypot masquerades as a normal undefended system, yet it logs every action taken against it and every operation that is performed on it. The goal of a honeypot operator is to lure an attacker into hacking the system in hopes of learning all of the details of the attack. A honeypot is a system designed to illustrate the methods used by black-hats to probe for, and exploit, a system. Honeynets are networks that contain at least one honeypot. Typically, honeynets present a virtual network complete with virtual services and applications that look to an attacker like a real network.

Honeypots and honeynets are learning tools, and can also be useful as canaries (canaries were used in mines to provide an early warning to miners if air conditions turned sour). Unlike NIDSs and HIDSs, where false positives are a common nuisance, honeypots and honeynets, if configured correctly, do not have a measurable false positive rate. Honeynets are often configured so that their IP space resides within unoccupied IP space in an organization’s internal network. In this configuration, anything that hits the honeynet is either an attack or a precursor to an attack since this IP space is supposedly unused. In its canary role, a honeynet can provide an early warning of a virus or worm attack.

Important NIDS Features

Let’s now discuss the important features of an NIDS.

Maintenance

Most NIDS systems support centralized installation, configuration, and updating since in an enterprise network a security administrator cannot physically access each sensor. In addition, most vendors support the automated download of signatures and software updates. Distribution and customization of the signature libraries and policies should be possible on a per-sensor basis and on a per-group basis (these groups should be defined by the security administrator) so the group signatures and policies do not have to be pushed to each sensor individually.

Communication between the IDS components (sensors and management console) should be encrypted using strong authentication (via key exchange or challenge). And as mentioned earlier, NIDS Ethernet interfaces should be stealthy Transmission of data via the sensing interface is prohibited, unless it is configured intentionally (TCP resets, which we discuss later in this chapter, may be an exception).

Alerting

The management console should be configurable to support alerting via a variety of mechanisms, including SNMP traps, e-mail alerts, pager messages, syslog messages, SMS (short message service), IM, and console alerts.

Logging

All alerts and header and payload data should be automatically stored in a central event database that is backed up regularly via SCP or other secure means.

Extensibility

The NIDS should support simple integration of additional vulnerability assessment tools such as Nmap or Nessus, and should provide support the correlation of data from other IDSs (for example, NIDS and HIDS).

Response

Some NIDS are able to actively respond to attacks or misuse by interfering with the particular message stream that generated the alert. This is normally accomplished via targeted TCP resets that eventually tear down the connection, or by dynamically altering firewall rules or Access Control Lists to block the connection. These active-response NIDSs are often referred to as intrusion prevention systems (IPSs).

Most administrators do not activate these features because of the risk of blocking normal traffic. Imagine that this functionality was enabled on a system directly connected to the Internet. A clever attacker could send traffic to an IPS with the source address spoofed to that of an upstream router, and designed to trigger the IDS. The resultant blocking of the upstream router could effectively remove the organization from the Internet. In a VoIP environment where availability is a key metric, IPSs are not recommended because of this potential to obstruct voice traffic.

Network Intrusion Detection Systems Placement

NIDSs should be located where they can most effectively monitor critical traffic. This doesn’t necessarily mean that NIDSs should be placed where they can monitor the mosttraffic. In Figure 1, an example network is diagrammed. This network consists of a single Internet connection, a DMZ (demilitarized zone), and three internal VLANs, configured for voice users, workstations, and servers. The circular network symbols signify routers or layer 3 switches, while the square network symbols signify layer 2 switches. The five NIDSs are shown as arrowed rectangular boxes.

 
Figure 1: NIDS Locations

In this figure, an NIDS is located on the external side of the firewall to monitor all inbound and outbound Internet traffic. NIDSs are located on internal layer 2 switches in the voice and server VLANs, and on the layer 3 switch that is used to truck these connections. An additional NIDS is situated in the DMZ. In this architecture, the NIDSs will have access to all the network traffic, but are they all really necessary?

Frankly, no. Several of the NIDSs are either redundant or will report so many events as to be meaningless. The external NIDS is unnecessary since it is exposed to the Internet. Even those of you with broadband connections realize that your single IP address is constantly bombarded with exploit probes and port scans. The following is a sample from 30 minutes of scans against a typical home system.

- # tail -f /var/log/messages | egrep -v "repeated"
Oct  3 00:27:33 nsl /kernel: Connection attempt to TCP 192.168.20.20:135 from
24.193.208.77:2258 flags : 0x02
Oct  3 00:30:26 nsl /kernel: Connection attempt to TCP 192.168.20.20:901 from
211.172.40.72:4896 flags : 0x02 swat
Oct  3 00:30:27 nsl /kernel: Connection attempt to TCP 192.168.20.20:901 from
211.172.40.72:4896 flags : 0x02 swat
Oct  3 00:30:58 nsl /kernel: Connection attempt to TCP 192.168.20.20:445 from
83.37.160.160:3400 flags : 0x02
Oct  3 00:31:00 nsl /kernel: Connection attempt to TCP 192.168.20.20:135 from
24.199.122.40:3829 flags : 0x02
Oct  3 00:31:01 nsl /kernel: Connection attempt to TCP 192.168.20.20:135 from
24.199.122.40:3829 flags:0x02,
Oct  3 00:31:01 nsl /kernel: Connection attempt to TCP 192.168.20.20:445 from
83.37.160.160:3400 flags : 0x02
Oct  3 00:31:01 nsl /kernel: Connection attempt to TCP 192.168.20.20:135 from
24.199.122.40:3829 flags : 0x02
Oct  3 00:31:43 nsl /kernel: Connection attempt to UDP 192.168.20.20:137 from
66.63.173.19:1316           netbios-ns
Oct  3 00:31:47 nsl /kernel: Connection attempt to UDP 192.168.20.20:137 from
66.63 .173.19:1316
Oct  3 00:31:54 nsl /kernel: Connection attempt to TCP 192.168.20.20:1433 from
212.33.102.36:2784 flags:0x02 mssql/slammer
Oct  3 00:32:03 nsl /kernel: Connection attempt to UDP 192.168.20.20:137 from
66.63.173.19:1316
Oct  3 00:32:27 nsl /kernel: Connection attempt to UDP 192.168.20.20:137 from
66.63.173.19:1316
Oct  3 00:33:01 nsl /kernel: Connection attempt to UDP 192.168.20.20:137 from
66.63.173.19:1316
Oct  3 00:43:18 nsl /kernel: Connection attempt to TCP 192.168.20.20:445 from
24.199.80.94:1831 flags : 0x02
Oct  3 00:47:55 nsl /kernel: Connection attempt to TCP 192.168.20.20:5000 from
24.84.67.76:4593 flags:0x02 UPnP backdoor
Oct  3 00:47:58 nsl /kernel: Connection attempt to TCP 192.168.20.20:135 from
24.84.67.76:3254 flags : 0x02
Oct  3 00:48:50 nsl /kernel: Connection attempt to TCP 192.168.20.20:4899 from
69.60.111.98:1361 flags:0x02 Radmin exploit
Oct  3 00:49:26 nsl /kernel: Connection attempt to TCP 192.168.20.20:135 from
24.199.105.227:3192 flags : 0x02
Oct  3 00:52:19 nsl /kernel: Connection attempt to TCP 192.168.20.20:5000 from
24.199.230.130:4456 flags : 0x02
Oct  3 00:52:22 nsl /kernel: Connection attempt to TCP 192.168.20.20:135 from
24.199.230.130:4276 flags : 0x02
Oct  3 00:52:22 nsl /kernel: Connection attempt to TCP 192.168.20.20:135 from
24.199.230.130:4276 flags : 0x02
Oct  3 00:55:37 nsl /kernel: Connection attempt to UDP 192.168.20.20:1029 from
203.21.20.30:30065 ICQNuke98
Oct  3 00:55:42 nsl /kernel: Connection attempt to TCP 192.168.20.20:135 from
24.199.105.227:4103 flags : 0x02
Oct  3 00:56:17 nsl /kernel: Connection attempt to TCP 192.168.20.20:135 from
24.167.27.37:2995 flags : 0x02

As you can see, most of this traffic is the result of automated scanning by worms and viruses, or by simple automated scanning tools. In an enterprise environment, where IDS and log data accumulates in copious amounts, these external data can be ignored. It is more important to focus on the events that occur within the firewall perimeter.

The NIDS situated on the layer 2 switches can also be eliminated since this traffic can be monitored at the central layer 3 switch. In addition, the management connection passes through the firewall and may allow an attacker to piggyback into the network if the sensor is compromised. Although there are no hard and fast rules for deploying NIDS, most system administrators deploy them on uplinks and at devices where many VLANs are trunked so that the fewest number of NIDSs can monitor the most traffic. In our sample network, two NIDSs are suitable to monitor most of the network traffic—one NIDS in the DMZ, and one on the central layer 3 switch.

Note that on the layer 3 switch, there are two interesting and separate traffic flows—one on the uplink between the switch and the firewall, and one port that trunks inter VLAN traffic. The choice of how to monitor both traffic flows depends on how the switch-NIDS connection is configured. Two common methods for allowing NIDS access to network traffic are port mirroring (spanning) and the insertion of a tap (we recommend NetOptics taps because they have two power connections and a wide choice of physical interfaces; check them out at www.netoptics.com). Port mirroring, depending upon its configuration, can enable the NIDS to inspect all of the traffic traversing the switch, and is an inexpensive option. However, some vendor’s switches or OS revisions break when port mirroring is enabled. Be sure to check this with your vendor before connecting an NIDS. The second option, a network tap, is more expensive but offloads the mirroring to a separate device. In this simple example network setting, two network taps would have to be used to visualize all of the traffic—one on the uplink, and one on an inter VLAN port.

Network Intrusion Detection Systems Types

NIDSs are normally classified according to the methods they use for attack detection; either as signature-based, or anomaly detection. Note, though, that almost all current NIDSs use a mixture of these approaches. Signature-based approaches, as mentioned earlier in this chapter, rely on some type of pattern matching. NIDS sensors parse the entire IP packet, and make decisions by means of a simple rule-based logic that is based upon signatures or regular expression matching. In other words, they compare the data within a packet payload to a database of predefined attack signatures (a string of bytes). Additionally, statistical or historical algorithms may supplement static pattern matching. Attack signatures usually consist of one or more of the following fields:

• Source and destination IP addresses, or an address or range
• TCP/UDP source and destination ports and ICMP type/code
• IP header flags and options
• TCP header flags and options
• A definition of the payload data to search (hex or ASCII)
• A starting point for the payload search (offset) and the search depth

Analysis of packet headers can be done economically since the locations of packet header fields are restricted by protocol standards. However, the payload contents are, for the most part, unconstrained. Therefore, searching through the payload for multiple string patterns within the datastream can be a computationally expensive task. The requirement that these searches be performed at wirespeed only adds to the cost.

Anomaly detection NIDSs are based on the assumption that normal traffic can be defined, and that attack or misuse patterns will differ from “normal” traffic. Heuristic-based signatures, on the other hand, use some type of algorithmic logic on which to determine their alarm decisions.

Note

Heuristic is the art and science of discovery and invention. The word comes from the same Greek root as “eureka,” which means “I find.” Heuristics defines a problem-solving technique in which the most appropriate solution is selected at successive stages of a program for use in the next step of the program. Heuristic approaches utilize simplification or an educated guess to reduce or limit the search for solutions. A heuristic can be a single algorithmic solution to a problem, but unlike an algorithm, heuristics does not guarantee optimal, or even feasible, solutions.

These algorithms are often statistical evaluations of the type of traffic being inspected. An example of a heuristic signature is a signature used to detect a port scan. This signature defines a particular threshold number of external probes against unique ports or a specific combination of ports. The signature may be further restricted by specification of the types of packets (for instance, SYN only) it reacts to. Interesting trends can be learned from these data, and it is possible to detect ongoing attacks based on these algorithms; however, the information that these systems provide is generally very nonspecific and requires extensive human investigation before actionable intelligence is gathered.

By creating baselines of normal behavior, anomaly-based NIDSs are able to detect when current network behavior deviates statistically from the norm. This capability theoretically gives an anomaly-based NIDS the capacity to detect new attacks that are either unknown or to detect attacks for which no signatures exist.

The major problem with this type of approach is that normal network traffic is difficult or impossible to define. Since normal network behavior can change easily and readily,anomaly-based NIDSs are prone to false positives. Additionally, inconsistency of detector performance, training issues (for example, how often an anomaly-based detection system should be retrained to ensure acceptable performance), and inadvertent incorporation of intrusive behavior into an NIDS concept of normal behavior during training negatively affect performance.

Network Intrusion Detection Systems Components

Most NIDSs are configured in a client- (sensor) to-server (management console) configuration. Many sensors normally report to one or several management consoles. Sensors can be dedicated appliances, can run as an application on a host running other applications, or can run independently in a virtual subsystem such as VMware or Xen. Note that if the sensor does not reside on a dedicated appliance, then the OS of the host computer should be hardened.

Because NIDSs do not reside in the datapath (normally one NIC is used as a sensor and a second NIC is used for management traffic), the sensor Ethernet interface can be configured in a number of ways as receive only. Sensor hardware requirements are not particularly strict since the sensor application normally inspects packets, and upon finding a signature or pattern match, sends the subsequent data upstream to the management console for processing and visualization.

The term “signature” refers to a set of conditions that, when met, indicate some type of intrusion event. Typical modern sensors contain a signature database consisting of 1000 to 2000 entries. Often, sensors inspect traffic based upon a mixture of signature matching as well as pattern matching. Pattern matching is based on looking for a fixed sequence of bytes in a single packet. A more sophisticated method is stateful pattern matching. Stateful pattern matching is useful when the intrusion signature spans more than a single packet. Similar to antivirus software, a signature-based IDS requires regular access to an up-to-date database of attack signatures so recent exploits are not missed.

Figure 1 is a simple illustration of the basic logic used by NIDS management stations when resolving an event reported by a remote sensor. The “Match IDS Rule” logic normally resides on the sensor. When a rule is matched (for example: “packet from outside to inside contains illegal SIP rerouting headers”), the data is forwarded to the management console where it is prioritized, logged, and visualized.

Figure 1: NIDS Logic

The management console (MC) hardware requirement is normally stricter than that of the sensor since the MC is responsible for data correlation from multiple sensors, as well as storage, alerting, and visualization. Often, the MC also includes an integrated sensor.

Network Intrusion Detection Systems

Network Intrusion Detection Systems (NIDSs) are designed to alert administrators when malicious or illegitimate traffic is detected. Malicious traffic can consist of worm or exploit-based code, while illegitimate traffic (often termed “misuse”) consists of traffic that deviates from established security policy such as surfing porn sites or peer-to-peer connections. Network-based IDSs can monitor an entire, large network with only a few well-situated nodes or devices and impose little overhead on a network. NIDSs are found in most networked computing environments today because, no matter how well security controls are implemented, it is impractical to maintain defenses against all known and potential threats to networked systems and applications. In VoIP environments, NIDSs provide an additional layer of defense.

NIDS Defined

NIDSs detect suspicious activity in three ways. First, the security community maintains an extremely large database of specific attack signatures. These signatures are programmed into the NIDS sensor, and are updated on a regular basis. Examples of attack signatures include Code Red, NIMDA, DoS attacks, buffer overflows, ASP, and CGI vulnerabilities. Second, the NIDS sensors contain preprocessors that continuously monitor the network for anomalous behavior. Though not as specific as attack signatures, these anomalies are still highly effective for the detection of port scans, distributed network probes, new forms of buffer overflows, and Denial-of-Service attacks. Third, all NIDS appliances can apply and detect security policy deviations. These policy deviations include the detection of unauthorized network services, applications running on unusual ports, and backdoor/Trojan activity.

Signature-based NIDSs are essentially network sniffers combined with a database of attack signatures. One of the most difficult (and necessary) tasks when initially configuring the NIDS is the job of de-tuning it. It is important that the number of false positives be reduced; otherwise, they will make meaningful analysis of the data impossible.