Chapter 06: Network Security Using Cisco IOS IPS (Part02)

Host and Network IPS

IPS technology can be network based and host based. There are advantages and limitations to HIPS compared with network-based IPS. In many cases, the technologies are thought to be complementary.

Host-Based IPS

HIPS audits host log files, host file systems, and resources. A significant advantage of HIPS is that it can monitor operating system processes and protect critical system resources, including files that may exist only on that specific host. HIPS can combine the best features of antivirus, behavioral analysis, signature filters, network firewalls, and application firewalls in one package. Note that the Cisco HIPS solution, Cisco Security Agent (CSA), is signature-free that reduces the maintenance required to be performed on that software.

A simple form of HIPS enables system logging and log analysis on the host. However, this approach can be extremely labor intensive. When implementing HIPS, the CSA software should be installed on each host to monitor all activity performed on, and against, the host. CSA performs the intrusion detection analysis and protects the host.

A Cisco HIPS deployment using CSA provides proactive security by controlling access to system resources. This approach avoids the race to update defenses to keep up with the latest exploit, and protects hosts even on day zero of a new attack. For example, the Nimda and SQL Slammer worms did millions of dollars of damage to enterprises on the first day of their appearance, before updates were even available; however, a network protected with a CSA stopped these attacks without any updates by identifying their behavior as malicious.

Host-based IPS operates by detecting attacks that occur on a host on which it is installed. HIPS works by intercepting operating system and application calls, securing the operating system and application configurations, validating incoming service requests, and analyzing local log files for after-the-fact suspicious activity.

More precisely, HIPS functions according to the following steps, as shown in Figure 6-5:

Step 1	An application calls for system resources.
Step 2	HIPS checks the call against the policy.
Step 3	Requests are allowed or denied.

Figure 6-5: HIPS Operations Steps

HIPS uses rules that are based on a combination of known attack characteristics and a detailed knowledge of the operating system and specific applications running on the host. These rules enable HIPS to determine abnormal or out-of-bound activity and, therefore, prevent the host from executing commands that do not fit the correct behavior of the operating system or application.

HIPS improves the security of hosts and servers by using rules that control operating system and network stack behavior. Processor control limits activity such as buffer overflows, Registry updates, writes to the system directory, and the launching of installation programs. Regulation of network traffic can help ensure that the host does not participate in accepting or initiating FTP sessions, can rate-limit when a denial-of-service (DoS) attack is detected, or can keep the network stack from participating in a DoS attack.

The topology in Figure 6-6 shows a typical Cisco HIPS deployment. CSA is installed on publicly accessible servers, corporate mail servers, application servers, and on user desktops. CSA reports events to a central console server that is located inside the corporate firewall. CSA is managed from a central management console.

Figure 6-6: HIPS deployment

The advantages and limitations of HIPS are as follows:

• Advantages of HIPS: The success or failure of an attack can be readily determined. A network IPS sends an alarm upon the presence of intrusive activity but cannot always ascertain the success or failure of such an attack. HIPS does not have to worry about fragmentation attacks or variable Time to Live (TTL) attacks because the host stack takes care of these issues. If the network traffic stream is encrypted, HIPS has access to the traffic in unencrypted form.

• Limitations of HIPS: There are two major drawbacks to HIPS:

  • HIPS does not provide a complete network picture: Because HIPS examines information only at the local host level, HIPS has difficulty constructing an accurate network picture or coordinating the events happening across the entire network.

  • HIPS has a requirement to support multiple operating systems: HIPS needs to run on every system in the network. This requires verifying support for all the different operating systems used in your network.

Network-Based IPS

Network IPS involves the deployment of monitoring devices, or sensors, throughout the network to capture and analyze the traffic. Sensors detect malicious and unauthorized activity in real time and can take action when required. Sensors are deployed at designated network points that enable security managers to monitor network activity while it is occurring, regardless of the location of the attack target.

Network IPS sensors are usually tuned for intrusion prevention analysis. The underlying operating system of the platform on which the IPS software is mounted is stripped of unnecessary network services, and essential services are secured (that is, hardened). The hardware includes the following components:

• Network interface card (NIC): Network IPS must be able to connect to any network (Ethernet, Fast Ethernet, Gigabit Ethernet).

• Processor: Intrusion prevention requires CPU power to perform intrusion detection analysis and pattern matching.

• Memory: Intrusion detection analysis is memory intensive. Memory directly affects the capability of a network IPS to efficiently and accurately detect an attack.

Network IPS gives security managers real-time security insight into their networks regardless of network growth. Additional hosts can be added to protected networks without needing more sensors. When new networks are added, additional sensors are easy to deploy. Additional sensors are required only when their rated traffic capacity is exceeded, when their performance does not meet current needs, or when a revision in security policy or network design requires additional sensors to help enforce security boundaries.

Figure 6-7 shows a typical network IPS deployment. The key difference between this network IPS deployment example and the previous HIPS deployment example is that there is no CSA software on the various platforms. In this topology, the network IPS sensors are deployed at network entry points that protect critical network segments. The network segments have internal and external corporate resources. The sensors report to a central management and monitoring server that is located inside the corporate firewall.

Figure 6-7: Network-Based IPS Deployment

The advantages and limitations of network IPS are as follows:

• Advantages of network IPS: A network-based monitoring system has the benefit of easily seeing attacks that are occurring across the entire network. Seeing the attacks against the entire network gives a clear indication of the extent to which the network is being attacked. Furthermore, because the monitoring system is examining only traffic from the network, it does not have to support every type of operating system that is used on the network.

• Limitations of network IPS: Encryption of the network traffic stream can essentially blind network IPS. Reconstructing fragmented traffic can also be a difficult problem to solve. Possibly the biggest drawback to network-based monitoring is that as networks become larger (with respect to bandwidth), it becomes more difficult to place network IPS at a single location in the network and successfully capture all the traffic. Eliminating this problem requires the use of more sensors throughout the network. However, this solution increases costs.

Caution

It is recommended that applications responsible for the management of security, such as syslog servers, IPS alarms, and so on be separated from the main corporate network by a firewall, in essence creating a network management network. Figure 6-8 shows the details of the Enterprise Campus architecture as envisioned by the Cisco SAFE Blueprint. For more information, visit http://www.cisco.com. Figure 6-8: Enterprise Campus Topology with Its Management Module

Comparing HIPS and Network IPS

Table 6-5 compares the advantages and limitations of HIPS and network IPS.

Table 6-5: Advantages and Limitations of Host-Based IPS and Network-Based IPS
Open table as spreadsheet

	Advantages	Limitations
HIPS	Is host specific	Operating system dependent
	Protects host after decryption	Lower-level network events not seen
	Provides application-level encryption protection	Host is visible to attackers
Network IPS	Cost-effective	Cannot examine encrypted traffic
	Not visible on the network	Does not know whether an attack was successful
	Operating system independent
	Lower-level network events seen

A host-based monitoring system examines information at the local host or operating system. Network-based monitoring systems examine packets that are traveling through the network for known signs of intrusive activity. As you move down the feature list toward network IPS, the features describe network-based monitoring features; application-level encryption protection is a HIPS feature, whereas DoS prevention is a network IPS feature.

Note

Network-based monitoring systems do not assess the success or failure of the actual attacks. They only indicate the presence of intrusive activity. That is where Cisco MARS can be useful. Different sensors might report an intrusion; however, if all those sensors send their individual alarms to a Cisco MARS appliance, it could perform correlation analysis on those different alarms and discover that they are all part, let’s say, of a common attack.

Introducing Cisco IPS Appliances

Cisco IPS solutions run on a variety of devices, either as standalone sensors or as a module inserted into another appliance. The following is a brief description of the available Cisco IPS appliances. Each appliance is introduced further later in this section:

• Cisco Adaptive Security Appliance Advanced Inspection and Prevention Security Services Module (ASA AIP SSM): The Cisco ASA AIP SSM uses advanced inspection and prevention technology to provide high-performance security services, such as intrusion prevention services and advanced anti-x services, defined as antivirus and antispyware. The Cisco ASA AIP SSM products include a Cisco ASA AIP SSM-10 module with a 1-GB memory, a Cisco ASA SSM AIP-20 module with a 2-GB memory, and a Cisco ASA SSM AIP-40 module.

• Cisco IPS 4200 series sensors: Cisco IPS 4200 series sensors offer significant protection to your network by helping to detect, classify, and stop threats, including worms, spyware and adware, network viruses, and application abuse. Using Cisco IPS Sensor Software Version 5.1, the Cisco IPS solution combines inline intrusion prevention services with innovative technologies that improve accuracy. As a result, more threats can be stopped without the risk of dropping legitimate network traffic. Cisco IPS Sensor Software includes enhanced detection capabilities and improved scalability, resiliency, and so forth.

• Cisco Catalyst 6500 Series Intrusion Detection System Services Module (IDSM-2): The Catalyst 6500 Series IDSM-2 is part of the Cisco IPS solution. It works in combination with the other components to efficiently protect your data infrastructure. With the increased complexity of security threats, achieving efficient network intrusion security solutions is critical to maintaining a high level of protection. Vigilant protection ensures business continuity and minimizes the effect of costly intrusions.

• Cisco IPS Advanced Integration Module (AIM): Cisco offers a variety of IPS solutions; the Cisco IPS AIM for the Cisco 1841 Integrated Services Router and the Cisco 2800 and 3800 Series Integrated Services Routers is made for small and medium-sized business (SMB) and branch-office environments. Cisco IPS Sensor Software running on the Cisco IPS AIM provides advanced, enterprise-class IPS functions and meets the ever-increasing security needs of branch offices. The Cisco IPS AIM can scale in performance to match branch office WAN bandwidth requirements today and in the future, because IPS functionality is run on its dedicated CPU, thus not hogging the router CPU. At the same time, the integration of IPS onto a Cisco Integrated Services Router keeps the solution cost low and effective for business of all sizes.

Cisco IPS 4200 Series Sensors

The Cisco IPS 4200 series sensors, shown in Figure 6-9, are market-leading dedicated appliances for intrusion detection and prevention, with the highest performance and lowest false alarm rates of the industry. The Cisco IPS 4200 series sensors are focused on protecting network devices, services, and applications. They are capable of detecting sophisticated attacks such as the following:

• Network attacks

• Application attacks

• DoS attacks

• Fragmented attacks

• Whisker (deprecated in favor of Nikto) attacks using IDS-evasive techniques

Figure 6-9: Cisco IPS 4200 Series Sensors

Cisco ASA AIP SSM

The Cisco ASA AIP SSM, shown in Figure 6-10, provides the intrusion detection and prevention security feature set for the Cisco 5500 series adaptive security appliances. It runs the same Cisco IPS Sensor Software Version 6.0 or later software image as the sensor appliances and, therefore, provides the same security features as the sensor appliance.

Figure 6-10: Cisco ASA AIP SSM

The Cisco ASA AIP SSM is available in three models:

• The Cisco ASA AIP SSM-10

• The Cisco ASA AIP SSM-20

• The ASA AIP SSM-40

The Cisco ASA AIP SSM-20 has a faster processor and more memory than the Cisco ASA AIP SSM-10. The Cisco ASA AIP SSM-40 works only in the Cisco ASA 5520 and 5540 and has a maximum throughput of 650 Mb/s.

Tip

Although Cisco markets the AIP SSM as “full-featured intrusion prevention services,” it is worth noting that the sensor can operate as an IDS or IPS device. As shown in Figure 6-11, the AIP SSM can be configured in either IDS mode (promiscuous) or in IPS mode (inline). Figure 6-11: Modes of Operation for Cisco ASA AIP SSM

Cisco Catalyst 6500 Series IDSM-2

The Cisco Catalyst 6500 Series IDSM-2, shown in Figure 6-12, provides full-featured intrusion protection in the core network fabric device. The Cisco Catalyst 6500 Series IDSM-2 is specifically designed to address switched environments by integrating the IDS functionality directly into the switch. The Cisco Catalyst 6500 Series IDSM-2 runs the same software image as the sensor appliances and can be configured to perform intrusion prevention.

Figure 6-12: Cisco Catalyst 6500 Series ISDM-2 Module

Cisco IPS AIM

The Cisco IPS AIM for the Cisco 1841 and Cisco 2800 and 3800 Series Integrated Services Routers, shown in Figure 6-13, is an internal security service module that provides dedicated CPU and memory to offload inline and promiscuous intrusion prevention processing. The AIM runs the Cisco IPS Sensor Software Version 6.0 to provide feature parity with Cisco IPS 4200 series sensors and Cisco ASA 5500 series adaptive security appliances.

Figure 6-13: Cisco IPS AIM

By integrating IPS and branch-office routing, Cisco Integrated Services Routers can secure remote branch networks from threats originating from the Internet and reduce the WAN link overload from infected hosts at the branch. The integration of IPS into the branch-office router provides numerous important customer benefits:

• Physical space savings: The Cisco IPS AIM occupies the internal AIM slot on the router motherboard and can possibly saves space in the wiring closet.

• Inline and promiscuous modes: Both inline and promiscuous IPS inspection modes are supported. Inline mode places the IPS module in the packet path and can be configured to drop violated packets.

• Common management tool for Cisco IPS solution: Cisco Security Manager supports Cisco IPS AIM, with the same management tool used on Cisco IPS 4200 series sensors, enabling you to use one centralized management system for both appliance and router sensors.

• Flexibility in monitoring interfaces: The Cisco IPS AIM connects directly to the router backplane and can monitor packets coming in and going out of any router interface, including T1, T3, DSL, ATM, Fast Ethernet, and Gigabit Ethernet.

• In-band management: An internal Gigabit Ethernet port is used for in-band management of the Cisco IPS AIM CLI and for the web-based management application, Cisco IDM. Access to the IPS AIM can be done through the router console port or through the Secure Shell (SSH) protocol to any Layer 3 interface. No physical management port is required.

• Simple power and cable management: Cisco IPS AIM takes advantage of the power options of the router, including DC power and redundant power.

• Dedicated processor to maximize performance: Cisco IPS AIM has its own CPU and DRAM for all IPS functions. It offloads the router CPU from processor-intensive tasks, such as deep packet inspection from the host router.

• Performance: The Cisco IPS AIM can monitor up to 45 Mb/s of traffic and is suitable for T1, E1, and up to T3 environments.

• Security in depth: The Cisco IPS AIM interoperates with security and WAN optimization features such as VPN, firewall, Network Address Translation (NAT), Web Cache Control Protocol (WCCP), and Cisco Wide Area Application Services, and all common Cisco IOS Software functions.

Note

Cisco IOS IPS and the Cisco IPS AIM cannot be used together. Cisco IOS IPS must be disabled when the AIM IPS is installed. Cisco IOS IPS is discussed in the next section of this chapter.

Signatures and Signature Engines

A signature is a set of rules that an IDS and an IPS use to detect typical intrusive activity, such as DoS attacks. You can easily install signatures using IDS and IPS management software such as Cisco IDM. Sensors enable you to modify existing signatures and define new ones.

As sensors scan network packets, they use signatures to detect known attacks and respond with predefined actions. A malicious packet flow has a specific type of activity and signature, and an IDS or IPS sensor examines the data flow using many different signatures. When an IDS or IPS sensor matches a signature with a data flow, the sensor takes action, such as logging the event or sending an alarm to IDS or IPS management software, such as the Cisco SDM.

Signature-based intrusion detection can produce false positives because certain normal network activity can be misinterpreted as malicious activity. For example, some network applications or operating systems may send out numerous Internet Control Message Protocol (ICMP) messages, which a signature-based detection system might interpret as an attempt by an attacker to map out a network segment. You can minimize false positives by tuning your sensors. You can tune built-in signatures (tuned signatures) by adjusting the many signature parameters.

Examining Signature Micro-Engines

A signature micro-engine is a component of an IDS and IPS sensor that supports a group of signatures that are in a common category. Each engine is customized for the protocol and fields that it is designed to inspect and defines a set of legal parameters that have allowable ranges or sets of values. The signature micro-engines look for malicious activity in a specific protocol. Signatures can be defined for any of the supported signature micro-engines using the parameters offered by the supporting micro-engine. Packets are scanned by the micro-engines that understand the protocols contained in the packet.

Cisco signature micro-engines implement parallel scanning. All the signatures in a given signature micro-engine are scanned in parallel fashion, rather than serially. Each signature micro-engine extracts values from the packet and passes portions of the packet to the regular expression engine. The regular expression engine can search for multiple patterns at the same time (in parallel). Parallel scanning increases efficiency and results in higher throughput.

When IDS (promiscuous mode) or IPS (inline mode) is enabled, a signature micro-engine is loaded (or built) on to the router. When a signature micro-engine is built, the router may need to compile the regular expression found in a signature. Compiling a regular expression requires more memory than the final storage of the regular expression. Be sure to determine the final memory requirements of the finished signature before loading and merging signatures.

Note

A regular expression is a systematic way to specify a search for a pattern in a series of bytes. As an example, a regular expression to be used to prevent data containing .exe or .com or .bat from crossing the firewall could look like this: “.*\.([Ee][Xx][Ee]|[Cc][Oo][Mm]|[Bb][Aa][Tt])”.

Note

For the list of currently supported signature micro-engines, refer to the “Lists of Supported Signature Engines” section in the Cisco IOS Security Guide, Release 12.4 available at http://www.cisco.com/en/US/partner/products/ps6350/products_configuration_guide_chapter09186a00804453cf.html. This information requires a Cisco.com login.

Table 6-6 summarizes the types of signature engines available in Cisco IOS Release 12.4(6)T. Table 6-7 provides more details on signature engines.

Table 6-6: Summary of Supported Signature Engines
Open table as spreadsheet

Signature Engine	Description
Atomic	Signatures that examine simple packets, such as ICMP and UDP
Service	Signatures that examine the many services that are attacked
String	Signatures that use regular expression-based patterns to detect intrusions
Multi-string	Supports flexible pattern matching and supports Trend Labs signatures
Other	Internal engine to handle miscellaneous signatures

Table 6-7: Details on Signature Micro-Engines
Open table as spreadsheet

Signature Micro-Engine	Description
ATOMIC.IP	Provides simple Layer 3 IP alarms
ATOMIC.ICMP	Provides simple ICMP alarms based on these parameters—type, code, sequence, and ID
ATOMIC.IPOPTIONS	Provides simple alarms based on the decoding of Layer 3 options
ATOMIC.UDP	Provides simple UDP packet alarms based on these parameters: port, direction, and data length
ATOMIC.TCP	Provides simple TCP packet alarms based on these parameters: port, destination, and flags
SERVICE.DNS	Analyzes the Domain Name System (DNS) service
SERVICE.RPC	Analyzes the remote procedure call (RPC) service
SERVICE.SMTP	Inspects Simple Mail Transfer Protocol (SMTP)
SERVICE.HTTP	Provides HTTP protocol decode-based string engine; includes anti-evasive URL de-obfuscation
SERVICE.FTP	Provides FTP service special decode alarms
STRING.TCP	Offers TCP regular expression-based pattern inspection engine services
STRING.UDP	Offers UDP regular expression-based pattern inspection engine services
STRING.ICMP	Provides ICMP regular expression-based pattern inspection engine services
MULTI-STRING	Supports flexible pattern matching and supports Trend Labs signatures
Other	Provides internal engine to handle miscellaneous signatures

Note

It is recommended that you run Cisco IOS Release 12.4(11)T or later when using Cisco IOS IPS.

Note

Cisco IOS IPS and the Cisco IPS AIM cannot be used together. Cisco IOS IPS must be disabled when the AIM IPS is installed. Cisco IOS IPS is an IPS application that provides inspection capabilities for traffic flowing through the router. Although it is included in the Cisco IOS Advanced Security feature set, it uses the router CPU and shared memory pool to perform the inspection. Cisco IOS IPS also runs a subset of IPS signatures. The Cisco AIM IPS, discussed earlier in this chapter, runs with a dedicated CPU and memory, offloading all processing of IPS signatures from the router CPU. It can load a full signature set and provide enhanced IPS features not available on Cisco IOS IPS.

Signature Alarms

The capability of IDS and IPS sensors to accurately detect an attack or a policy violation and generate an alarm is critical to the functionality of the sensors. Attacks can generate the following types of alarms:

• False positive: A false positive is an alarm triggered by normal traffic or a benign action. Consider this scenario: A signature exists that generates alarms if the enable password of any network devices is entered incorrectly. A network administrator attempts to log in to a Cisco router but enters the wrong password. The IDS cannot distinguish between a rogue user and the network administrator, and it generates an alarm.

• False negative: A false negative occurs when a signature is not fired when offending traffic is detected. Offending traffic ranges from someone sending confidential documents outside of the corporate network to attacks against corporate web servers. False negatives are bugs in the IDS and IPS software and should be reported. A false negative should be considered a software bug only if the IDS and IPS have a signature that has been designed to detect the offending traffic.

• True positive: A true positive occurs when an IDS and IPS signature is correctly fired, and an alarm is generated, when offending traffic is detected. For example, consider a Unicode attack. Cisco IPS sensors have signatures that detect Unicode attacks against Microsoft Internet Information Services (IIS) web servers. If a Unicode attack is launched against Microsoft IIS web servers, the sensors detect the attack and generate an alarm.

• True negative: A true negative occurs when a signature is not fired when nonoffending traffic is captured and analyzed. In other words, the sensor does not fire an alarm when it captures and analyzes “normal” network traffic.

Table 6-8 provides a summary of the alarm types. To understand the terminology, think in terms of “Was the alarm triggered?” A positive means that the alarm was triggered and a negative means that the alarm was not triggered. Thus the expression false alarm, which is the same as false positive (positive because the alarm was triggered, but false because the intrusion did not happen or the intrusion was not detected by the sensor).

Table 6-8: Alarm Types
Open table as spreadsheet

	Intrusion Occurred/Detected	Intrusion Did Not Occur / Not Detected
Alarm was triggered	True positive	False positive
Alarm was not triggered	False negative	True negative

Alarms fire when specific parameters are met. You must balance the number of incorrect alarms that you can tolerate with the capability of the signature to detect actual intrusions. If you have too few alarms, you might be letting in more suspect packets, but network traffic will flow more quickly. If IPS systems use untuned signatures, they will produce many false positive alarms. You should consider the following factors when implementing alarms that a signature uses:

• The level assigned to the signature determines the alarm severity level.

• A Cisco IPS signature is assigned one of four severity levels:

  • Informational: Activity that triggers the signature is not considered an immediate threat, but the information provided is useful information.

  • Low: Abnormal network activity is detected that could be perceived as malicious, but an immediate threat is not likely.

  • Medium: Abnormal network activity is detected that could be perceived as malicious, and an immediate threat is likely.

  • High: Attacks used to gain access or cause a DoS attack are detected, and an immediate threat is extremely likely.

• You can manually adjust the severity level that an alarm produces.

• To minimize false positives, study your existing network traffic patterns and then tune your signatures to recognize intrusion patterns that are atypical (out of character) for your network traffic patterns. Do not base your signature tuning on traffic patterns that are based only on industry examples. Use an industry example as a starting point, determine what your own network traffic patterns are, and use them in your signature alarm tuning efforts.

---

Retiring Signatures

Router memory and resource constraints might prevent a router from loading all Cisco IOS IPS signatures. Therefore, it is recommended that you load only a selected set of signatures that are defined by the categories. Because the categories are applied in a “top-down” order, you should first retire all signatures, followed by “unretiring” specific categories. Retiring signatures enables the router to load information for all signatures, but the router will not build the parallel scanning data structure.

Retired signatures are not scanned by Cisco IOS IPS, so they will not fire alarms. If a signature is irrelevant to your network or if you want to save router memory, you should retire signatures, as appropriate. However, be aware that retiring and reinstating signatures are a CPU-intensive process. For more information about this topic, refer to http://www.cisco.com/en/US/docs/ios/12_4t/12_4t11/ips_v5.html.

---