Robert S. Sielken
Anita K. Jones
Application Intrusion Detection Systems: The Next Step
Operating system intrusion detection systems (OS IDS) are frequently insufficient to catch internal intruders who neither significantly deviate from expected behavior nor perform a sequence of specific intrusive actions. We hypothesize that application intrusion detection systems (AppIDS) can use application semantics to detect more subtle attacks such as those carried out by internal intruders who possess legitimate access to the system and act within their bounds of normal behavior, but who are actually abusing the system. To test this hypothesis, we developed two extensive case studies from which we were able to discern some similarities and differences between the OS IDS and AppIDS. In particular, an AppIDS can observe the monitored system with a higher resolution of observable entities than an OS IDS allowing tighter thresholds to be set for the AppIDS’ relations that differentiate normal and anomalous behavior thereby improving the overall effectiveness of the IDS.
As information systems have become more comprehensive and a higher value asset of organizations, intrusion detection systems have been incorporated as elements of operating systems, although not typically applications. Intrusion detection involves determining that some entity, an intruder, has attempted to gain, or worse, has gained unauthorized access to the system.
Intruders are classified into two groups. External intruders do not have any authorized access to the system they attack. Internal intruders have at least some authorized access to the system. Internal intruders are further subdivided into the following three categories. Masqueraders are external intruders who have succeeded in gaining access to the system and are acting as an authorized entity. Legitimate intruders have access to both the system and the data but misuse this access (misfeasors). Clandestine intruders have or have obtained supervisory (root) control of the system and as such can either operate below the level of auditing or can use the privileges to avoid being audited by stopping, modifying, or erasing the audit records [Anderson80].
Intrusion detection systems (IDS) have a few basic objectives. Among these objectives are Confidentiality, Integrity, Availability, and Accountability.
Intrusion detection has traditionally been performed at the operating system (OS) level mostly by comparing expected and observed system resource usage. OS intrusion detection systems (OS IDS) can only detect intruders, internal or external, who perform specific system actions in a specific sequence known to constitute an intrusion or those intruders whose behavior pattern statistically varies from a norm. Internal intruders are said to comprise at least fifty percent of intruders [ODS99], but OS intrusion detection systems are frequently insufficient to catch such intruders since they neither perform the specific intrusive actions because they are already legitimate users of the system, nor significantly deviate from expected behavior.
We hypothesize that application specific intrusion detection systems can use the semantics of the application to detect more subtle attacks such as those carried out by internal intruders who possess legitimate access to the system and its data and act within their bounds of normal behavior, but who are actually abusing the system. This research will explore the opportunities and limits of utilizing application semantics to detect internal intruders through general discussion and extensive examples. We will also investigate the potential for application intrusion detection systems (AppIDS) to cooperate with OS intrusion detection systems (OS IDS) to further increase the level of defense offered by the collective intrusion detection system.
The rest of the paper is structured as follows. The next section will describe OS intrusion detection and intrusion detection in general. Then, two case studies will be presented that will be followed by a section describing the observations obtained from the case studies. The last section will present general conclusions.
State of Practice – OS IDS
OS IDS that monitor resource usage of the operating system and the network represent the state of practice. They only monitor the resource usage of the application and not the application activity itself. OS IDS typically obtain the values necessary to perform intrusion detection from the existing audit records of the system.
Intrusion Detection Approaches
Currently there are two basic approaches to intrusion detection. The first approach, anomaly detection, attempts to define and characterize correct static form of data and/or acceptable dynamic behavior. In effect, it searches for an anomaly in either stored data or in the system activity. IDS utilizing anomaly detection include Tripwire [Kim93], Self-Nonself [Forrest94], and NIDES [Anderson95].
The second approach, called misuse detection, involves characterizing known ways to penetrate a system in the form of a pattern. Rules are defined to monitor system activity essentially looking for the pattern. The pattern may be a static bit string or describe a suspect set or sequence of events. The rules may be engineered to recognize an unfolding or partial pattern. IDS utilizing misuse detection include NIDES [Anderson95], MIDAS [Sebring88], and STAT [Porras92].
Intrusion detection systems have been built to explore both approaches: anomaly detection and misuse detection. In some cases, they are combined in a complementary way in a single intrusion detector. There is a consensus in the community that both approaches continue to have value. Systems also apply these same approaches to detect intrusions across a network of computers. Representative systems include NADIR [Hochberg93], NSTAT [Kemmerer97], GrIDS [Staniford-Chen96], and EMERALD [Porras97].
Generic Characteristics of IDS
After analyzing the approaches taken by IDS at the operating system and network levels, some generic characteristics of intrusion detection became apparent. To characterize OS ID and then compare it to Application Intrusion Detection (AppID), we first need to define some terminology that will allow us to discuss the characteristics of both more precisely. This terminology is similar to that used for prior software and hardware error detection research.
A relation is an expression of how two or more values are associated. An observable entity is any object, such as a user, data object, or system device, that has or produces a value in the monitored system that can be used in defining a relation. Examples of operating system level observable entities include CPU time usage, the number of files associated with a user, and the timestamp of the last modification to a file. There are two basic types of relations although some blending between the two is possible. Statistical relations can be used to compare the current value of an observable entity to a profile, a collection of statistical and other relevant information characterizing normal or anomalous behavior. These are most often used in anomaly detection. Rule-based relations relate the immediate or accumulated value to a predefined expected value and are most often used in misuse detection.
Thresholds can be set for the relations regardless of whether they are statistical or rule-based. Thresholds determine how the result of the relation will be interpreted; results outside of the threshold will be considered anomalous and results within the threshold will be considered normal. Thresholds are normally characterized by a certain number of standard deviations for statistical distributions or by a range, either fixed in size or as a percentage of the expected value, for rule-based analysis.
Setting the thresholds will impact the effectiveness of the IDS in detecting intrusions. Tighter thresholds, permitting less discrepancy, allow for greater detection but at the risk of more false alarms, an indication of an intrusion in the absence of an intrusion. Looser thresholds produce fewer false alarms but potentially at the cost of diminished detection.
The frequency with which a relation is evaluated can also impact the effectiveness of the intrusion detection system. It is possible for the IDS to evaluate all relations immediately after each event, the results of actions taken by users, processes, or devices that may be related to a potential intrusion. However, this may place an intolerable processing burden on the IDS. Therefore, events are typically collected in audit records over a period of time. Audit records entries can be reduced by combining some events into a single entry for analysis. For example, a single, failed log-in attempt is most likely insignificant, but many failed log-in attempts over a relatively short period of time may indicate a possible intrusion. The period of time between audit record analysis may be determined using real time or logical time where the relations are evaluated after a certain number of events have occurred. Audit records only deal with notions defined by the OS. Many aspects of the application are not visible to the OS and thus are not in the audit records.
Case Studies of Application Intrusion Detection
OS IDS have matured since their inception. However, the rate of improvements to their effectiveness in detecting intrusions has probably decreased as intruders have become increasingly savvy. Therefore, a significant change in the approach to intrusion detection is needed to further increase the effectiveness of intrusion detection. We hypothesize that major improvements may be made by incorporating intrusion detection into an application intrusion detection system (AppIDS). We use three questions to guide the exploration of using the basic intrusion detection techniques and the additional knowledge of application semantics to improve the effectiveness of intrusion detection.
Opportunity – what types of intrusions can be detected by an AppIDS, especially those not visible to an OS IDS?
Effectiveness – how well can those intrusions be detected by an AppIDS?
Cooperation – how can an AppIDS cooperate with the OS IDS to be more effective than either alone?
Since the concept of intrusion detection at the application level is fairly new, there is a lack of established literature on the subject for use in answering these questions. Therefore, we have decided to develop case studies. From them, we hope to glean some general understanding about AppIDS and determine its viability. By developing the examples, we also hope to develop a possible method of reasoning about such systems more generally.
Electronic Toll Collection
The first case study involves an electronic toll collection (ETC) system comprised of numerous devices interconnected to expedite toll collection on highways. Despite being specific to transportation, we felt that ETC provides a particularly interesting example because of certain properties that it possesses. We looked at several real ETC systems on which to base this case study, but to not increase the risk to any of the observed systems, this case study is based on a generic ETC system. The system incorporates numerous devices distributed throughout the transportation infrastructure to collect ETC specific data such as vehicle identity, weight, number of axles, and license plate numbers. These devices are configured in a hierarchical fashion. The ETC system differs from an OS in that it has these independent, but linked, devices from which it gathers data about the external behavior that it monitors.
The ETC system has a three level hierarchy. At the lowest level are the individual toll booth lanes and the equipment installed in each lane. A collection of adjacent toll lanes comprises a toll plaza, the middle level in the hierarchy. The toll management center, the single node constituting the highest level in the hierarchy, is the central control headquarters that manages all of the system’s toll plazas as well as any other devices from the highway system that are not directly related to the toll lanes or toll plazas.