Traffic analysis

ABSTRACT
Due to the fact that, nowadays, it is possible to capture traffic in 1-10 Gigabit Ethernet networks using commodity hardware, many traffic monitoring systems, and especially capturing tools, have been proposed in recent years. This paper presents a comparison between two software probes named Adviser and Ksensor. Both of them are multi-processor systems and are built over conventional hardware. However, while Adviser is designed in user space, Ksensor runs in kernel space. This work compares the performance results of the two probes considering several capture engines (NAPI, PF_RING with DNA, PFQ) and, at the same time, different application or analysis loads. The evaluations of the probes with the different settings have been performed on the same hardware multi-core configuration. The results of the evaluations let conclude which solution is better in each situation and which solution must be discarded.

ABSTRACT
NQaS research group of the UPV/EHU focuses its activity on three main lines: the analysis of quality of service in data networks, the resource management in the distribution of multimedia content over NGN environments and the traffic analysis in high-speed networks using probes. This paper deals with the latter whose objective is to investigate the most efficient way to analyse the traffic in 1/10 Gbps or above networks with a traffic probe and to treat that data flow online. This paper shows the work that the group is developing, firstly, in the design of own software systems (Ksensor, Adviser) which are able to meet the computational needs that traffic analysis requires, secondly, in proposals for migration of certain components and logic to hardware systems based on FPGA in order to improve performance, and finally, in creating mathematical models to estimate the utilization of computational resources depending on the traffic analysis load.

ABSTRACT
Nowadays traffic monitoring is a must for manypurposes (IDS, antivirus, QoS monitoring, network problem detection, etc.) Deployment of high speed networks implies problems with these kind of systems to be able to cope with all the traffic in the network. Therefore, it would be interesting to know in advance whether our system will be able to do its task correctly, or it needs more processing power. This paper presents a simulator for network traffic capturing systems that use commodity hardware and general purpose operating systems. In order to establish the different elements of the simulator we carried out an in depth study of the network capturing subsystem in the Linux kernel. We identified the different stages of the travel of packets from wire to application, as well as the particular behavior of the system and computational cost for each one of them. With this information we have built up a model that simulates these different stages of a capturing system. This model allows us to estimate the performance a network application will be able to achieve, when packet losses will start and where they will appear.

ABSTRACT
The need to monitor and analyse network traffic grows with the deployment of new multimedia services over high speed networks. Predicting the overall capturing performance is crucial to know if the traffic monitoring system will be able to cope with all the traffic packets, or if it needs more processing power. In this paper, we present an analytical model based on a Markov chain to study the efficiency of the Linux network subsystem. Improving the capturing stage of Linux has been an extensively covered research topic in the past years. Although the majority of the proposals have been backed by experimental evaluations, there are few analytical models. We identify the softIRQ process as the main element in the Linux capturing stage and we have built a model that represents the different steps in the softIRQ and the computational cost for each one of them. The goodness of the model is checked by comparing analytical results with practical ones obtained from a real traffic monitoring system. Prior to obtaining the theoretical performance results, it is necessary to introduce some input parameters for the model. These initial necessary values are also extracted from experimental measurements, making use of an appropriate methodology. The results of all this process indicate us that the behaviour of the system performance depends on the network traffic rate and this has become our work in progress.

ABSTRACT
In traffic capturing and analysis systems, it is important to measure the performance in terms of throughput, packet loss, CPU availability, latency, interrupt frequency, etc. If these metrics are the result of theoretical assumptions, then it is necessary to validate those results by running appropriate testing. This paper presents a generic test framework composed of four elements (a manager, agents, daemons and formatters). With these four elements, every phase of the validation process is automated, from test configuration to result formatting. The architecture presented in this paper has been applied to validation tests of traffic monitoring systems devoted to high speed network traffic analysis. The performance tests have been made modifying different parameters such as packet injection rate, packet length, the number of processors on the probe, analysis load or probe’s configuration mode. In spite of having this infrastructure and configuration complexity, the deployment of this framework has led to a reduction in the time needed for the test phase and the number of errors due to mistakes.

ABSTRACT
Traffic monitoring is an increasingly important discipline for nowadays networking, as accounting, security and also Quality of Service (QoS) lay on it. Besides, traffic bandwidth has increased exponentially in the last few years, and high speed network monitoring is a challenging aim. Performance requirements are highly relevant for monitoring systems. In a previous work, our research group NQaS (Networking, Quality and Security) provided an architecture able to cope with high-speed traffic monitoring using commodity hardware. Its design was also intended to exploit the parallelism available. This paper shows the main features of this kernel-level monitoring system (ksensor) and presents an analytical model for a multiprocessor network traffic analysis system. The model which is based on Markov chains, evaluates the efficiency of the system. The goodness of the model will be checked by comparing analytical results with practical ones obtained in laboratory, using ksensor, which runs on a multiprocessor platform in the testing system.

ABSTRACT
Traffic monitoring is an increasingly important discipline for nowadays networking, as Accounting, Security and Traffic Engineering lay on it. Besides, traffic bandwidth has increased exponentially in the last few years, and high-speed network monitoring has become a challenging task. Performance requirements are highly relevant for passive QoS monitoring systems. A low-level study of the capturing and processing stages on a traffic analysis system (TAS) has shown room for improvement. We provide an architecture able to cope with high-speed traffic monitoring using commodity hardware. Our system is intended to exploit the parallelism available in up-to-date workstations, which also introduces constraints for multithreaded QoS analysis. This paper presents a kernel-level framework (ksensor) that, keeping the previous requirements, removes some issues from user-level processing and effectively integrates QoS algorithms, improving the overall performance.

ABSTRACT
Traffic monitoring is an increasingly important discipline for nowadays networking, as Accounting, Security and also Quality of Service (QoS) lay on it. Besides, traffic bandwidth has increased exponentially in the last few years, and high-speed network monitoring is a challenging aim. Performance requirements are highly relevant for monitoring systems. A low-level study of the capturing stages on a traffic analysis system (TAS) has shown room for improvement. We provide an architecture able to cope with high-speed traffic monitoring using commodity hardware. Our design is also intended to exploit the parallelism available in up-to-date workstations. This paper presents a kernel-level monitoring system (ksensor) that, keeping the previous requirements, removes some issues from user-level processing, improving the overall performance.

ABSTRACT
The importance of security issues in network environments has increased greatly lately. Intrusion Detection Systems play an important role in network security environments. Nevertheless, nowadays, data network speed is so high that performing intrusion detection tasks becomes challenging. This paper presents a software architecture that intends to exploit the parallelism available on up-to-date and future workstations to apply intrusion detection rules in high speed networks. To achieve this, a shared memory multiprocessor system has been developed. The system includes a powerful rule language that adds big flexibility to the system.

ABSTRACT
Simulation models have been developed in order to foresee characteristics of networks, systems or protocols when doing tests in laboratories are very expensive or even impossible. This paper presents a simulation model of a multiprocessor network traffic analysis system. The model, which is based on closed networks of queues, evaluates the efficiency of the system depending on the hardware/software platform features. Therefore, this model is able to estimate performance early in the design and development stages simulating a multiprocessor architecture in charge of analysing network traffic. The goodness of the model will be checked by comparing analytical results with practical ones obtained in laboratory using a traffic analysis system that runs in a multiprocessor platform.

ABSTRACT
Data communication is a widespread service in our society and the need for controlling the information interchange increases. Intending to solve this need, different kind of data capture systems (sensors or probes) have been implemented. The aim of this paper is to present a new architecture that analyses data providing essential information like complex quality of service statistics, intrusion detection, accounting... Nowadays, performance requirements limit the functionalities of sensors. Therefore, we have developed a high performance multiprocessor architecture which can process passively and online the captured packets in different ways. This improvement allows to apply complex QoS algorithms whose implementation is quite difficult if the number of connections to follow is high.

ABSTRACT
The current expansion of new services, which require high bandwidth rates, has enforced an increase of network throughput hindering network monitoring. Conventional network traffic analysis tools like tcpdump or ethereal were not designed to keep up with new rates so loss rates are very high. Consequently, we have developed a sensor that exploits multiprocessor features in order to reduce loss rates in high speed networks. Most network traffic analysis require communication among instances yielding synchronization troubles that must be solved. As the sensor splits traffic analysis loads among different processes, they will concurrently access to shared memory areas. Conventional synchronization mechanisms like semaphores are not a good approach because they harm sensor’s performance highly. Hence, in this paper we introduce a number of mechanisms that improve the overall performance of the sensor as a result of an effective communication among instances.

ABSTRACT
This document describes several design proposals to enhance network sensor performance on multiprocessor architectures. Our main contributions are related to the design of an autonomous sensor and to the idea of performing some parallelization of the analysis. These proposals can be implemented in network sensors such as intrusion detection systems, network antivirus appliances, QoS monitors and any other device based on network traffic analysing. Taking a certain model of traffic analysis as our starting point, we look deeply into some design proposals to address the difficulties involved in the parallelization. In this work, we propose a series of resources that can help us to solve these difficulties. Later, we study the prototypes developed in order to test different design alternatives and, finally, present selected case studies. We finish by quantitatively analysing the results to validate our design proposals.