A comparative analysis of software-defined network controllers in terms of network forensics process

Abstract

Keywords: Computer Networks; Cyber Security; Forensics; Software-Defined Networks; OpenFlow; Southbound Interface; Ryu; ONOS; OpenDaylight; POX

Introduction

Software-defined network (SDN) is an innovative network architecture with lower cost, higher flexibility, better

agility, and simpler management compared to traditional networks [1]. It is hardware-independent and any network device from any manufacturer can be controlled with SDN. As shown in Figure 1, the basic structure of SDN consists

*Corresponding author. *E-mail address: altug.cil@tubitak.gov.tr This paper was recommended for publication in revised form by Editor-in-Chief Ahmet Selim Dalkılıç Published by Yıldız Technical University Press, İstanbul, Turkey Copyright 2021, Yıldız Technical University. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Sigma J Eng Nat Sci, Vol. 42, No. 2, pp. 425−437, April, 2024

of three different layers: The application, Control, and Data (Infrastructure) layer. The data layer contains physical or virtual forwarding devices, while the control layer is responsible for making the decisions that dictate the behavior in the network. These two layers communicate via the southbound interface. The application layer hosts various functions such as firewall, monitoring, load balancing, etc., and communicates with the control layer via the northbound interface. Keeping in mind the increasing prevalence of intelligent programmable devices in the network, SDN is able to provide new network management solutions to improve overall network performance [2]. With this new architecture, SDN makes it easier to focus on network management processes and offers network administrators great opportunities in terms of traffic monitoring and management [3]. Additionally, SDN provides increased security, easier policy management, and less energy usage [4]. OpenFlow is the most common southbound interface protocol used to provide a communication channel between the control and data layers in SDN. It facilitates innovation and enables efficient service orchestration and automation [5], and allows for the communication of devices in the data layer with the SDN environment [6]. Different tasks such as network management, security applications, and load balancing are realized in SDN by the flow rules generated by the controller and sent to the data layer using OpenFlow messages [7]. OpenFlow-enabled data layer devices contain flow tables to store rules with information on how to forward packets. For OpenFlow to be enabled in hybrid networks during the transition from traditional networks, certain sub-domains must be defined, and communication settings must be made for traditional networks [8]. SDN ensures the security of SDN infrastructures and leverages SDN-based approaches to improve security [5]. Research on SDN security is relatively new, thus requiring further analysis. The innovations provided by SDN allow

for improved security via security solutions produced with SDN itself, allowing for increased ability to follow up on attacks and facilitate attack detection [9]. These capabilities may be used to strengthen SDN forensics. Despite these advantages, a downfall is that SDN technology may be open to modern attacks [2]. SDN security research has focused on many different types of attacks and protocols. For instance, the attacker can create a new flow table with the help of a controller and forward the new rules to the network, mislead the network or create location spoofing by exploiting the weaknesses of the users through LLDP (Link Layer Discovery Protocol) packets [10]. In a solid infrastructure, a well-established authentication service between the switches and the controllers should be present. Otherwise, it will allow the attacker to examine the traffic [11]. An attacker can also perform a denial-of-service attack in a network environment using SDN by changing the flow tables [9]. This type of attack can be mixed with other attack types such as smurf attack, fraggle attack, and DNS amplification flooding attack. New methods have been employed to reduce the possibility of the controller being disabled [12]. The ARP (Address Resolution Protocol) poisoning attack may also be used to compromise the whole SDN topology by altering the identity of the controller, which is the most central and important part of SDN [13]. While many security issues and forensic processes in traditional networks are also applicable to SDN, new opportunities and new challenges have arisen in this young domain. This study considers the types of attacks emphasized in the literature in cyber attack scenarios within the scope of forensic experiments. Network forensics is of great importance in combating cyber attacks as it can enable both a better response during an attack and a more thorough forensic analysis after the attack. SDN offers a new and effective way to manage networks through programmability and increased flexibility [2]. As in all forensic analysis studies, the time record (stamp) must be kept in a reliable manner to comply with the legal ground. Network analysis is generally the examination of the data flowing through the network, focusing on the activities of the attacker [14]. The major questions to be answered by the analysis are “What”, “Why”, “Who”, “How,” and “When”. Network analysis techniques are composed of examination, report, analysis, investigation, presentation, identity, protection, decision-making, and collection and are grouped into proactive and reactive methods [15]. The general concepts in datasets that must be fulfilled are Findability, Accessibility, Interoperability, and Reusability (FAIR) [16]. To ensure information security in institutions, security systems like firewalls and intrusion detection systems (IDS) are used, and network analysis studies have a significant place in the development of these devices [17]. Although security studies have been carried out on SDN, they are poor in terms of forensic analysis [18]. Finding the root cause of attacks against SDN by applying forensic methods employed in traditional networks is insufficient and

Sigma J Eng Nat Sci, Vol. 42, No. 2, pp. 425−437, April, 2024

requires new approaches [19]. The innovations brought by SDN, such as data size, traffic monitoring, and the control layer being at the center of events, will allow for improved security [20]. Researchers have recently employed advanced methods such as deep learning to enable tracing the source of an attack [21]. This, in turn, will facilitate specifically targeting the source and identify the type of denial-of-service attack. Some researchers have gone beyond security analysis and developed modules that can solve system requirements. A module developed in the Python-based POX controller analyzes web server accesses to extract web statistics allowing to distinguish between web traffic and non-web traffic flows in the SDN infrastructure [22]. Ping flood attack packets on the POX controller can be prevented by detecting anomalies using machine learning [23]. Balarezo et al. carried out an experiment on denial-of-service attacks using TCP’s retransmission timeout to the southbound interface via OpenFlow, and they proposed a durable model by revealing the vulnerabilities of the system [24]. All layers of SDN can be used to obtain evidence against suspicious activity. The controller is a vital evidence source as it is placed in the center of the structure, but it also makes it a target for possible attacks, casting doubt on the reliability of the data. The northbound and southbound interfaces can also be used for forensic analysis as they connect the controller to applications and infrastructure [25]. Pandya et al. tried different ways to obtain data from three different SDN controllers (OpenDaylight, Project Floodlight, and Ryu), and examined them in the Wireshark environment. The study concluded that the southbound interface was the best location to obtain data. A Python-based tool was used to conclude that the OpenDaylight controller gave the most successful results in terms of forensics [18]. Khan et al. proposed a forensic layer where the northbound interface is excluded, and the data produced by the network devices is examined by placing a central layer outside the southbound interface [26]. In another study, some varying attack types were tested in an environment where only a single controller was used; Ryu. The study proposed that previous studies do not adequately focus on the data obtained from the controllers. Thus is the records created in the controller are vital in the detection of DoS attacks. However, the difficulty in keeping topology poisoning attacks in controllers arose due to the temporary nature of the flow table [27]. A new method titled “Virtual Network Forensic Process” was proposed but only focused on packets flowing over the network, and the records from the controller were ignored [28]. Network policies built on OpenFlow rules can be used by a malicious user for various attacks but can also be prevented [29]. In a study, forensic applications at different levels of SDN were carried out by considering the security perspectives called “SDNForensics”. In the framework where runtime logs, memory, and network packets were taken through three different data layers, memory information and logs were also obtained from users suggesting

the integration of sources [30]. Another study proposed a new tool seeking the cause of the problem by searching the control and data layers for deficiencies in connection and packet transmission. This tool is beneficial in SDN as it is designed to diagnose actual control layer attacks by placing itself on the floodlight controller located between the control and data layers [31]. SDN controllers, despite their similarities, also have differences between them in terms of speed, security, infrastructure, programming language, environment, etc. The objectives of this study are to • compare SDN controllers in terms of forensic informatics, taking into account the fundamentals of forensic information, • determine whether the southbound interface data is sufficient for forensics, • determine whether the best controller can be selected in terms of forensics by comparing different controllers during routine communication and under cyber attacks. Beyond these objectives, the determination of whether the forensic processes can be carried out with minimum data size and amount was emphasized in all processes of this study. The variable parameter in our study, which occurs in the data layer and the southbound interface, is the change of controllers. This study is different from similar articles in that it investigates the differences in the data that can be obtained from the southbound interface during routine communication and attacks for four popular controllers, which were evaluated and compared in terms of forensics for the first time in the literature. The test steps allow for the visualization of differences between the information obtained from the southbound interface and the virtual switches. Important parameters of forensics such as accessed information, access speed, packet size and percentage, and access time have been tested during routine communication and with attacks that are considered important in terms of cyber security. With the absence of similar studies, this article may inspire other studies comparing different SDN controllers in detail under various network attack scenarios and evaluating these issues in terms of forensic computing.

Methods And Experiments

Based on the problem of the lack of a standard forensic process in software-defined networks, the main idea of this study is the following research questions: • Are there differences in forensic processes in different controllers? • Is southbound interface data sufficient for the forensic process? • Is the behavior of the controllers and the evidence obtained different during routine communication and attack? • Is it possible to choose the best controller in terms of SDN forensics?

Sigma J Eng Nat Sci, Vol. 42, No. 2, pp. 425−437, April, 2024

Within the scope of these objectives, seven different scenarios detailed below were tested with four different controllers (Ryu, POX, ONOS, and OpenDaylight). As shown in Figure 2, the test environment for each experiment had a tree topology with 16 hosts with various tools in a virtual infrastructure. The tests were performed in the same way by changing only the controllers of each experimental environment. While performing the tests, the information created on the network could be watched live and stored for later analyses. Since the controllers were the only variable during the tests, it was possible to see what kind of differences the controller change made on the data layer and the southbound interface from the forensic point of view. Tests’ approaches were not only what data can be obtained, but also from the point of view of what size data can be accessed for the necessary information. In the experimental environment, the details given below, the primary focus is on the information that can be obtained in the data layer and southbound interface of software-defined networks during routine communication. Then, whether the message content detection can be done from these sections and whether it can be done in the case of which controllers are used were tested. Observations were made on how the controller changes differed in line with the purpose of port scanning and web server scans used in preparation for attacks. Finally, the results were obtained by applying different types of denial-of-service attacks separately for each controller to see how the correct information was obtained and the integrity of communication during the attack. The reason for choosing routine communication, message content detection, and denial-of-service attacks as different and independent from each other is to cover the entire forensic processes. As a result of varying controllers, different results from each scenario were obtained where attackers and victims were selected from 16 users. The details of

the network observations and experiments carried out are shared in detail under each heading.

Infrastructure Information In the SDN topology, Mininet [32] was used in a virtual server environment. In the relevant environment Ryu [33], ONOS [34], OpenDaylight [35], and POX [36] controllers were used. Ubuntu Desktop [37] was used as the environment infrastructure and as virtualization infrastructure, Virtualbox [38] were used on a laptop with 8 GB RAM, four virtual processors, and 150 GB HDD. The laptop had an Intel Core i7-10510U CPU 1.8GHz, 16GB DDR4 RAM, and 500GB SSD Disk capacity. OpenvSwitch [39] was used as a virtual switch. Packet inspections were carried out with Wireshark [40], and for forensic computing in network topology, listed programs were used; Yersinia [41], Nikto [42], Iperf [43], Nmap [44], Netcat and Hping3 [45]. Figure 2 shows the SDN environment, which is established with the help of Mininet, tree topology was used, and the experiments were carried out by creating a virtual environment consisting of 16 users (10.1.1.1-10.10.1.16), 15 virtual switches, and 1 controller. While Ryu, ONOS, and POX controllers were installed on Ubuntu 20.04 Desktop, the OpenDaylight controller was running on Ubuntu 14.04 Desktop. It was assumed that controllers have no rootkits and control channel is trusty. Before starting the experiments, the bandwidth was measured with the help of iperf in the topologies to which all controller types were connected. There were changes in bandwidth between the hosts depending on the controller type, but the lowest one had 1.5Gbits/s bandwidth in the normal operating mode. These values are sufficient to carry out the planned measurements and studies within the scope of this study carried out in the local network environment. Additionally, this study also has an experiment about

Sigma J Eng Nat Sci, Vol. 42, No. 2, pp. 425−437, April, 2024

whether the controllers changed the delay values between the hosts, and the results show us that the delay times were between 0.1ms and 0.25ms on average for each controller. Also, the experiments showed that the lowest latency values occur in the Ryu controller with an average of 0.1 ms.

Table 1. Routine communication forensics experiments packet count and size information

Forensic Process Start - Routine Communication In the routine communication experiment, simultaneous ICMP packets were transmitted from all hosts to each other on Mininet (with the pingall command), and all data for all controllers were captured and filtered. Figure 3 shows the processes performed in this experiment and re-run for all four controllers, with details given below. Then an investigation was made to observe what kind of differences occurred between topologies in which the controllers differ in the network communication, which can be considered the simplest level, the information to be obtained from the data layer, and the packet sizes. In the same scenario, all data such as time, source, and destination through the files of southbound interface network captures in topologies were accessible where the controllers were different. IP address and MAC address information were also available in all captures. Packets were obtained in environments where Ryu, ONOS, and OpenDaylight controllers have OpenFlow

Evaluation And Recommendations

While forensic computing is continuing to gain importance and with the difficulty of storing the expanding data size in the environment of the most popular four software-defined network controllers, the experiments done with these four controllers show results that support some examples in the literature with more details regarding forensic computing. The process of testing the durability

Sigma J Eng Nat Sci, Vol. 42, No. 2, pp. 425−437, April, 2024

Table 4. In the tests applied, whether the relevant scenario could be detected in the southbound interface of each controller (+ Detected, ± Partially Detected, - Undetected) Detection Status in Scenario (Southbound Interface)

of the controllers during routine forensic operations and attacks and revealing their differences by examining them in terms of forensic information was done for the first time. To interpret the whole process, data collection was on the main focus, which is an important early stage of forensic processes. Table 4 shows how the information and attacks in experiments can be obtained in the southbound interface packets of four different controllers directly or by interpreting the obtained packet information. The information in the table shows that in routine communication and attack scenarios, the highest detection success in southbound interface-oriented packets is achieved in experiments using the POX controller. The Ryu controller did not achieve detection using southbound interface packets in two of the scenarios, while OpenDaylight and ONOS were unable to detect CDP attacks. In experiments, routine communication could be detected on the southbound interface in every scenario using each of the different controllers, whereas the CDP attack was the most difficult experiment to detect in southbound interface packets. Beyond the most obvious parameters of forensic processes, such as attack detection, source, and target, it is also important that these detections can be made as quickly as

possible and with as little data as possible. Although experiments were carried out on the same topology and under the same conditions, there were significant differences in the average packet size as well as the number of packets and packet sizes that differ from each other in the southbound interface, as shown in Figure 8 & Figure 9. It was observed that the same situation even in the number of packets and packet size accumulated on the switches, especially in the port scanning scenario. Another remarkable result is that the completion time of the scenario with the POX controller in the web server vulnerability scanning experiment is 8-10 times longer than the other experiments. The fact that critical information cannot be seen on the controller, which is seen as the most successful in other attack types in some attacks, is proof that it would not be right to prefer a single controller for routine operations and attacks. The delay times were also different when the controllers were changed in routine communication, and lower delay values were obtained in different controllers in the scans. For example, when there was no operation on the network, the Ryu controller provided the lowest latency, while the lowest latency in Nmap scanning was provided in the environment where the OpenDaylight controller was running.

Figure 8. Southbound interface packets’ percentages in all test traffic.

Figure 9. Southbound interface packet sizes’ percentages in all test traffic.

Sigma J Eng Nat Sci, Vol. 42, No. 2, pp. 425−437, April, 2024

While there was a difference of up to 3 times in one controller in the same experiment between the number of packets of the switch interfaces to which the hosts are directly connected, there was almost no difference in the same experiment, while another controller was operating. As shown in this study, which also focuses on packet size and number, the small number of packets or the small size of the packet does not mean much in terms of forensic analysis. For example, the necessary determinations could not be made in the Ryu controller scenario with the lowest packet size and number in the web server scan. Despite this situation, it is an important part of the forensic method in software-defined networks. Table 5 shows an observational comparison where the most valuable results were obtained. It focuses on the controller with the lowest packet number, size, and percentage, the controllers which have the communication and attack information, time information, source-destination information, and where the system is disabled in the attack experiments and the southbound interface analysis. There are serious differences between the results from the experiments, and these conclusions can be a useful guide for researchers who will work in this context to choose the controller that best fits their usage ideas. Below, is a brief discussion of the results presented in Table 5. • The lowest percentages of packet count and size obtained from the southbound interface are in the OpenDaylight controller, and Ryu follows the OpenDaylight controller. • Obtaining time information is clearly critical in forensic computing. The information about the experiments performed on the packets obtained from the southbound interface can be accessed a large majority of the time. The inaccessibility of time information as a parameter related to the inability to detect the experiment. The results show that there will be no lack of

time information in packets from the southbound interface. Reaching the source and target packets can be considered as a parameter depending on the experiment not being detected and the time information is not being reached. As an exception to all experiments, in the scenario where the OpenDaylight controller was used for web server scanning, the source and destination addresses could not be reached in the southbound interface packets. • It is also important that the system from which the data will be retrieved is not out of service as a result of the attacks. Although the data is obtained not directly from the system itself but from the data flowing on the network, the deactivation of a centralized system such as a controller will lead to incomplete data. Some controllers went out of service shortly after some denial-of-service attacks started. The out-of-service controllers started to serve again after the attack ended, and in some of them, the communication was partially or completely restored. This shows that availability is a priority and an important issue for forensic investigations. • Overall, controllers that produced the best results in experiments considering all the forensic features are mentioned. The OpenDaylight controller was the most successful in three of the seven different experiments, the ONOS controller in two experiments, and the Ryu controller in one experiment. In the last experiment, no controller was successful in terms of forensic computing. According to the focus of new studies, it will be possible to determine which controller will be more useful with the help of this table as it summarizes attacks carried out for the first time with different controllers. Another issue that can be addressed beyond examining the results of all these experiments is why there are differences in these results. When these reasons are handled in

Table 5. The general result of applied experiments and attacks Detection Status in Lowest Scenario/ Packet Count Southbound Percentage Interface

Non-disabled Best Controller Controllers for Southbound Forensics

ONOS, POX

Sigma J Eng Nat Sci, Vol. 42, No. 2, pp. 425−437, April, 2024

the focus of this study, even though they are in the same topology in cases such as detecting, packet size, number of packets, and not being out of service, the big differences experienced only with the controller change can also create new study areas. To determine the reasons for the differences that come with the controller change, it will be useful to focus on the development of the controller, which is the only parameter that changes in the processes. In addition to OpenDaylight and ONOS controllers using the Java infrastructure, the development process of the related software is at the forefront of the differences that occur in the scenarios where the Ryu and POX controllers developed with the python language are used. This situation may cause differentiation of the logs to be obtained from the software directly on the controllers themselves. Even though the data obtained from the network environment contains major differences, this can be considered a threat in terms of the development of forensic processes in software-defined networks. In addition, although the communication between the control layer and the data layer is realized through OpenFlow, additional information, routing method differences, and other added rules specific to the controller also differentiate the forensic data that can be detected beyond the packet size and number. The results of this study, which are shown for the first time in the literature, are also likely to be guiding in the field of software-defined network forensics. For example, examining the topology type change in terms of software-defined network forensics and conducting studies on the change of forensic view capabilities as a result of the change of OpenFlow versions while using the same controller will increase forensic knowledge in the literature. In addition, it is possible to make improvements by transferring the good aspects of controllers in terms of forensic computing to other weak controllers in modules. Considering the differences in the experiments, it is extremely important to progress starting from the controller development stage and to consider a minimum forensic meeting expectation for all controllers as a standard to strengthen forensics in software-defined networks. This will prevent the differences in forensic detection shown in Tables 4 and 5, as well as the differences in the number and size of packets in Figures 8 and 9.

detection, attack and attacker information, and the effects of denial-of-service attacks, took into account. This study focuses on the ability to obtain the most meaningful communication data from the smallest possible size and number of packets. The speed of obtaining this data is also extremely important for live attack detection and affects the integrity of the entire network in communication. The results of experiments that were done with these points in mind show that different controllers show advantages and disadvantages in terms of event detection, packet size or service delivery in different tests, and differences between controllers in southbound interface-oriented forensic processes. The ONOS controller, one of the four different controllers, was used for the first time in this study, where data such as packet size, number and percentage, and information about it are shared. Another novel contribution of this study is an examination of the kind of information that can be accessed during routine operations and during cyber attacks and where this information can be obtained from in the network. This study also shows the new result about the service status of the four controllers during cyber attacks and reveals that the controllers also exhibit differences in terms of service continuity and quality. It would be beneficial to ensure that controllers meet certain standards to be dependable in forensic investigations while also being adaptable according to the laws and regulations of different states. The experiments show that there may be information deficiencies in some controllers and scenarios if the data is taken, collected, and stored only from the southbound interface, ignoring other data sources. However, as a common result, focusing on the southbound interface in terms of forensics in SDN will be the right choice due to its advantages, such as service delivery, detection, and log size. This will enable the necessary inspections to be carried out from a single point, with high quality, low overhead, and in a vendor-independent manner. On the other hand, contrary to the claims in certain previous studies, it is not correct to accept any single controller as the most successful option in terms of forensics. It is essential to examine the reasons for these differences in terms of forensic information and to design an SDN controller that provides optimum benefit in terms of forensic information. The implementation of a forensic management system as a general protocol will be of great benefit, considering the rapidly increasing use of SDN infrastructures. In this context, it is planned that these studies will continue, taking into account the specific situations of the application layer.

Conclusion

Network forensics has great potential in the SDN realm, but to fully exploit this potential, it is necessary to know the limitations and capabilities of SDN components in forensics. The controller is the most critical component in SDN as it maintains a central view of the entire network and makes forwarding decisions. In this study, for the first time in the literature, seven different experimental attack scenarios were applied with four different controllers, and the packets captured over the data layer and southbound interface were examined in detail. In doing so, the important issues in terms of forensics, such as packet size, source-target

Authorship Contributions

Altuğ Çil: Conceptualization, Methodology, Resources Software, Formal analysis, Investigation, Data curation, Writing – Original Draft, Visualization. Mehmet Demirci: Validation, Writing – Review & Editing, Supervision, Project administration.

Sigma J Eng Nat Sci, Vol. 42, No. 2, pp. 425−437, April, 2024

Data Availability Statement

The authors confirm that the data that supports the findings of this study are available within the article. Raw data that support the finding of this study are available from the corresponding author, upon reasonable request.

Conflict Of Interest

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethics

There are no ethical issues with the publication of this manuscript.

Share and Cite