Portal de Periódicos da CAPES

Detection and differentiation of application layer DDoS attack from flash events using fuzzy‐GA computation

2018; Institution of Engineering and Technology; Volume: 12; Issue: 6 Linguagem: Inglês

10.1049/iet-ifs.2017.0500

1751-8717

Khundrakpam Johnson Singh, Khelchandra Thongam, Tanmay De,

Advanced Malware Detection Techniques

IET Information SecurityVolume 12, Issue 6 p. 502-512 Research ArticleFree Access Detection and differentiation of application layer DDoS attack from flash events using fuzzy-GA computation Khundrakpam Johnson Singh, Corresponding Author Khundrakpam Johnson Singh johnkh34@gmail.com Department of CSE, NIT Durgapur, West Bengal, IndiaSearch for more papers by this authorKhelchandra Thongam, Khelchandra Thongam Department of CSE, NIT Manipur, Imphal, IndiaSearch for more papers by this authorTanmay De, Tanmay De Department of CSE, NIT Durgapur, West Bengal, IndiaSearch for more papers by this author Khundrakpam Johnson Singh, Corresponding Author Khundrakpam Johnson Singh johnkh34@gmail.com Department of CSE, NIT Durgapur, West Bengal, IndiaSearch for more papers by this authorKhelchandra Thongam, Khelchandra Thongam Department of CSE, NIT Manipur, Imphal, IndiaSearch for more papers by this authorTanmay De, Tanmay De Department of CSE, NIT Durgapur, West Bengal, IndiaSearch for more papers by this author First published: 01 November 2018 https://doi.org/10.1049/iet-ifs.2017.0500Citations: 9AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Distributed Denial-of-Service (DDoS) attacks are serious threats in the data center application, mainly affecting the web server. Even though there are various techniques to detect and mitigate such attacks so far they fail to meet in the case of application layer attack and Flash Events (FE). In the paper, we aim at detecting application layer DDoS attacks and distinguish it from FE. We have considered a DDoS attack model and selected the parameters in the incoming packets that correspond in causing the attack. Based on the attack model we have analysed the statistical parameters of the incoming packets such as inter-arrival time, the probability of uniqueness of an IP address in given time frame and the unavailability of HTTP (Hyper Text Transfer Protocol) GET acknowledgment bit in the header field. These parameters are the input to the Fuzzy classification model. We have used Genetic Algorithm (GA) to provide an optimised value range for the input parameters. The optimised values are now applied to Fuzzy logic to identify whether the web accessing clients shows the behavior of attack, normal or FE. The experimental results show that Fuzzy-GA model provides an accuracy of 98.4% in detecting DDoS attack and 97.3% in detecting FE.. Nomenclature probability of total attack consumption of victim resources total bandwidth of channel bandwidth consumed by attacking clients bandwidth consumed by legitimate clients probability of bandwidth consumption transfer packet size for the attacking clients in context to bandwidth transfer packet size for the legitimate clients in context to bandwidth average packet size inter-arrival rate of the attacking clients inter-arrival rate of the legitimate clients C number of open channels or free bandwidth total memory size of Victim probability of memory consumption average arrival rate of attacking users packets average arrival rate of legitimate users packets average processing time of attack packet(s) average processing time of legitimate packet(s) average processing time of incoming packet(s) average arrival time of the clients at victim end probability of central processing unit (CPU) consumption total CPU utilisation CPU consumption during attack period CPU consumption during normal period number of request during an attack period number of request during a normal period probability of not getting acknowledgment for HTTP requests 1 Introduction Cyber security is a major concern in today's society [1]. All information on a governmental organisation, military, corporate, business sector etc. are important and must be kept safe. This information must be out of reach from unauthorised or unintentional third-party. The confidentiality, integrity, and availability triads of information security must be fulfilled in the security system. In the current situation, there is a high risk of a security breach to gain access to the secured information. The most common threats are the distributed denial-of-service (DDoS) attack on the web server that tries to bring down a site out of enmity, competitions, ransoms, political issue, fun etc. [2]. A DDoS attack is the massive transmission of requests to the targeted victim server with an intention to make it offline [3]. To carry out such attacks, the attacker recruits a huge number of malicious clients called bots and forms a network called botnet [4]. The whole process of recruitment of bots and providing instruction or information to them for attacking an application server is the work of a bot-herder [5]. There are cases when a website suddenly put up a sale with a huge offer for a limited amount of time and customers. Seeing the advertisement, millions of clients access the given order of the sale. This activity is termed as a flash event (FE) [6, 7] and often leads to a slowdown of web access or sometimes crashes at a critical stage. A web server goes offline due to many reasons. One of the most crucial reasons is the DDoS attack on the web server. This attack not only exhausts server resources but also causes havoc to the legitimate clients. So, the detection of the DDoS attack becomes one of the primary objectives of current researchers. The DDoS attack is broadly divided into either network layer (layer 3) or application layer (layer 7) DDoS attack [8]. Layer 3 DDoS attack makes use of open system interconnection (OSI) layer 3 & 4 protocols such as internet control message protocol (ICMP), user datagram packets (UDP), transmission control protocol (TCP), synchronise (SYN), etc. Layer 7 DDoS attacks are caused by manipulating OSI layer 7 protocols such as HTTP, domain name service etc. Firewall and intrusion detection system is unable to detect layer 7 DDoS attack because of their silent use of genuine packets and protocols [5]. Therefore, in this study, we aim at detecting layer 7 DDoS attack. The FE has a similar effect on the web server as that of the layer 7 DDoS attack and therefore difficult to separate from the two [8, 9]. People assumed FE to be a layer 7 DDoS attack. To eradicate the confusion between the two, we need to differentiate between layer 7 DDoS attack and FE. A proper and separate solution must be proposed for each application layer DDoS (AL-DDoS) attack and FE. In the study, we have analysed the data taken from CAIDA 2007 dataset [10], EPA-HTTP dataset [11] and 1998 FIFA World Cup dataset [12]. Based on the heuristics, we have analysed the characteristics of the normal, attack and FEs. We have further examined the difference in the parameters for an attack, FE, and normal access through an attack model. We have analysed the inter-arrival time of the packet, the probability of uniqueness of an internet protocol (IP) address during a period and the unavailability of HTTP GET acknowledgement bit in the packet header. In the study, we find the optimal value range for each membership function (MF) of the linguistic values using a genetic algorithm (GA) [13]. The optimised range of the MFs is applied in Fuzzy computing [14] to obtain the status (either attack, normal or FE) of the clients accessing the web server through the computation of crisp output. Fuzzy logic is a mathematical method for answering questions with precise information. It deals with the approximate reasoning rather than fixed and accurate. In contrast to traditional logic theory where it is only true or false, fuzzy logic deals with the in-between. The rest of the paper is organised as follows: Section 2 provides some of the related works. Section 3 provides the attack model used. Section 4 describes the proposed methodology with parameters selection, optimisation, and fuzzy controls. Section 5 provides the experimental results and discussions. Finally, we present the concluding remarks in Section 6. 2 Related work Numerous research studies have been carried out in the field of intrusion detection and DDoS attack detection. A novel algorithm based on calibration of user's signature using completely automated public turing test to tell computer and human apart (CAPTCHA) or are you a human (AYAH) page is proposed [15]. This method generates a signature for each user and determines the behaviour of the user. The server load is used as a threshold to trigger the AYAH page. The method sends 1 AYAH page out of N request based on their signature. However, the computation and verification of the AYAH page is time-consuming and annoys the users similar to the CAPTCHA page. This method produces false positives and consumes more bandwidth in processing the AYAH pages. A wavelet recurrence clustering (WRC) and recurrence clustering (RC), intrusion detection model, based on the wavelet transformation and recurrence analysis is proposed [16]. The method considers the multi-scale property and recurrence assets of the network traffic by deploying wavelet transforming at a different time scale and dynamic characteristics of the network traffic at various frequencies, respectively. RC has the same methodology with the WRC except for the case that RC uses the recurrence properties rather than wavelet transformation. A supervised artificial neural network (ANN; feed-forward, error back-propagation with a sigmoid activation function) classification model is proposed [17]. The model uses the training pattern such as instances of packet headers, including source addresses, ID and sequence numbers along with source destination port numbers. The study analyses the DDoS attack carried out mostly by network protocols such as TCP, UDP, and ICMP. DDoS attack detection comes under malware detection tools. Malware detection tools are categorised into three groups such as static malware detection tools, dynamic malware detection tools, and online malware detection tools [18]. Resource Hacker, Regshot, and Virus Total are some of the malware detection tools under static, dynamic, and online malware detection tools, respectively. Resource Hacker is based on the signature of the incoming packets; Regshot relies on coloured set size for string matching, and Virus Total relies on the heuristic approach of anomaly detection. These tools are some of the most recent malware detection tools available. Zhou et al. [19] introduce a real-time mechanism for detecting layer 7 DDoS (AL-DDoS) attack in the backbone network. They construct a real-time frequency vector that characterises the traffic into a set of models. The study inspects the entropy of AL-DDoS and FE against the set of models generated and is used to detect the attack in real time. Detecting and distinguishing a DDoS attack from that of FEs by utilising traffic cluster entropy are proposed [20]. The clustered entropy is derived from the calculated entropy of the source address. Entropy features are useful, except that they can be spoofed. Depending on the header fields used, the entropy value may either increase or decrease. Liao et al. [21] proposed a new algorithm using K-nearest neighbour (KNN) classifier and found to be effective in text categorisation. In the study, for the experimental purpose, they utilise DARPA 1998 BSM audit data and are able to detect intrusive program behaviour. The experimental results show that the false positive rate (FPR) is reduced to a larger extent. The KNN classifier method works well with a dynamic environment which requires frequent updates of the training data. A mechanism known as Score for Core is proposed [22] for detecting and filtering DDoS attack packets. In this method, a score is calculated for every incoming packet using the nominal profile and current profile of the traffic. The method identifies whether the traffic is an attack or normal based on the calculated score of the incoming packets. The nominal profile is created by counting the number of packets during a normal period and the attack profile is created during the attack period. The method considers the attributes such as IP address, port number, protocol type, packet size, Time to Live (TTL) value, and TCP flag for the detection. The conventional classifiers provide a discrete boundary between the multiple classes for classification. This will increase the FPR while predicting the different classes. As we are discussing the detection of the DDoS attacks and FEs, we cannot consider the discrete boundary, but also need to deal with the values in between. Fuzzy logic will provide the solutions to the above raised problems as it deals with the degree of truth. Fuzzy has shown the great result for handling uncertainty problems. Uncertainty can be caused by vague knowledge or ambiguous information of the scenario. The problem for DDoS detection comes out to be an uncertainty problem. GA has been used to find the optimal MF of its subsets (linguistic values) and the optimisation has been done offline. For this reason and more we are applying a combination of the Fuzzy-GA method in the detection and differentiation of the AL-DDoS attack from the normal traffic and FEs. Most of the works mentioned in the related works are based on detection of layer 3 DDoS attack and therefore, we need to focus more on layer 7 DDoS attack. 3 Description of the attack model The DDoS attack on a victim server exhausted the resources of the target server. It makes the victim server refuse connection from new legitimate clients. The exhaustion of resources can be either concerning bandwidth or buffer size of the victim server. Equation (1) gives the total probability depletion of resources at the victim end [23]. All notations used in the attack model are described in the nomenclature. (1)To describe the attack model, we consider three different cases in the context of bandwidth depletion, memory exhaustion, and CPU utilisation. 3.1 Case 1: bandwidth depletion The probability of bandwidth depletion is given by the following equation [23, 24]: (2)where . Since throughput is given by the ratio of transfer packet size by transfer time we can further deduce the description of as We assume that the transmitted packet size is the same in case of an attack and normal traffic, therefore we have Since the ratio of the average packet size to that of the total bandwidth consumption is constant, we can rewrite the above equation as During a normal traffic, the inter-arrival time between two packets is much larger than that of an attack, we have , then (3)Using (2) and (3) we have the following two conclusions: (4) 3.2 Case 2: memory exhaustion The probability of memory or buffer exhaustion is given by the following equation [23-25]: (5)where Assuming the average processing time of the attack packets and normal packets to be similar we have Since the average arrival rate of the packets from the attacker is much larger than that of the normal client we have (6)Using (5) and (6) we have the following two conclusions (7) 3.3 Case 3: CPU consumption The probability of CPU consumption is given by (8)where Assuming the CPU usage of a request either normal or attack to be similar we have where R is the total request sent to the server (9)Using (8) and (9) we have the following two conclusions: (10)From (4), (7) and (10) we conclude the following points: Total exhaustion of resources (bandwidth and memory) of the victim depends inversely on the arrival rate of the attacking clients. It concludes that , and are directly proportional to the probability of the occurrence of IP addresses or number of accessing clients sending a request to the target server, as we are dealing with a DDoS attack. Since we are considering application layer protocol attack (HTTP), we consider as (11) The description of the HTTP ACK features is described in Section 4.1.3. 4 Proposed methodology Fig. 1 shows the workflow of the proposed detection mechanism. It consists of a packet capturing module, parameter selection module, optimisation module, fuzzy computing module and finally an intrusion analyser module. In the study, we have used network capturing tools such as Wireshark [26] to analyse the already obtained datasets namely CAIDA 2007, 1998 FIFA World Cup and EPA-HTTP datasets. The details of the remaining modules are explained in the following subsections. Fig. 1Open in figure viewerPowerPoint Architecture of the proposed DDoS detection system The objectives of the proposed method are stated as follows: What are the parameters of DDoS attack, FE and normal to be selected? How to provide a valid range of values for the MFs instead of assuming the ranges (expert knowledge)? How fuzzy control detects the behaviours of clients accessing the targeted web server and differentiates them from one another? The intrusion analyser module examines the outputs from the fuzzy controller and detects whether the incoming traffic is normal, FEs or DDoS attack. This module can be further extended in the classification of the detected DDoS attacks. Alert system and prevention mode are parts of this module. In the study, we are concentrating on the detection of layer 7 DDoS attack and segregation form FEs; hence the details of the intrusion analyser module will not be presented in the paper. 4.1 Parameters selected from the DDoS attack model It is mandatory to examine the web accessing behaviours to detect the AL-DDoS attack, FE, and normal clients. For the analysis of a DDoS attack, we have considered CAIDA 2007 dataset; to analyse the effect of FE, we have considered 1998 FIFA World Cup dataset. For analysing the normal behaviour, we have considered MIT Lincoln Laboratory tcpdump data [27] as it contains only pure data [28] and EPA-HTTP dataset for visualising the normal and abnormal working of the HTTP protocol. Based on the conclusion of the attack model discussed in Section 3, we have selected the statistical parameters of the clients accessing the web server. The first parameter is the inter-arrival time between the instances of an IP address. The second parameter is the probability of the uniqueness of an IP address within a period. The third parameter is the occurrence and non-occurrence of an HTTP GET acknowledgement bit in the packet header when clients request a web server. 4.1.1 First parameter: inter arrival time The DDoS attack is carried out by multiple clients. Since a medium attacker could not recruit massive clients, it triggers multiple instances from each client. A network of a bot called botnet has to bring down a web server during a short period. Therefore, the inter arrival time of the packets must be tiny. The clients have to send packets almost simultaneously to the web server so that the victim target is unable to respond to the requests. In CAIDA 2007 dataset, the attackers used the technique with a mean inter-arrival time of 0.017 s. That is the time gap between any two packets from the same client on an average is 0.017 s. However, during a normal traffic, the mean inter-arrival time from the same client on an average is 0.36 s. In this study, we have set 0.017 s as the lower bound threshold value; 0.36 s as the upper bound threshold value and assign the low and high value of the first parameter as in Algorithm 1 (see Fig. 2). Fig. 2Open in figure viewerPowerPoint Algorithm 1: Assignment of low and high value of inter-arrival time In the case of FE, the inter-arrival time between any two consecutive requests is also tiny. The only difference from DDoS attack is in the number of instances from an IP address, which is much larger in case of DDoS attack. The inter-arrival time of the packets in the case of attack, normal and FE for few clients is given in Fig. 3 a. Fig. 3Open in figure viewerPowerPoint Difference of Various features (a) Inter-arrival time difference for attack, FE, and normal clients, (b) Probability difference for attack, FE, and normal clients 4.1.2 Second parameter: probability of the uniqueness of an IP address During an attack, the attackers use instances from IP addresses to overwhelm the web server. Therefore, within a given time frame the probability of the occurrence of the same IP address in case of an attack is more than a normal scenario. In CAIDA 2007 dataset, the mean probability of the occurrence of same IP address in a 20 s time frame window is 0.276, i.e. 27.6%. In the case of FE, when we consider the World Cup 1998 dataset, the probability is 0.031 i.e. 3.1%. In this study, we have set the value as 0.076, i.e. 7.6% (MIT Lincoln tcpdump dataset) as the lower bound of the threshold value; 0.276 as the upper bound of the threshold value and assign the small and large value of the second parameter given by Algorithm 2 (see Fig. 4). We have selected the threshold value as the average number of occurrence of an IP address during the given time frame in a normal case. Fig. 4Open in figure viewerPowerPoint Algorithm 2: Assignment of small and large value The difference in the probability of uniqueness of an IP address in a given time frame window in the case of an attack, FE and a normal scenario is depicted in Fig. 3 b. 4.1.3 Third parameter To get the glimpse of the working of HTTP GET protocol, we have considered EPA-HTTP dataset. In case of a normal scenario, the sender sends an HTTP GET packet to the web server; the web server returns an acknowledgment for the request. There are even cases when the acknowledgment bit is not received at the sender side, then the sender retransmits the HTTP GET request after waiting for a fixed time interval. In case of an attack, the attacker sends an HTTP GET request and checks whether the web server is active or down. If it receives an acknowledgement, the attacker transmits massive HTTP GET request without even waiting for the acknowledgement [29]. Fig. 5 shows the bombardment of HTTP GET request from the attacker to the web server. Fig. 5Open in figure viewerPowerPoint Sequence of message use to realise HTTP GET attack In the case of FE, there are excessive transmissions of HTTP GET request from multiple clients but the acknowledgement from the web server is mandatory. In the study, we have assigned 'No' to the unavailability of HTTP GET acknowledgment and 'Yes' to the availability of its acknowledgment. The approximate ranges of value for the parameters discussed will be given by the optimisation technique in Section 4.2. 4.2 Optimisation with GA The input variables used for GA optimisation are followed: (a) TIMEIAT with the linguistic values low and high, where IAT is the mean inter arrival time of packets. (b) Prob with the linguistic values small and large, where Prob is the probability of the occurrence of a unique IP address within a given time frame window. (c) HTTPGET with the linguistic Yes and No, where HTTPGET is the arrival of acknowledgment bit form the receiver side for any HTTP GET request. The output variables used are as follows: (a) Status with the linguistic normal, FE and attack, where Status defines the behaviour of the web accessing clients. GA is used to find the optimised range of value for the above discussed parameters. The linguistic values in each input variable represent an MF. In the study, we have used a triangular MF as it has better performance than other MFs [30] and also easy to implement. The value of the input variables TIMEIAT, Prob and HTTPGET ranges from 0 to 1. The values of the output variables are 0.0, 0.5 and 1.0 for normal, FE and attack, respectively. The optimisation technique with GA is given step by step below: Step 1. Initialisation The six input MFs are low, high for the input TIMEIAT, small, large for Prob. No, yes for the HTTPGET of entry. In this problem, the unknown variables are the lengths of the bases of the MFs. In the study, we use 6-bit binary string to define the base of each six input MFs. The six strings, each of 6 bits are then concatenated to form a 36-bit string which will be a solution for the population. Step 2. Evaluation The evaluation stage consists of a mapping of values representing the lengths of the bases of the membership functions. This mapping process is computed using the following equation: (12)where and are user-defined constants and are usually chosen as the minimum and maximum value of the input variable. In (12), d is the decimal value of each sub-string; L is the number of bits in each sub-string and is the i th base of the MFs. In the beginning, the GA randomly creates a population of ten strings. For a string, the bases of the MFs are calculated using (12). The block diagram of triangular MFs for the antecedent and consequent part of the association rule is given in Figs. 6 a and b, respectively, where initial: i (i = 1, 2, 3), final: i (i = 1, 2, 3) and middle: i (i = 1, 2, 3) locates the two feet and peak value of the i th triangle, respectively; xd, xd 1, and xd 2 represent the index of fuzziness. Fig. 6Open in figure viewerPowerPoint Different MF representation (a) Triangular MF for antecedent, (b) Triangular MF for consequent of the association rule In the study, we consider the index of fuzziness to be 0.09. The calculated value of the base is used to find the initial, middle and final value (i.e. a, middle, and b) of the triangular MFs. The generalised form of computation for initial, middle and final values of the MFs are given in Table 1. Table 1. Triangular MFs and their corresponding values MFs Initial value Middle value Final value low alow = 0.0 middlelow = (alow + blow)/2 blow = base(low) high ahigh = blow –xd middlehigh = (ahigh + bhigh)/2 bhigh = 1.0 small asmall = 0.0 middlesmall = (asmall + bsmall)/2 bsmall = base(small) large alarge = bsmall −xd middle = (alarge + blarge)/2 blarge = 1.0 no ano = 0.0 middleno = (ano + bno)/2 bno = base(no) yes ayes = bno −xd middleyes = (ayes + byes)/2 byes = base(yes) normal an = 0.0 middlen = (an + bn)/2 bn = base(normal) FE aFE = bn –xd 1 middlelow = (aFE + bFE)/2 bFE = 0.5 attack aA = bFE –xd 2 middlelow = (aA + bA)/2 bA = 1.0 In Table 1, a, middle, and b are the initial, middle, and final value of the triangular MFs of the linguistic values. Let us consider an association rule of the form, where A is the antecedent part and the B is the consequent part; x1, x2, …, xn represent the antecedent attributes; yr represents the r th output consequent. Suppose we consider the association rule given as where O (i) represents the status of the participating client. We find the degree of membership of the input contained in the rule as follows: deg of mem for TIMEIAT = low(input1, alow, blow, middlelow) deg of mem for Prob = small(input2, asmall, bsmall, middlesmall) deg of mem for HTTPGET = no(input3, ano, bno, middleno) We then find the weight of rule i denoted by W [i] is given as follows: In this way, we find the weight of all the eight association rules given in the study (W [0], W [1],…, W [7]). Using the weights, we then compute the crisp output for column i input values given as (13)where v 1 to v 8 represent the values assigned for the status of the output. The status can be either normal (vi = 0), FE (vi = 0.5) or attack (vi = 1.0). The fitness value for the optimisation is given as (14)The values of the actual output are taken from the pre-observed data in Table 2. We assign 0 score for a normal client, 0.5 score for FE and 1 for an attack. The values in Table 2 are the mean values of the whole datasets taken. We have considered 22 distinct IP addresses and set them as a reference. To convert the function from minimisation to a maximisation problem, the fitness is subtracted from 10,000. The above processes are repeated for all the strings or solutions of the population to find the fitness of all the strings. Step 3. Selection We then choose a set of strings at the end of 1000 generation. In the study, the selected fitness value is 9994.77 using (11) at the end of 1000 generation. Step 4. Reproduction The population is modified using operators such as crossover and mutation. These whole processes (evaluation, selection, and reproduction) are repeated for many generations and finally, we then choose the bit string with the largest fitness value. This string with the largest fitness value will give the most optimal range of values for all the membership functions of the linguistic values given in Table 3. Table 2. Pre-observed output for training dataset through self- learning TIMEIAT Prob HTTPGET Status Actual output 0.016 0.194 0 attack 1 0.028 0.117 0 attack 1 0.017 0.187 0 attack 1 0.015 0.109 0 attack 1 0.021 0.184 0 attack 1 0.01 0.237 0 attack 1 0.014 0.154 0 attack 1 2.689 0.003 0 normal 0 1.263 0.002 0 normal 0 1.429 0.011 0