Portal de Periódicos da CAPES

Agile Management and Interoperability Testing of SDN/NFV-Enriched 5G Core Networks

2018; Electronics and Telecommunications Research Institute; Volume: 40; Issue: 1 Linguagem: Inglês

10.4218/etrij.2017-0236

2233-7326

Taesang Choi, Tae‐Yeon Kim, Wouter Tavernier, Aki Korvala, Jussi Pajunpää,

Advanced MIMO Systems Optimization

ETRI JournalVolume 40, Issue 1 p. 72-88 ArticleFree Access Agile Management and Interoperability Testing of SDN/NFV-Enriched 5G Core Networks Taesang Choi, Corresponding Author Taesang Choi choits@etri.re.kr Search for more papers by this authorTaeYeon Kim, TaeYeon KimSearch for more papers by this authorWouter Tavernier, Wouter TavernierSearch for more papers by this authorAki Korvala, Aki KorvalaSearch for more papers by this authorJussi Pajunpää, Jussi PajunpääSearch for more papers by this author Taesang Choi, Corresponding Author Taesang Choi choits@etri.re.kr Search for more papers by this authorTaeYeon Kim, TaeYeon KimSearch for more papers by this authorWouter Tavernier, Wouter TavernierSearch for more papers by this authorAki Korvala, Aki KorvalaSearch for more papers by this authorJussi Pajunpää, Jussi PajunpääSearch for more papers by this author First published: 15 February 2018 https://doi.org/10.4218/etrij.2017-0236Citations: 7 Taesang Choi (corresponding author, choits@etri.re.kr) and TaeYeon Kim (tykim@etri.re.kr) are with the Hyper-connected Communication Research Laboratory, ETRI, Daejeon, Rep. of Korea.Wouter Tavernier (Wouter.Tavernier@UGent.be) is with the Department of Information Technology, Gent University, Belgium.Aki Korvala (aki.p.korvala@nokia.com) and Jussi Pajunpaa (jussi.pajunpaa@nokia.com) are with Nokia, Oulu, Finland. AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract In the fifth generation (5G) era, the radio internet protocol capacity is expected to reach 20 Gb/s per sector, and ultralarge content traffic will travel across a faster wireless/wireline access network and packet core network. Moreover, the massive and mission-critical Internet of Things is the main differentiator of 5G services. These types of real-time and large-bandwidth-consuming services require a radio latency of less than 1 ms and an end-to-end latency of less than a few milliseconds. By distributing 5G core nodes closer to cell sites, the backhaul traffic volume and latency can be significantly reduced by having mobile devices download content immediately from a closer content server. In this paper, we propose a novel solution based on software-defined network and network function virtualization technologies in order to achieve agile management of 5G core network functionalities with a proof-of-concept implementation targeted for the PyeongChang Winter Olympics and describe the results of interoperability testing experiences between two core networks. 1 Introduction In the fifth generation (5G) era, the radio internet protocol (IP) capacity is expected to reach 20 Gb/s per sector (mobile speeds up to 20 Gb/s), and ultralarge content traffic (for example, ultrahigh definition video streaming, augmented reality (AR), and virtual reality) will travel across a faster wireless/wireline access network. All 5G mobile/fixed traffic has to travel via the packet core network (CN). Currently, in the fourth generation (4G), most mobile operators (even large-scale ones) have only a few sites with packet gateways (PGWs) across their entire networks. The software-defined network (SDN) paradigm provides a new capability for faster service provisioning of the 5G CN through standard programmable interfaces. Moreover, with cloud computing, datacenters promote the on-demand provisioning of computing resources and services 1. If the 5G core nodes are distributed closer to cell sites, content servers (or caching servers) can be placed on the rack right next to the distributed 5G core with network function virtualization (NFV) technologies. This can help significantly reduce backhaul traffic by having mobile devices download content immediately from the content server. Thus, it is desirable to distribute packet core functionality to a number of local sites near end users in the coming 5G era. The 5G core functionality and applications can then run on virtualized servers at the local network sites. Other important 5G services—massive and mission-critical Internet of Things (IoT) services—are the main differentiator from 4G services. Mission-critical IoT (ultrareliable and low-latency communications) applications include remote-controlled machines, autonomous driving, and others. These types of ultra-real-time services require a radio latency of less than 1 ms and an end-to-end latency of less than a few milliseconds 2. To address such challenges, we present a novel agile management and orchestration (MANO) architecture based on enabling key technologies for 5G core functionalities, a proof-of-concept (PoC) implementation targeted for PyeongChang Winter Olympics, and deployment and interoperability testing experiences. The proposed solution is an interim result of a collaboration project between the Republic of Korea (KR) and the European Union (EU) 3. The rest of the paper is organized as follows. We describe the enabling key technologies in Section II. We present the agile MANO architecture in Section III. Our prototype implementation and deployment experiences are described in Section IV. A performance evaluation of the proposed system, including the interoperability testing results, are provided in Section V. Finally, we conclude our paper with the plans for potential future work in Section VI. 2 Enabling Key Technologies This section examines the key technologies for the SDN, NFV, MANO, mobile edge computing (MEC), mobility management, and control plane (CP) security and their associated design principles for the support of the proposed CN functionalities and their agile management. 2.1 Software-Defined Networking and Orchestration Standardization efforts for SDNs were mainly carried out by the Open Networking Forum 4 and the International Telecommunications Union – Telecommunications Study Group 13 (ITU-T SG13) 5 by defining the requirements, reference architecture, protocols, and use cases. Open-source projects such as Open Daylight 6 and Open Networking OS 7 have played important roles in realizing the SDN concept in real life. The SDN started with a limited networking environment such as cloud data centers and enterprise networks and has widened its coverage to wide-area transport networks and wireless/wireline integrated multidomain networks. Instead of applying it as a standalone network control tool, it is now used with NFV and as a component of an end-to-end orchestration solution. It provides an intelligent knowledge plane for making control decisions via traffic steering, traffic engineering, and flexible service chaining for latency-sensitive and reliability-seeking applications. It can be used in efficient communications among distributed core functional components. 2.2 Network Function Virtualization The virtualization of core and radio access network functions will optimize the use of network resources and add scalability and agility. To this end, the European Telecommunications Standards Institute (ETSI) NFV Industry Specification Group has defined the architecture, open application programming interfaces (APIs), and reference points, leveraging open-source PoC projects and communities to drive open standards of NFV. In 2016, it published Release 2 specifications and reports, including the functional requirements, interface, and information model for the reference points for the MANO function block called NFV-MANO 8. These open standards are intended to enable third-party vendors to develop framework components that can collaborate with various vendor components so that content service providers) are not restricted in selecting functional and management components. The main appeal of the use of NFV to deploy network elements and virtual network functions (VNFs) is that services can be launched more quickly by installing software on a standard hardware platform. This is akin to the way software applications could be developed and launched for the personal computer (PC) platform when it first emerged. Another advantage is lower capital expenditures because standardized hardware platforms tend to drive costs down. Such advantages can be directly applied to the distributed core functional components in the communications environment. 2.3 Mobile Edge Computing In order to support the requirements for the market's expected throughput, latency, scalability, and programmability, ETSI established the Industry Specification Group on Mobile Edge Computing in 2014 9. It develops a standardized and open environment that offers distributed cloud-computing capabilities and an IT service environment for application developers and content providers. By February 2016, the group finalized three specifications: the terminology, the technical requirements, and the framework and reference architecture. This group also works on specifications for MEC platform application enablement, the API principles and guidelines, the service APIs for radio network information and location, user equipment (UE) identity and bandwidth management, system/host/platform management, lifecycle and policy management, the UE application interface, the deployment of MEC in an NFV environment, and the end-to-end mobility. By offering distributed cloud-computing capabilities and exposure to real-time radio network and context information, MEC provides the following characteristics: Ultralow latency: Mobile edge services can be run close to end-user devices to provide the lowest possible latency, Proximity: Being close to the source of information, MEC is particularly useful for capturing key information for analytics and big data, High Bandwidth: The mobile edge location at the edge of the network combined with the use of real-time radio network information can be used to optimize the bandwidth for applications, Location awareness: A mobile edge can leverage the low-level signaling information to determine the location of each connected device, Real-time insight into radio network and context information: Real-time network data can be used by the applications and services to offer context-related services. MEC can provide a significant improvement in a mobile user's quality of experience for latency- or quality of service (QoS)-sensitive services such as edge video orchestration, mobile video throughput guidance, AR, intelligent video analytics, and others. Most importantly, MEC enables the implementation of mobile edge applications as software-only entities that run on top of a virtualization infrastructure, which is located in or close to the network edge. 2.4 Distributed Mobility Management It is essential to support distributed mobility management to enable agile management of the CN functionality. Currently, the Internet Engineering Task Force is conducting standardization efforts to define a distributed mobility management architecture and mechanism in a layer 3 IP network environment. The 3rd Generation Partnership Project (3GPP) also initiated work on defining layer 2 distributed mobility management requirements for a mobile communications environment. The functional decomposition and distribution of global service management will span multiple points of presence (PoPs) over the network, including network slices in a 5G environment. It would be better to determine the anchoring and mobility management tailored to such a network environment at the central node, unlike exiting hierarchical and IP mobility. Composition functions and resources will be orchestrated for dynamic mobility management. Various experiments and simulations are under study by the research community, and extensive testing and verification of the concepts of distributed mobility management are needed. 2.5 Security of the 5G Core Network Control Plane A software-defined mobile network (SDMN) controller will provide the necessary services to the CN functions by working as an intermediary between the access and core functions. The network control functions of the core elements, for example, the mobility management entity (MME), serving/packet data network gateways (S/P-GWs), and others, will reside in a centralized cloud in the form of SDN applications that will leverage NFV technology to be instantiated on different hardware or even at different network perimeters for a higher scalability and availability. Hence, the main security concern in such architectures will be the SDN controller since it can become a potential bottleneck for the overall network. To mitigate the risks of controller failure due to scalability or the chances of denial of service (DoS) attacks due to its centralized role, controller resilience strategies have been proposed. These strategies include controller resilience through redundancy, maximizing the storage and processing capabilities of the controller, and distributing controller functionalities among multiple control points in the network. The OpenFlow variant of an SDN supports wildcard rules so that the controller sends an aggregate of client requests to server replicas. By default, microflow requests are handled by the controller that can create potential scalability challenges, increasing the chances of failures due to DoS attacks. Normally, reactive controllers that act on a flow request when it arrives at the controller are used. Proactive controllers install flow rules in advance, thus minimizing the flow request queue in the controller. Similarly, various load-balancing techniques that would balance the load among multiple controllers in a network have been suggested. We have worked on a novel communication architecture based on the host identity protocol (HIP) to secure both control and data channels in SDMNs. 3 System Architecture This section describes the proposed CN and agile management system architecture based on a combination of the key technologies described in Section II. 3.1 Core Network Architecture We designed our CN architecture (Fig. 1) 10 to support CN functionalities and agile management on the basis of the various key technologies described in Section II. Specifically, the CN functionality is realized by leveraging an SDN and NFV in order to facilitate the dynamic provisioning of CN functions. By using SDN capabilities, traffic flows can be dynamically controlled, redirecting the traffic to gateways according to the workloads. Simultaneously, the introduction of NFV permits the separation of service functionalities from the capacity-constrained specific network entities and allow dynamic instantiation in commodity and powerful servers. Starting from late 1990s, the 3GPP has been taking steps towards a clear separation of the data and control planes and the respective elements in the architecture. We propose to take this concept to the next level following the SDN paradigm. Figure 1 also presents the 5G network control as a group of SDN applications. They are the Base Station App, Backhaul App, Mobility Management App (MM App), Monitoring App, Access App, and Secure Service Delivery App. The network applications are orchestrated via the Controller Northbound API. Multiple SDN applications operate without conflicts. Figure 1Open in figure viewerPowerPoint CN architecture. The Base Station App runs the control software that is now vertically integrated with the evolved Node B (eNB). The physical base stations under its control consist of an antenna, a band-pass filter, and an Ethernet card for backhaul connectivity 11. The MM App implements mobility as a service and incorporates the MME. In addition, it needs to manage the QoS for each user, balance the load among alternative paths across the aggregation network, and route the user to a cache when possible. The MM App also chooses the path for a device. The load-balancing decision is made on the basis of the input from the Network Monitoring App. In any case, it is desirable that the point of attachment of a mobile device to the Internet is fixed while it remains within the coverage of the current mobile network 11. In one physical mobile network, there may be many Access Apps. In this case, an Access App is owned and operated by a particular mobile virtual network operator. Putting mobility aside, the Access App is responsible for providing data services to mobile users. The key properties of the Access App include providing Internet access, firewalling unwanted traffic, and providing access to premium content 11. The main CN functions are designed and implemented in the form of virtual functions, namely, virtual evolved packet cores (vEPCs). Both the EU and KR provide their own implementations of vEPCs based on this architecture. They are described as follows. 3.2 European vEPC Architecture (5GTN) The EU vEPC consists of the following VNFs: Mobile gateway: The cloud mobile gateway provides the service provider-gateway (SP-GW), gateway general packet radio service (GPRS) support node, and traffic detention functions (TDFs), evolved packet data gateway, and trusted wireless access gateway. Mobility management: The cloud mobility manager provides the MME and servicing GPRS support node functions. Policy control and charging: The dynamic services controller built on patented agile rules technology engine provides the policy and charging rules function (PCRF) and wireline radius/change of authorization. Element and network management: The service-aware manager provides end-to-end network management visibility across the entire mobile network. To support the scalability required to meet the expected 5G and IoT service requirements, the packet core VNFs provide three key design innovations: The packet core VNFs are decomposed into separate CP and data-plane virtual machine (VM) instances. This enables a distributed architecture where data-plane resources can be deployed in edge data centers closer to the device, while CP resources can be centralized. State-efficient VNF processing unpins the subscriber/device state information from the VMs, freeing up the underlying computing resources to be reused to process other subscribers/devices. The remote cloud database synchronizes the subscriber/device state information into a real-time data store. The 5GTN functional architecture 10 is given in Fig. 2. Figure 2Open in figure viewerPowerPoint 5GTN vEPC functional architecture. 3.3 Korean vEPC Architecture The CN of 4G Long Term Evolution (LTE) is in charge of mobility, authentication, and charging, allowing all mobile traffic to pass through the CN to access services incurring traffic congestion in the CNs. Our architectural decision for 5G is to distribute mobile core functions to the edge nodes. A 5G core is generally divided into a 5G core user plane (UP) in charge of bearer delivery and a 5G Core CP in charge of signaling and control of the 5G CN. The key CN architectural design principle is a centralized CP with a distributed UP over the edge nodes. If the CN where bearers are terminated is located closer to the cell sites, the application servers follow naturally, and the backhaul traffic will significantly decrease, resulting in a cost reduction for continual backhaul enhancement. A 5G network is supposed to be able to provide ultra-real-time services such as highly sensitive remote control and automatic driving vehicles. These types of services may generate much lesser traffic than video streaming applications but require an ultralow latency. Figure 3 illustrates the high-level architecture of a Korean vEPC. It is realized as a highly scalable vEPC (HSvEPC) 12. Its functionality and architecture are described below. Figure 3Open in figure viewerPowerPoint High-level architecture of Korea's distributed vEPC. 3.4 HSvEPC Network Architecture It is possible to deploy different types of virtual mobile packet cores depending on the demand or network access environment in an HSvEPC network architecture. Two types of vEPCs are designed (shown in Figs. 4 and 5): Figure 4Open in figure viewerPowerPoint HSvEPC functional architecture: S-vEPC. Figure 5Open in figure viewerPowerPoint HSvEPC functional architecture: MHN-vEPC. Split vEPC (S-vEPC): The first type is an expansion of a vEPC by separating conventional consolidated functions into UP and CP functions for dynamic scaling operations. Mobile hotspot network vEPC (MHN-vEPC): The other type is an optimized case for a hotspot area to enhance the agility of the network. For faster and more dynamic mobility management in a mobile hotspot area, the S1 (single interface between the LTE radio access network and the EPC) interface of the virtual EPC has been modified in terms of the UP and CP. 3.5 Management and Orchestration Architecture Figure 6 shows overall MANO architecture on the EU side based on NFV MANO and an SDN. The architecture has two management entities: Figure 6Open in figure viewerPowerPoint Overall EU MANO architecture with an SDN. The VNF manager is in charge of instantiating and controlling EPC functions. It is responsible for interacting with VNFs, chaining VNFs, and handling their lifecycle—instantiation, maintenance, and others. It is in charge of the operation and configuration of VNFs through the operations support system (OSS)/base station subsystem (BSS). It will handle multifunctional EPC components such as the MME and home subscriber server (HSS) as well as specific-functionality VNFs such as firewalls and deep packet inspectors. The infrastructure manager interacts with (or incorporates the capability of) the SDN controller in the service stratum when deploying VNFs for configuring the computing and storage resources for the VNF of interest. It also supports the attachment of the VNFs to the border of the underlying transport network for the networking part to make them reachable from outside the data center. This is only for the service-layer part. It also has to determine a path for the transport-layer VNFs. The KR CN MANO are also based on NFV and SDN components. Figure 7 shows the MANO architecture. It comprises three different entities: the NFV orchestrator (NFVO), VNF manager (VNFM), and virtual infrastructure manager (VIM). Figure 7Open in figure viewerPowerPoint Overall KR M&O architecture with MANO and an SDN. The NFVO is responsible for managing functions such as network service (NS) lifecycle management and overall resource management. Service management or orchestration deals with the creation and end-to-end management of services by composing different VNFs. Resource management helps to ensure that the NFV infrastructure (NFVI) resources are abstracted cleanly (independent of the VIM) to support the services that access these resources. The VNFM oversees the lifecycle (which typically involves provisioning, scaling, and terminating) management of instances of a VNF. In this case, each VNF is associated with a VNFM that will manage that particular VNF's lifecycle. A VNFM may manage multiple instances of the same type of VNF or different types of VNFs. The VIM controls and manages the NFVI computing, storage, and network resources. The VIM component has been the focus of a large amount of research and various open-source solutions such as OpenStack and has been used to realize the virtualized infrastructure management functionality of MANO. 3.6 Autoscaling Based on Performance/Fault Management In our M&O, autoscaling functionality is provided as shown in Fig. 8. After instantiation of a 5G mobile CN service, the NFVO sends a supervision request to the supervisor, which performs performance monitoring and fault notification over virtualized resources and functions. Scaling is conducted autonomously by the orchestrator on the basis of the information provided by the supervisor. Figure 8Open in figure viewerPowerPoint MANO autoscaling process. 3.7 Automation by Event Chaining An event chaining process is another important functionality that is supported, which is defined as a sequence of event units occurring from inside or outside the target VNF and virtual data unit (VDU). It enables the automation of 5G mobile CN management. A combination of internal events that are significant in a single VNF or VDU and external events between VNFs and VDUs enables the automated management of a lifecycle of a mobile CN service (see Fig. 9). Figure 9Open in figure viewerPowerPoint MANO automation process. 3.8 Security Management Architecture The security of the CN can be grouped into two parts: the security of the CN elements and the security of the communication channels in the CN. In SDNs, controlling the behavior and interworking of different heterogeneous networks is carried out with a logically centralized control architecture that has a global view of all forwarding elements. An operating system maps the entire network to services and applications that are implemented on top of the control plane. Hence, security services will be implemented as security applications using the network stats provided either proactively or reactively by the network control platform. Centralized control, which can be either logically or physically centralized, enables the programmability of the network and will thus provide fine-grained network security control, remote monitoring, and dynamic security service insertion. The security management architecture is presented in Fig. 10. Figure 10Open in figure viewerPowerPoint SDN architecture showing the security services and their deployment. 4 Implementation and Deployment Experience Both the EU and KR edge and CN functions are under development. The development of some components has been completed, such as EU's edge and core functions in a 5GTN solution. KR's vEPC development is underway with the core functionality completed. The KR vEPC currently supports up to 100 UEs and a channel throughput of 20 Gbps toward an eNB. To meet the 5G key performance indicator (KPIs), we are trying to fill the gaps in both vEPC systems. We are targeting the completion of our system development by October 2017. We are also developing our agile CN MANO systems based on the architectures described in Section II. Initial prototypes are available, and their functionality as separate systems and their interoperability are being tested as well across the EU and KR over interconnected research and development networks between the EU and KR via the Korea Research and Education Network (KOREN)–Trans Eurasia Information Network (TEIN)–Nordic Countries Network (NORDUNET)–Finnish University and Research Network (FUNET). 4.1 vEPC Implementation 5GTN vEPC VNF functions have been implemented, deployed, and tested on CloudBand's NFVI and its MANO solution. CloudBand is a hardened, production-ready NFV solution based on OpenStack and other open-source technologies. This open approach allows service providers to benefit from a vast community of engineers and supports investments in a mainstream solution with open interfaces. The HSvEPC implementation consists of a vMME, vSGW-CU, vPGW-CU, vSGW-DU, and vPGW-DU. The CU is a CP that controls the device management, and the data unit (DU) is a UP that controls the data transfer between devices. The main reason why we separated functions by each plane is to support scalability depending on the demand situation. Since the functions in the HSvEPC are implemented as VNFs, they can be modified on demand and controlled per VNF level. One important use case of such flexibility is network slicing support. Figure 11 shows the access point name (APN)-based CN slicing use case. An IoT device may have a different APN against a UE, and discrimination of each device at the MME is required. The above use case illustrates our implementation of an MME that can classify different devices by categorization based on their APNs and map appropriate resources in the SGW and PGW. Moreover, the HS-vEPC can be scaled in or out depending on the demand, which can reduce the cost, and other unused parts of network functions can be relocated to only the necessary parts. Figure 11Open in figure viewerPowerPoint HSvEPC core slicing use case. 4.2 Management and Orchestration Implementation MANO in 5GTN has been implemented and deployed. It consists of CloudBand infrastructure software, a CloudBand application manager, and a CloudBand network director that have been optimized to fit the key NFV MANO shown in Fig. 12. Figure 12Open in figure viewerPowerPoint 5GTN MANO implementation. CloudBand Infrastructure Software The CloudBand infrastructure software is a multipurpose NFVI and VIM. It virtualizes and manages computing, storage, and network resources. CloudBand Application Manager The CloudBand application manager is a VNFM that automates lifecycle management actions by managing resources and applying associated workflows. CloudBand Network Director The CloudBand network director is an NFV resource and NS orchestrator. It manages virtual resources across geodistributed NFV infrastructure nodes. It visualizes and automates the lifecycle of NSs, such as virtual customer premise equipment (CPE), including their forwarding graphs and service chains. The KR MANO implementation is shown in Fig. 13. We have implemented it in a rack of servers consisting of a VIM built and extended over OpenStack, a VNF manger, and an orchestrator. The management target is, of course, a set of virtual functions implementing CN functionality and networks that interconnect those virtual core functions.