Hardware-Based Network Slicing for Supporting Smart Grids Self-Healing over 5G Networks

As a fundamental technology in the Fifth-Generation (5G) mobile networks, network slicing creates multiple logical networks for various vertical businesses over the same physical 5G infrastructure to achieve cost-effective service provisioning. Meanwhile, mission-critical vertical businesses would require a very high level of guaranteed Quality of Service (QoS) that is beyond software-based network slicing approaches. In this paper, we address a highly demanding Smart Grid Self-Healing Automatic Reconfiguration use case to ensure ultra-Reliable and Low-Latency communications (uRLLC) through improved network slicing based on programmable hardware acceleration. Empirical results have demonstrated the superior performance of the proposed approach in meeting the strict and challenging QoS requirements of this use case.


I. INTRODUCTION
Power substations are the closer element to final users of an electrical generation, transmission and distribution system.This part of the distribution network is where high-voltage power lines are transformed to medium-voltage lines.These are critical infrastructures which provide the electricity that we all consume at home everyday.The transport of highvoltage power lines from power generation plants to power substations is inspected by power distribution networks to detect any failure along the transportation network, which would likely produce a power interruption.To avoid any power interruption, novel self-healing algorithms are running into the power distribution network.These algorithms are able to deal with self-healing capabilities to avoid a power disruption in the distribution network.The main causes are short-circuits, climatic phenomena and animals.Self-healing algorithms allow the detection and automatic reconfiguration of the power network.Thus, when a fault appears in a given segment of the power network, the self-healing algorithm should be able to reconfigure the power network and redirect the power to other paths, avoiding a power service disruption in final users.
The speed of the electricity is close to speed of light and that current power lines are working at 50-60 Hz on alternate electricity, a complete alternate cycle is happening around 50-60 times in a second, i.e. 20-16 ms.In order to detect problems in the network, it is needed to perform 2 different measurements along a given cycle [1], around 10-8 ms, which means a very strict uRLLC.To be enforced the self-healing capabilities, these values should be transmitted to the next hop of the power grid before the next alternate cycle reading starts, leading to an effective maximum 20-16 ms round-trip delay.In this time, a hop should be able to send a packet and receive a response from the closer Smart Substation Controller (SSC) to sort out the given problem.Apart from the SSC, the self-healing system is composed by the Supervisory Control And Data Acquisition (SCADA) system, which receives realtime information by the different Intelligent Electronic Devices (IEDs) of the power system to perform monitoring and control operations within ranges of 50 ms.
The most known and cited definition of a 5G slice, is presented by the NGMN, which describes a 5G slice as a collection of 5G network functions and specific Radio Access Technology (RAT) settings that are combined together for the specific use case or business model, supporting the communication service of particular connection type with a specific way of handling the control and user plane for this service.Taking into consideration all the definitions proposed by the different organizations and industry alliances, this paper follows the approach presented in [2], that describes a 5G slice as the isolation of performance of a set of network flows, allowing the deployment of multiple logical networks on the top of a common shared physical device.
This research work proposes a hardware-based network slicing framework to accelerate the 5G edge-to-core network segment and demonstrates the viability of using 5G networks infrastructures to truly isolate the critical control and monitoring traffic used for self-healing smart grids from the 5G background user traffic.The deployment of this framework has been carried out by using programmable hardware with the NetFPGA platform and an user Application programming interface (API) which allows the definition and configuration of the hardware offloaded network slicing in real-time.This framework aims to meet the challenging communication requirements currently imposed by smart grid applications, which are currently not fully achievable, so this work will provide the necessary isolation, very low latency, ultra-reliable wide-area communication infrastructure to achieve it.

II. RELATED WORK
5G communications support uRLLC as a key enabler for tactile internet, remote operator, automated controls and intelligent transportation systems.URLLC are highly demanded from many vertical sectors, being electricity distribution one of the major use cases.[3] presents an overview with uRLLC services under network slicing.
A priority-based uRLLC uplink resource scheduling for smart grids is proposed in [4].This paper presents an scheduling algorithm to guarantee services priority and ensure low latency.Unfortunately, there is a lack of empirical validation, only a simulation shows the expected behaviour of this solution.Also, the scheduling algorithm presented in this paper has not being tested in any real device, however the scheduling priority-based algorithm used in our solution has being implemented and fully tested in a Field-Programmable Gate Array (FPGA) network card.
In [5], a 5G wireless solution for communication protection in smart grid scenarios, ensuring reliability, security and low latency is presented.An extensive test environment is also provided to validate its system and communication performance both in wireless communication and in the core of the network.Nonetheless the results obtained do not fulfil the 5G smart grid KPIs in terms of delay.They present and average end-toend (E2E) delay of 33.65ms for wireless communications and 0.7ms for a 5G core communication.Our QoS-aware solution allows the isolation of the smart-grid traffic, achieving better values in terms of delay and fulfilling the 5G smart grid KPIs.Kurtz et al. [6] achieved the 5G smart grid KPIs and allows the uRLLC services demanded from the smart grids.However, their solution does not present support for Routable Generic Object Oriented Substation Event (R-GOOSE) protocol, which is the protocol used to support self-healing capabilities in the power grid.Kurtz et al. approach is based on Open vSwitch [7], and their results are still far away from the ones presented in our work.
Aleixo et al. [8] present a use case and a testbed of a smart grid protection and automation scenario over a 5G network.The architecture proposed in this work is very similar to the one presented in our work, however they do not present yet any empirical validation and not any QoS-aware implementation for these smart grid self-healing scenarios.[8] is an ongoing work of the H2020 5G-PPP SliceNet project [9].Our work is also funded and has being tested in the SliceNet project infrastructure and offer now empirical results.
Despite the above advances in the state of the art, there is no existing hardware-or software-based solution achieving the demanding 5G KPIs required in smart grid self-healing scenarios in the edge-to-core network segment.This gap in the state of the art has been the main motivation of this research work, which provides a prototype of a fully functional FPGAbased solution, allowing R-GOOSE traffic isolation over the edge-to-core network segment and also fulfilling the ambitious 5G KPIs.

III. SMART GRID SELF-HEALING
Self-healing solutions rely on power system protection and distributed automation, with the goal of rising energy supply QoS, reducing the number and duration of power outages and therefore, cutting down the number of final users affected by these.To achieve this aim and to de-energised the smallest possible portion of the power system network, a self-healing scenario that detect and isolate this failing is required.
Figure 1 shows a smart grid self-healing automatic reconfiguration use case, where after a problem produced in a segment of the power system network is detected, the SSC is able to react in an extremely short period of time, re-configuring automatically the distributed IEDs of the network to ensure that the minimum number of final users are affected by the power cut.Generally, healthy network sections are also deenergised to clear the fault, in most of the cases this allows to reduce the power outages duration for a larger number of final users.In this context, an efficient QoS-aware 5G network is deployed to guarantee the isolation of the R-GOOSE traffic between the IEDs with the SSCs, and the IEDs with the SCADAs control centre.R-GOOSE is the protocol used to exchange information between IEDs.In Figure 1, a use case with 5 IEDs and 2 SSCs is presented.This is divided into 5 different steps which allow representing how a smart-grid self-healing scenario is able to automatically reconfigure itself after a short-circuit in a segment of the power system network.Also, the scenario is divided into differentiated 6 segments, a) to f), delimited by the IEDs and SSCs.This use case is described as follows: 1) The power system network is working normally.All communications between IEDs are working properly and all segment of the system are transmitting electricity to the costumers, 2) A fault occurs on segment a) of the network.
3) SSC1 detects the fault and it starts its protection functions by opening the circuit at SSC1 and sending a message via IEC 61850 R-GOOSE [10] to IED1 to open the circuit and clearing the fault.In this stage, only users of the segments d), e) and f) have the electricity supply working properly.4) Once IED1 is open, the power fault has been isolated and power can be restored in the other network segments.
To do so, it sends a close message via IEC 61850 R-GOOSE to IED3 to allow that powers supply can arrive from the southbound of the scenario to the segments c) and b).At this stage, IED2 is still closed.5) Segments b), c), d), e) and f) are working normally.segment a) is de-energised due to the problem that happened in step 2, the electricity interruption in segment b) and c) have been mitigated due to the self-healing protocol and they are now working normally.
The success of the smart grid self-healing is mainly based on the R-GOOSE events transmitted.If these R-GOOSE messages are received within the predefined time, which is a maximum round trip time (RTT) of less than 60ms and an optimal RTT of less than 16ms [11], then the power system network would be reconfigured successfully and the problem would be controlled.Ensuring that R-GOOSE events are transmitted in less than 20 ms is critical to minimize the impact on power system stability and on distributed energy resources and for guaranteeing QoS for end-users, which is particularly important for critical loads, such as hospitals or industry, where stopping the production will have a considerable impact.

IV. 5G NETWORK ARCHITECTURE
Figure 2 depicts an overview of a 5G smart grid selfhealing network scenario, deployed in our premises, using a Mobile Edge Computing (MEC) architecture with multizone support and integrated with a smart grid self-healing infrastructure.The 5G infrastructure is fully operations whereas the smart grid infrastructure has been emulated using VNFs with R-GOOSE protocol emulation tools.This figure also shows different architectural components, network segments and smart grid components associated with different geographical locations.The smart grid area defines the deployment and distribution of the power system topology conformed by substation IEDs and remote IEDs installed in electricity poles.
These are connected to the 5G wide-area network through Antenna Units/Remote Radio Units (AAU/RRU), which receive messages via IEC 61850 R-GOOSE protocol and transmit them to the SCADA and SSC through the 5G wide-area network, divided into different 5G network segments.A 5G Cloud-RAN (C-RAN) allows the communications between the IEDs and Distributed Units (DUs) with a wireless system based on 5G Radio Interface [13].These DUs are connected using optical fibres to a pool of Centralised Units (CUs), which process the R-GOOSE traffic into the network in the edge-tocore segment.Each of this CUs, deployed as Virtual Network Functions (VNFs) in virtual machines represent a single tenant in each Edge of the 5G architecture, which is deployed in geographically separated physical machines and functional desegregation along with the 5G network topology.Notice that the packets processed by this 5G architecture should present a double level of encapsulation headers to create different tunnelled communications which allow the communication between the IEDs and the virtual machines deployed inside the edge and core of the network.A VXLAN encapsulation header (or similar) is needed to establish a tunnelled communication between the IED and the CU inside the edge, via the VXLAN Network Identifier (VNI) field inside a VXLAN header and also, another tunnelled communication with the tenant inside the core via the Tunnel End-point Identifier (TEID) of a GTP header.
Each of this edges is connected to the core of the 5G network using optical fibers.The 5G slicing framework presented in this research work is allocated in the edge-to-core network segment and is able to isolate the R-GOOSE traffic exchange between IEDs, SCADA and SSCs, allowing to achieve the demanding KPIs [11], in terms of delay, required by the smart grid self-healing scenario proposed.The NetFPGA cards, where a novel queuing discipline has been implemented to achieve network slicing, transmits the R-GOOSE traffic from the different edges to the 5G network core.In the control plane, and inside the Core of the 5G architecture, the Access and Mobility Management Function/Session Management Function (AMF/SMF) controls the registration, mobility management and connection of users; and the Unified Data Management/Authentication Server Function (UDM/AUSF) consists of databases with all user information and subscriptions.In the data plane of the core the User Plane Function (UPF) preservers sessions during the handover process and allows the interconnection of the infrastructure.
The core of the 5G architecture relays the R-GOOSE traffic to the SCADA and SSCs where a decision over the smart grid infrastructure is taken.SCADA systems control and monitoring the remote IEDs and the SSC performs automatic and manual operations, e.g., remote control switchgear, record retrieval, configuration and firmware deployment, etc. Operations at SSCs must exist side-by-side with the demanding KPIs achieving the critical peer-to-peer communications time.Smart grid self-healing scenarios KPIs must coexist with the network KPIs, so a network that ensures Ultra-Reliable Low-Latency Communications (uRLLC) latency is required.5G networks Fig. 2. Smart grid self-healing 5G edge-to-core architecture require very low values in terms of latency and packet loss depending on the use case, thus table I shows some of the most demanded use cases values.In the smart-grid self-healing use case presented on this research work, optimal communication should be achieved in an RTT delay lower than 16ms, which is similar to the E2E 8ms value shown in table I.Although in this solution it is only presented the improvement of the latency achieved in the edge-to-core network segment, in communications between the IEDs, SCADA and SSCs, there are other network segments involved.The most and critical segment in this communications is the wireless, which interconnects the remote IEDs with the AAU/RRU.As demonstrated by Chang et al [15], wireless communication has an average RTT latency of 10ms in C-RAN segment of the 5G network.Thus, the solution proposed in this paper should achieve that R-GOOSE messages are transmitted in an RTT of less than 6ms to achieve the 5G smart grid self-healing KPIs.

V. 5G HARDWARE-BASED NETWORK SLICING PROTOTYPED
The implementation carried out to test, validate and evaluate the proposed 5G hardware-based network slicing framework is base on the NetFPGA SUME platform [16], an opensource hardware development platform for NetFPGA cards, which allows the programming and extension of the internal cores to achieve networking capabilities.Figure 3 shows the architecture of the NetFPGA-based network slicing solution implemented in this research work.This is divided into three different main stages: parser, match/action and deparser.After the deparser, and before network traffic arrives to the kernel of the operative system, the UWS-FPGA core with the slicing discipline implementation is allocated.
Pasers stage: Every packet is parsed to discover the type of headers are being processing and extract the data from them.The most important data extracted in this stage is the destination port of the UPD header inside the GTP encapsulation header and the APPID inside the R-GOOSE header.These data is used in the Match/Action stage.
Match/Action stage: At this stage, a Ternary Content-Addressable Memory (TCAM) table has been implemented.The data extracted at the previous stage is used as keys of the TCAM entries to match the 5G R-GOOSE flows and to apply an action over them.If a packet matches with an action, then it will be sent through the slice defined in the TCAM rule matched.For this prototype, 4 rules have been implemented for each of the 4 different types of traffic that is parsed.Control traffic of the 5G infrastructure is the most critical one and it is identified by well know ports defined by the IANA [17].This type of traffic is sent through the slice 4 (with the most priority).The slice 3 and 2 are used by the R-GOOSE traffic, which is sent to the port 102, standardised by the IANA.If the APPID contains the identifier of the SSC, then the R-GOOSE traffic is sent through the slice 3, however, if the APPID belongs to the SCADA Management infrastructure it is sent through the slice 2. Traffic which has not match any rule is sent via slice 1 (with the least priority).

Deparser:
The packet is rebuilt a resend to the core of the system through the UWS-FPGA Slicing Core implemented.This FPGA core is conformed by 4 different priority queues allowing R-GOOSE smart-grid self-healing traffic isolation.
In case the reader is more interested in understanding the delays of the UWS-FPGA Slicing Core implemented, our previous work Ricart-Sanchez et al [2] provides an in-depth explanation.This contribution is focused on the application to such previous results into the smart grid use case, including the support of the R-GOOSE protocol.3) The R-GOOSE traffic is sent to a CU in the edge and forwarded back to the same NIC n; 4) Traffic is sent from the edge to the core through the physical interface of the NIC; 5) R-GOOSE traffic is sent to the kernel of the Linux system through the UWS-FPGA slicing core implemented.This FPGA core allows the isolation and acceleration of the most critical traffic; 6) The traffic received in the kernel of the Linux system is stored in a PCAP file.7) The traffic sent in step 1 and the traffic received in step 6 are analysed to carry out the empirical validation.

B. Evaluation Test
Executions have been carried out 5 times and the values are the average of those 5 executions.Figures 5 and 6 show empirical results where 4 different slices are deployed and fully working over the same physical infrastructure.These slices isolate and accelerate the R-GOOSE flows in the segment delimited by the edges of the 5G network and the core network.These slices match the previous descriptions.
Table II shows the traffic generated to carry out this experimentation, a total of 183,44 Gb sent over a 10Gb network link and divided into the 4 different types of traffic previously defined and where traffic amounts are equally distributed across slices, i.e. 45,86 Gpbs per slice.Figure 5 shows the analysis of delay and jitter achieved when network slicing is applied to a transmission with 256 IEDs involved.The maximum RTT delay achieved is almost 1.7ms for the background traffic sent through the slice 1 and the lowest RTT delay of 0.1ms is achieved in the slice 4 with the 5G infrastructure control traffic.However, the uRLLC traffic used for this smart grid self-healing use case is the transmitted by the slice number 3, achieving an RTT delay of 0.2ms.In terms of jitter, again, the maximum value is achieve by the slice 1 and the minimum by slice 4, with RTT values of 0.28ms and 0.003ms respectively.The RTT jitter value achieves for uRLLC smart grid control traffic is 0.02ms.Figure 6 presents the number of packet loss per slice during the transmission of the traffic previously commented.As it is shown, almost 15% of the background traffic is lost and on the contrary, the percentage of packet loss in slice 4 is almost 0%.Likewise, percentage of uRLLC smart grid control traffic is around 0.5%.Notice that these data have been obtained in a congested scenario where all the traffic is trying to compete for the available bandwidth on the network card.
The solution proposed achieves the 5G smart grid selfhealing KPIs in terms of delay, jitter and packet losses.It establishes a maximum RTT delay of 16ms, including all the network segments.I.e., this slicing implementation allows the achievement of a minimum RTT delay of 0.2ms for smart grid control traffic in the edge-to-core network segment, plus the 10ms needed for the C-RAN segment.It means that this implementation is able to achieve a total RTT delay of 10.02ms in a communication between the IEDs and the smart grid control centre by using a 5G network architecture.In terms of reliability, it is achieved a 0.3% packet losses when the system is under the highest levels of stress which clearly shows ultrareliable communications.

VII. CONCLUSION
Self-healing solutions rely on power system protection and distributed automation, with the goal of rising energy supply QoS, reducing the number and duration of power cuts and therefore, to cut down the number of final users affected these power distribution problems.To overcome this, a novel hardware-based network slicing solution for supporting smart grids self-healing over 5G networks is proposed in this paper.
The implementation and design of this work have been carried out by leveraging the platform of the project P4-NetFPGA project.Nowadays, there is no hardware solution that allows the acceleration and isolation of R-GOOSE traffic in the 5G edge-to-core network segment.Furthermore, the prototype yields good behaviour in stressed scenarios, incurring a minimum RTT delay.It easily allows the achievement of the demanding 5G KPIs for smart grid self-healing solutions.

Figure 4
Figure 4 depicts the experimental testbed developed to carry out the empirical validation of the proposed slicing solution.1) A PCAP file with R-GOOSE traffic from 256 different IEDs is sent from the core to the edge emulating the smart grid infrastructure; 2) Packets are transmitted from the NetFPGA in the Core to a normal NIC in the edge; 3) The R-GOOSE traffic is sent to a CU in the edge and forwarded back to the same NIC n; 4) Traffic is sent from the edge to the core through the physical interface of the NIC; 5) R-GOOSE traffic

Fig. 5 .
Fig. 5. Analysis of Delay and Jitter in a transmission with 256 IEDs.

Fig. 6 .
Fig. 6.Analysis of Packet Loss in a transmission with 256 IEDs.

TABLE I 5G
[14] REQUIRED BY ERICSSON FOR DIFFERENT USE CASES[14]

TABLE II SUMMARY
OF THE TRAFFIC USED TO TEST THE EXPERIMENTAL SETUP PROPOSED.