Resiliency Metrics for Monitoring and Analysis of Cyber-Power Distribution System With IoTs

The electric grid operation is constantly threatened with natural disasters and cyber intrusions. The introduction of Internet of Things (IoT)-based distributed energy resources (DERs) in the distribution system provides opportunities for flexible services to enable efficient, reliable, and resilient operation. At the same time, IoT-based DERs comes with cyber vulnerabilities and requires cyber-power resiliency analysis of the IoT-integrated distribution system. This work focuses on developing metrics for monitoring resiliency of the cyber-power distribution system, while maintaining consumers’ privacy. Here, resiliency refers to the system’s ability to keep providing energy to the critical load even with adverse events. In the developed cyber-power distribution system resiliency (DSR) metric, the IoT trustability score (ITS) considers the effects of IoTs using a neural network with federated learning. ITS and other factors impacting resiliency are integrated into a single metric using fuzzy multiple-criteria decision making (F-MCDM) to compute primary-level node resiliency (PNR). Finally, DSR is computed by aggregating PNR of all primary nodes and attributes of distribution level network topology and vulnerabilities utilizing game-theoretic data envelopment analysis (DEA)-based optimization. The developed metrics will be valuable for: 1) monitoring the DSR considering a holistic cyber-power model; 2) enabling data privacy by not utilizing the raw user data; and 3) enabling better decision making to select the best possible mitigation strategies toward resilient distribution system. The developed ITS, PNR, and DSR metrics have been validated using multiple case studies for the IoTs-integrated IEEE 123 node distribution system with satisfactory results.

With increasing Internet of Things (IoT)-based intelligent devices, the distribution system is becoming more adaptable and flexible. IoTs are now evolving to the Internet of Everything, as it incorporates and builds a system that includes wireless networks, sensors, cloud servers, analytics, smart devices, and advanced technologies. IoTs have been forming a regime that consists of millions of intelligent devices connected to analyze and influence our day-to-day activities [5]- [7]. IoTs record one of the fastest growth rates in computing technologies, with an estimation of 5.3 billion global Internet users and more than three times the global population of devices connected by the year 2023 [8]. As the grid becomes more connected via IoTs, computations and data managements are increasingly moving to the devices at the network edge. Encouraged by availability, latency, and privacy issues, IoT devices can be leveraged to perform local computations on these data to provide services to the users without transferring any personal data to a central server, thereby improving privacy. IoTs can provide connectivity between distributed energy resources (DERs) along critical energy supply corridors and within groups of vital facilities, accommodating privacy concerns and constraints of availability and increasing system resiliency. IoTs connected DERs will be instrumental to provide energy in islanded systems to critical loads and also within connected system given limited resources after an extreme events.
According to the Annual Threat Assessment Report of the U.S. Intelligence Community Report, critical infrastructures are being constantly targeted by the adversaries [9]. Being a critical infrastructure, the power grid has been embracing cyberattacks of gradual increasing complexity and intricacy, thereby adding to the various threats faced by the power grid. It is essential that the power grid remains resilient to such threats and supplies power to the critical loads when subjected to various stress levels. Considering that these risks cannot be eliminated, resiliency becomes vital to enable the essential infrastructure to continue to perform when faced with such threats. In 2017, the National Academy of Sciences, Engineering, and Medicine (NASEM) released a report titled "Enhancing the Resiliency of the Nation's Electricity System" [10], which, among other recommendations [11], details the need for defining resilience metrics that can drive planning and operational decisions. Any effective resiliency metrics with such capabilities need to encapsulate all the attributes of cyber-power systems. There are only a few works related to resiliency metrics for microgrids without IoTs [12], [13], which are limited in scopes and not applicable for advanced cyber-power distribution systems with IoTs. Therefore, the main focus of this work is to develop holistic cyber-power resiliency metrics by leveraging the ubiquitous presence of IoTs and other cyber-physical attributes of distribution system to facilitate an analysis and planning solutions to monitor resilience for distribution system and in supporting decision making to minimize impact subject to both cyber and physical events.
The increasing use of IoT devices in any system brings many other concerns, such as data integrity, data privacy, data quality, and network communication latencies. When it comes to cyber-power resilience analysis for distribution system planning and operational decisions in the presence of IoT devices, the concerns mentioned above are very critical to address. Various literature approaches use machine learning to tackle these concerns [14], [15] in different systems. But, in general, infrequent communications of most IoT devices in distribution systems make it harder to get enough data points to generate an accurate model of them. The federated learning (FL) architecture can overcome the challenges with data scarcity of these IoTs and generate effective models to enable intelligent applications for systems with IoTs [16]- [22]. Therefore, in this work, we have started with modeling the distribution system with IoTs, which includes detailed power system modeling of IoTs and realistic emulation of the IoTs cyber network, to better understand the overall changing behaviors of the distribution system for normal operation and evolving extreme cyber events. Then, we identified the cyber-power features of IoTs presented in the distribution grid and applied the appropriate unsupervised machine learning and FL architecture to identify anomalies and formulate IoT trustability score (ITS). This approach helps us tackle data scarcity and allows us to protect user data privacy by not bringing the raw user data out of the privacy-protected area. The ITS and all other cyber-physical factors from the secondary level feeder are combined using fuzzy multiple-criteria decision-making (F-MCDM) and multiplicative aggregation to calculate primary-level node resiliency (PNR). Finally, the overall distribution system resiliency (DSR) is formulated utilizing game-theoretic-data envelopment analysis (DEA)based optimized aggregation of PNR of all the primary nodes and factors of distribution level network topology and its vulnerabilities.
The key contributions of this work are summarized as follows.
1) Developed ITS for enhanced monitoring of IoTs using the unsupervised neural network model and the FL architecture. 2) Developed PNR metric with fuzzy multicriteria decision making considering IoTs. 3) Developed formulation for cyber-power DSR with PNR and attributes from the physical and cyber network using game-theoretic DEA-based optimized aggregation. 4) Validated the developed resiliency metrics through case studies for the IEEE 123 node distribution system with the IoTs. Modeled, simulated, and analyzed cyber-power distribution system with the IoTs for validation. The overall works translate to an effective and comprehensive resiliency metrics capable of addressing all the power system physical and cyber aspects, and providing detailed situational insights for planning and analysis activities of cyber-power distribution systems with IoTs to minimize the impact of extreme events with IoT-based DERs. Fig. 1 shows an overview of the proposed resiliency metrics formulation.

II. CYBER-POWER DISTRIBUTION SYSTEM WITH IOTS
With the modernization of the power system, distribution systems are also going through significant changes. More and more smart devices and appliances that are based on IoTs are replacing traditional distribution system loads and resources [23], [24]. DERs in distribution systems are also utilizing the IoT platform [25]. All these phenomenons are causing significant changes in the distribution system, and further study of distribution systems with IoTs is required to understand these changes for best possible performance under normal and extreme cyber events operation.

A. Distribution Power System IoTs
According to the IEEE IoT Initiative, IoTs should have features, such as identity, sensing/actuation capability, embedded intelligence, programmability, and communication capability using Internet infrastructure [26]. The initiative also points toward the necessities of application system-specific features when it comes to analyzing that system with IoTs.
The IoT applications in the power system primarily utilize the unique address of IoTs for the transmission control protocol (TCP) and Internet protocol (IP)-based communications, comprehensive sensing, and intelligent processing features [27]- [29]. Since resiliency focuses on supplying power to the critical loads, characteristics of IoTs associated with power generation and load components are of more interest here.
Therefore, any device with the following attributes are considered as IoTs and included in the modeling and analysis of the distribution system with IoTs for this study.
1) Connected to others and can exchange information.
2) Has unique identifier such as IP address.
3) Act as a power source or load. 4) Has computing capability. 5) Has some autonomous activity. 6) Plug and play. Following the above discussion, we have considered heating, ventilation, air conditioning (HVAC), solar PV, battery storage, and electric vehicle (EV) as IoTs of interest for this study as they are a significant part of the changing distribution power system.

B. Distribution System Architecture With IoTs
The changing distribution system architecture can be modeled from the primary level node to the secondary level downstream users since most IoTs are deployed from the primary level to downward. Let us consider the IEEE 123 test feeder system as shown in Fig. 1 where three representative primary nodes are expanded to their secondary level. The primary level nodes of this system can be modeled into three categories based on the configuration of each primary level node downstream.
1) Physical Power Primary Node: Though the modernization of the distribution system is going everywhere, there are still some feeders with the legacy connection type. This type of feeder has no digital component in the secondary level downstream, and every operation done here is manual in type.
2) Cyber-Power Primary Node Without IoTs: This category covers any primary node with digital devices but no IoT devices in the secondary level downstream. Digital relays, circuit breakers, switches, etc., are there in the secondary level feeder.
The expanded primary node (a) in Fig. 1 without the PV, battery, and other IoTs resembles the typical configuration of these two type nodes from the primary level to its downstream.
3) Cyber-Power Primary Node With IoTs: IoTs, such as HVAC, PV, energy storage, EV, etc., exist downstream of this kind of primary node. This type of node can have three types of feeder configuration in terms of connectivity. They are as follows.
1) Type-A: Feeders connecting individual buildings/houses with IoTs where the utility does not have access within the buildings/houses as shown in expanded primary node (a) in Fig. 1. 2) Type-B: Feeders connecting large buildings with IoTs, such as roof-top PV, building energy storage, and EV charging parks, where the utility has access as shown in expanded primary node (b) in Fig. 1. 3) Type-C: Big PV farms, energy storage type IoTs are directly connected at the primary voltage level, and there is no secondary level as shown in the expanded primary node (c) in Fig. 1.

III. MODELING AND EMULATION OF IOTS IN DISTRIBUTION SYSTEM
The introduction of IoTs in the distribution system increases availability of more data, which can be utilized for resiliency monitoring. Cyber-power modeling and emulation of IoTs in distribution system provides detailed insight into these data.

A. Power System Modeling in Gridlab-D
In order to model a cyber-power simulation of distribution system with IoTs, we have built a typical secondary level feeder from a primary level node "X" as shown in Fig. 2 in Gridlab-D [30]. A total of six houses and one commercial building are considered. Here, all of them are equipped with normal loads and HAVC. Five houses have solar PV, two houses have battery storage, one house has an EV, and the commercial building has battery storage and solar PV.
Gridlab-D simulation can include real-world climate data in the simulation. Here, all the individual houses and buildings have their own schedule for different common loads that vary from time to time. The house class in Gridlab-D is utilized for both.
EV does not have any model in Gridlab-D yet, so we have considered it a constant load that only turns on at night for charging. This overall Gridlab-D simulation provides all the necessary data, which can be easily used to determine the behavior of the power system part of IoTs.

B. IoT Network Emulation in MININET-WIFI
For cyber network emulation of power systems, the Mininet SDN network emulator is commonly used for cyber-power co-simulation [31], [32]. In this work, we are using MININET-WIFI [33], which is a fork of the Mininet. For four IoTs considered in this work, we have created four virtualized wifi stations and connected them to an access point. All of these are based on the standard Linux wireless drivers and the 80211_hwsim wireless simulation driver. For emulating the IoTs operation, we have developed applications that mimic general operations of sensing, calculating, and exchanging information for each IoT device. Now, we run the EMS/IoT Hub application in the access point. We run device-specific applications in the wifi station for each type of device for each IoT. This application reads the data generated by the specific device in the Gridlab-D simulation, represented by the IoT wifi station. We utilize client-server-based communication settings to exchange customized network packets encapsulating the device data generated by Gridlab-D and any instruction from the IoT Hub that communicates to the user via the Internet. Fig. 3 shows the overall network. We have captured IoTs network traffic through this emulation and utilized different traffic features for ITS formulation.

IV. IOT TRUSTABILITY SCORE FORMULATION USING FEDERATED LEARNING
Monitoring and operating a resilient power system require data from all over the system, including the users. With the increasing amount of data utilization for the operation and control purpose of the distribution system, the risk of exposing users' valuable data is also increasing. While more data help utilities operate better, this leads to privacy concerns. So, in scenarios where data privacy is required, data use needs to be done in a secure way that will provide the highest possible data utilization for monitoring and control purposes while maintaining privacy. Uses of IoT devices in secondary feeders of the distribution system as shown in Fig. 1 fall under this type of scenario. Again, typically these IoT devices only initiate communication to update about sensor readings and during interactions related to user commands, which are also few and far between. As a result, they do not generate enough data to train an accurate model covering all IoT devices' behaviors. A federated self-learning architecture can overcome this limitation of data points for training accurate models while keeping the IoT device data inside any privacy-protected area, such as buildings, houses, etc.
We have formulated an ITS utilizing the federated selflearning architecture in this work. Here, IoTs cyber network data and the power system data associated with the IoTs from Section III are considered in this formulation. ITS provides an insider view of the operating status of IoTs without accessing raw user data. Anomalies in IoTs data are the main factor in formulating the ITS. First, IoTs network packets were studied for feature selections. The features for all types of devices and IoTs network packet features will be the same. Then, features from the specific power system simulation data have been selected to be used for the specific device. More details of the features are shown in Table I.

A. Overview of Federated Learning
The FL architecture generally consists of a curator or server that sits at its center and coordinates the training activities. Here, buildings/houses are considered as clients who have IoT devices. The clients communicate at first with the server to receive the current global model weights of each IoT device and the communication they have from the server. Then, they train it on each of their local device data to generate updated parameters for that device model and upload it back to the server for aggregation.
Let us assume there are M clients. Then, utilizing the concept of the federated averaging algorithm [34], if the clients estimate their weight parameters W t i for minimum reconstruction error (RE) for each type of IoT device, we can scale all clients weight parameters and sum to get the final global weight W G for each type of IoT devices as shown in where n i is the number of data points in client i for one data type, and n is the total data point, which is the sum of the number of data points of that type of data of all M clients. An overview of FL is shown in Fig. 4.

B. Autoencoder for Unsupervised Learning
An unsupervised autoencoder neural network is used for the local training part of the FL architecture. The autoencoder is very useful for anomaly detection in cyber-power systems, especially with unlabeled data [35], [36]. The autoencoder efficiently captures the correlations and interactions between the various variables by compressing the data to a lower dimensional representation for complex data. The autoencoder model minimizes the RE during training, which is the mean squared distance between input and output. In this work, an autoencoder neural network model is built utilizing the Keras module [37] in python and modified to be used in the FL architecture. The optimized model is constructed with five fully connected hidden layers with 6, 3, 2, 3, and 6 neurons. The neuron number of the input and out layer depends on the data feature number.

C. ITS Formulation
The overall formulation of the ITS is shown in Fig. 5. The FL explained above is applied here for each type of data. For each client, there will be one autoencoder model for each IoT device to train on its power system data and one more autoencoder model to train only on IoT network packet data. During the training session, each client's IoT devices are closely monitored to ensure that the client trains the IoT network packet autoencoder model with normal network data and their device-specific autoencoder models with device-specific power system normal data received via IoT network packet communication. Once all the clients go through the FL process for each IoT device and receive the global weight parameter W G for all the autoencoder models, the monitoring session starts.
First, for each type of data, each client calculates RE values for their own training data points at their local autoencoder block with the global weight parameter W G and sends it to the ITS algorithm block to decide on a tolerance value T err for the RE. The ITS algorithm block selects the maximum RE value as T err . Then, each client monitors all the network packet and power system data of each of its IoT devices and reconstructs them using the global weight parameter W G in the local autoencoders block.
The ITS algorithm block flags any data point as an anomalous data point (ADP) if the data point (DP) crosses T err . Then, for any reporting time period t, the nonanomaly ratio (NAR) is calculated using Total ADP number over t Total DP number over t .
Now, the ITS algorithm block calculates the cumulative nonanomaly ratio (CNAR) to capture behaviors of the IoTs for some most recent time periods. CNAR at time t is formulated as (3) where T is a fixed total time period before t and always divisible by t. Based on the distribution system, operators of a specific distribution system can decide on the t and T Then, ITS for time t is calculated using (4) where CNARs part ensures the stability of the score by gradually changing it for actual anomalies in the data points over certain reporting t periods rather than sudden changes at time t due to some short events, which are not harmful to the IoT devices Here, CNAR max is calculated using (3) with maximum NAR value, which is NAR = 1 for whole T time period. The operator of the distribution system will choose w t and w t− based on the system satisfying (5) so that ITS t depends more on current time NAR while retaining immediate past behaviors of IoTs. Finally, to get the overall ITS of any primary node with IoTs, we average ITS t of all the clients of that primary node to calculate ITS as where M is the total clients or buildings/houses of that primary node.

A. Factors Influencing Resiliency
Modeling and analysis of cyber-power systems help us determine the factors responsible for the resilient operation of the system. These factors vary along with the configuration. Factors that can be determined directly from the secondary level configuration of each primary node are described as follows.
1) Available Generation: The total amount of generation capacity in the secondary level is considered in this factor. Generation from PV, stored energy from storage, etc., are included here. The total committed amount of power supply by all the participants from the downstream of any primary node is the available generation for that node.
2) Amount of Critical Load: For any primary node, the total amount of critical load located downstream of that node is considered for this factor.

3) Connectivity Redundancy:
Graph topology-based physical connectivity is used to determine the connectivity redundancy among all the critical loads presented downstream of any primary node, all the secondary nodes with power supply capacity, and the primary node. This considers all the possible paths through which a critical load can get a power supply for normal operation.

4) Device and Communication Vulnerabilities in Secondary
where N s is the number of total vulnerabilities presented in the secondary level. In case of the absence of any vulnerability, DCVS will be equal to 1.

5) IoT
Trustability Score: ITS determines the IoT devices' trustability presented in any primary node and its downstream. The formulation of ITS is described in Section IV-C. Table II shows the factors considered for resiliency calculation for each type of distribution system configurations.

B. Weight Assignment and Aggregation
Evaluating the impact of factors in the resiliency of cyberpower power systems is a very complex task. This requires expertise decisions from different domains, such as power systems, cyber-power systems, and cyber system experts. It may again raise ambiguities and uncertainties in the existing information, which can be handled by F-MCDM. In fuzzy MCDM models, the linguistic terms or comparisons of different experts are represented by fuzzy numbers [40].
The fuzzy analytic hierarchy process (Fuzzy AHP) is an improvement of a standard AHP [41] method using the fuzzy logic approach. The fuzzy AHP method incorporates the impreciseness of human judgment raised due to the subjective or qualitative nature of the criteria that exact numbers cannot represent. Fuzzy AHP [40] controls the uncertainty and vagueness in the decision makers' opinions through the fuzzy set theory. In this work, a fuzzy rating aggregation method [42] is integrated with fuzzy AHP to the incorporated decision of multiple experts. The fuzzy set theory can easily navigate and incorporate all the decisions to evaluate the impacts of each factor.
The linguistic preference values introduced by Saaty in [41] are fuzzified using the triangular fuzzy numbers. Table III shows the triangular fuzzy conversion scale along with Saaty's scale.
Let there be K number of experts. Once all the experts use above scale to provide their fuzzy pairwise comparison ratings R k = (l k , m k , u k ), k = 1, 2, . . . , K, the aggregated fuzzy ratings can be defined as [42] R = (l, m, u) (8) where The aggregated fuzzy pairwise comparison matrix D = [R ij ] is constructed using the aggregated ratings. For n number of factors, the fuzzy pairwise comparison matrix will be Then, the fuzzy geometric mean value r i , for each factor i is computed as The fuzzy weight w i for each factor is calculated as The center of the area method is used to defuzzify the fuzzy weights w i = (l i , m i , u i ) as below to get the weight w i for each factor Finally, normalization is done to get the final weight W i for each factor as follows: When it comes to aggregation, the multiplicative approach offers more superior performance than the Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.  [41], [43] additive approach [44]. Again, the adoption of multiplicative performance measures is preferred in the general systems performance theory [45]. Therefore, the weighted product model is used to get PNR for any node as follows: (13) where c, and n c indicate the category of the primary level node, and the total number of factors for that c category, respectively. This formulation of PNR ensures that for any primary level node, the resiliency value will remain within a value of 1.

VI. DISTRIBUTION SYSTEM RESILIENCY METRIC FORMULATION IN THE PRESENCE OF IOTS
DSR will give us the overall resiliency of the system.

A. Factors Influencing Resiliency
DSR calculation involves attributes from the primary voltage level of distribution system. 1) Primary-Level Node Resiliency: PNR considers all the attributes considering the secondary level configuration of a primary node. The value of PNR can be calculated following the method described earlier using (13).
2) Available Power Outflow: The available power outflow (APO) from the primary node is the difference between the available power from a different generation and storage resources, and the total amount of critical load presented downstream of that primary node.
3) Primary Node Centrality: Primary node centrality (PNC) provides the importance of a primary level node in the whole distribution in terms of connectivity. In this work, the concept of leverage centrality [46] is utilized to identify the criticality of each network node. The degree of a node relative to its neighbors is considered in leverage centrality. Leverage centrality identifies nodes connected to more nodes than their neighbors. A well-connected node i can pass information to many neighbor nodes. But if those neighbor nodes have a high degree, they do not need to rely much on that node i. Thus, node i ends up with low leverage in the network. Nodes with high leverage centrality control the content and quality of the information received by their neighbors. Although leverage is derived from degree centrality, it is very effective compared to other centralities in determining the importance of any node in a network where network flow can happen in any direction rather than only along the shortest path or in a serial fashion [46]. With modernization, the distribution system has also become this type of network as power flow can happen in any direction. So, to determine the importance of any individual node in the distribution system network, PNC is formulated using the concept leverage centrality as where N, d i , N i , and d j are the total number of nodes, degree of the given node, directly connected neighbors of the node i, and the degree of those neighbors, respectively. PNC formulation in (14) also does not increase computational burden as the distribution system becomes larger.

4) Device and Communication Vulnerabilities in Primary Network:
Once any vulnerability is identified using NVD presented in the primary level of the distribution system, it is assigned to its corresponding primary node based on the source of the vulnerability. In this way, all vulnerabilities presented at the primary level can be assigned to the primary nodes of the distribution system. Then, device and communication vulnerabilities of each primary node (DCVP) is calculated as where N p is the number of total vulnerabilities related to that specific primary level node. In case of the absence of any vulnerability, DCVP will be equal to 1.

B. Weight Assignment and Aggregation
The weight distribution for each factor of each primary node considered in DSR calculation determines the contribution of any node to overall system resiliency. In this work, this weight distribution problem is formulized as a DEA problem where the "weights" in DEA are derived from the data instead of being fixed in advance [47]. Once all the factors for each of the primary level nodes are calculated during normal operation, a game-theoretic DEA [48]-based concept is used to determine the weights so that each node will have the best set of weights.
Let F = (f ij ) ∈ R m×n + be the factors value matrix, where f ij is the value of factor i of the primary node j. The node will contribute more to resiliency metric in regard to that factor as higher the value of f ij goes. Now, following the DEA analysis, each node p can choose a set of weights w p = (w node p to the total contribution of all the nodes toward DSR as measured by node k's weight selection can be evaluated as Now, each node wants to maximize this ratio in (16) to have the best set of weights so that they can contribute to the maximum possible value in DSR. Again, dominance of any specific factor in comparison to other factors in DSR calculation for different distribution systems can vary depending upon the distribution system configuration. So, we have introduced an option to set a minimum threshold of weight w ex i for each factor depending upon the distribution system configuration by the operators or experts of that system. All of these result into the following program: where w ex i = [0, 1]. Once (17) provides the weight vector for each node, a combination of multiplicative and additive methods is used as below to get the DSR Few primary nodes in the distribution system do not expand into secondary voltage levels and are just connectivity with other nodes. For this type of primary node, PNC and DCVP factors are considered in (17) and (18). Since each node can contribute up to a value of 1, for a distribution system with n nodes can have a theoretical maximum DSR value of n.

VII. CASE STUDIES AND RESULTS
For case studies, we have selected IEEE 123 node test feeder system as our test system. This test feeder has spot loads in 85 nodes. Since this work focuses on the resiliency of the distribution system in the presence of IoTs, we have modified those spot load nodes to include IoTs. We have categorized 50 nodes as cyber-power primary nodes with IoTs, 20 nodes as cyber-power primary nodes without IoTs, and the rest of the nodes as a physical primary nodes as described in Section II-B. The case studies have also been structured around IoTs. In this study, we have selected t = 1, T = 12, w t = 0.5, and w t− = 0.5. The case scenarios are well-known cyber events.

A. Validation of Autoencoders With Federated Learning Architecture of ITS
The major components of the ITS formulation are the autoencoder models with the FL architecture. We created a validation data set for each type of data to validate those models. Since the effect of any adverse event in IoTs on their data point is still a topic of ongoing research, for now, it is safe to flag any abnormal data point as an anomaly for IoTs. We prepared the validation data set by introducing values for data points beyond the normal range for each data type. Then, after training and setting up T err , the local autoencoders block reconstructed those validation data points, and the ITS algorithm block flagged all the data points with abnormal values as ADP from their RE values as expected.

B. IoT Trustability Score
During normal operation, ITS is always 1.0, which can be seen in Fig. 6 till reporting time step, t = 14.
For the first case scenario, we have assumed that solar PV of one house from the primary node in Fig. 2 accidentally got disconnected from its smart IoT-based inverter during maintenance of the PV panel at t = 14. The maintenance person quickly noticed it and fixed the connection by t = 16. Data points generated during that event from the solar PV became corrupted and resulted in a drop in nonanomaly ratio (NAR) from 1 to 0.75 and bounce back to 1 at t = 16 as shown by line "NAR_sse" in Fig. 6. This event can be classified as an unintended nonmalicious event. Though ITS of that primary node registered the event and slightly dropped for that event periods as shown by line "ITS_sse" in Fig. 6 as expected.
For the second case scenario, we have assumed that two houses and the commercial building use smart IoT-based inverters from the same manufacturer for their solar PV and battery, and attackers have discovered vulnerabilities of the inverters of that manufacturer. Once the attackers have gained access to the inverters at t = 14, data points of PVs and batteries start to have irregular values due to the malicious activity of those attackers. NAR of one of those houses described by line "NAR_me" in Fig. 6 shows a significant drop. Since three out of seven users of that primary node are under attack, ITS of that primary node also degrades a lot, as shown by the line "ITS_me" in Fig. 6. As the attackers keep gaining full access to those inverters, anomalous data points continue, and ITS also remains degraded as expected.

C. Primary-Level Node Resiliency
Now, we will calculate the PNR of the primary node with IoTs shown in Fig. 2. Let us assume there are two operators or experts and they provide below pairwise comparison matrices Authorized licensed use limited to the terms of the applicable license agreement with IEEE. Restrictions apply.
Now, by following all the steps of fuzzy AHP explained in Section V-B, we get the weights shown in Table IV for the factors from these two pairwise comparison matrices. Now, once all the factors values are determined, we calculate PNR for this cyber-power primary node with IoT using (13) for both case scenarios explained in Section VII-B and plot in Fig. 6.
PNR for the first case is shown by line "PNR_sse" in Fig. 6. It does not have a noticeable change in its value as all the factors remain constant except ITS, which also does not change much during the event periods.
For the second case scenario, PNR exhibits significant change as shown by line "PNR_me" in Fig. 6. At t = 14, PNR starts dropping as ITS drops. Then, at t = 25, the attackers successfully disconnect the solar PVs and batteries associated with those attacked inverters. These disconnections result in the reduction of available generation. Now that another factor loses its value at t = 25 along with previously reduced factor ITS, PNR drops again from the next t as shown by line "PNR_me" in Fig. 6.

D. Distribution System-Level Resiliency
Since PNR covers all the secondary voltage areas, which is the most significant part of the distribution system's operation area, it should contribute most to the overall distribution system level resiliency. Therefore, this work chooses the minimum threshold of weights for factors as 0.4, 0.1, 0.1, and 0.1, respectively, for PNR, APO, PNC, and DCVP. Then, the factors value and their weights are calculated for each reporting time step t and combined to get the distribution system level resiliency as explained in Section VI.
For the first case scenario, we have extended the unintended nonmalicious event similar to the event explained earlier to four cyber-power primary nodes with IoTs. One house from those four nodes suffers from an unintentional nonmalicious Fig. 7. DSR during case scenarios on a scale of 0-123 where 123 is the total primary node number of the system. event. As this type of event hardly affects PNR of respective primary node, the overall DSR remains the same as shown by line "DSR_sse" in Fig. 7.
For the second case scenario, we have extended a similar malicious event by considering the presence of smart IoTbased inverters from the same manufacturer in 30 cyber-power primary nodes with IoTs. As attackers attack all of those inverters at t = 14, all of 30 nodes experience a significant drop in ITS, reducing PNR of each node as shown earlier.
Reduction in PNR of 30 nodes brings down DSR. Then, at t = 25, the attackers successfully disconnect all the solar PVs and batteries associated with those attacked inverters. These disconnections result in the reduction of available generation of each of those 30 nodes, which further reduces PNR of each primary node. The APO factor of each of those 30 nodes also reduces as generation in their secondary level decreases due to these disconnections. Now that another factor APO loses its value at t = 25 along with further reduced factor PNR, DSR drops again from the next t as shown by line "DSR_me" in Fig. 7.

VIII. CONCLUSION
The introduction of more IoTs-based DERs, loads, and other devices leads to better and efficient operation with flexibility but also brings vulnerabilities. Detailed monitoring of all the resources is becoming critical due to the increasing cyberattack surface and complexity of the system. This also leads to privacy concerns for users owning IoTs. FL-based monitoring can elevate these problems if data tracking is not possible. Thus, without breaching privacy, the overall resiliency metrics formulation presented in this work can provide situational awareness and critical information to the distribution system operators for a distribution system with IoTs. DSR metrics offer visibility to the edge of the system. With DSR applications running in the distribution system control center, the operators can easily navigate through that information and reach the components impacting resiliency. Then, factors related to those components can be used to quickly investigate the system to analyze the event and take suitable remedial actions. For example, during an event similar to the second case scenario, the operators can utilize the information from DSR, PNR, and ITS to detect malicious activities of the latent threat in the system. Then, operators can take timely action such as importing power from external sources to feed the critical loads when PVs and batteries are out of operation due to the attack.
The proposed metrics are capable to facilitate resiliencybased monitoring and operation for any advanced power distribution system. As the metrics quantify all the attributes of the distribution system for a given configuration, one can easily set the DSR for that configuration as the baseline and find out the weaknesses based on the scores. Then actions for improvements can be prioritized based on their contribution to incrementing the DSR score from the baseline and taken accordingly considering available resources. It can easily adapt to any upgrade or change in the distribution system, such as installing new DERs, new smart buildings coming into the grid, etc. Since it calculates the DSR metric according to the total number of primary level nodes in the distribution system, it can be used to make a DSR percentage comparison of different systems. Another benefit of the metrics can also be extended toward offline study for designing the possible resilient distribution system for a given cost. For future work, we can make DSR more robust by developing better models of IoT devices using advanced learning algorithms. Developed metrics can also be utilized to develop an operational reconfiguration algorithm to have the most possible resilient configuration during adverse events.