A Dataset to Evaluate IEEE 802.15.4g SUN for Dependable Low-Power Wireless Communications in Industrial Scenarios

Abstract

This article presents a dataset obtained from the deployment of an IEEE 802.15.4g SUN (Smart Utility Network) single-hop network (11 nodes) in a large industrial scenario (110,044 m${}^2$) for a long period of time (99 days). The dataset contains ∼11 M entries with RSSI (Received Signal Strength Indicator), CCA (Clear Channel Assessment), and PDR (Packet Delivery Ratio) values. The analyzed results show a high variability in the average RSSI (i.e., between −82.1 dBm and −101.7 dBm) and CCA (i.e., between −111.2 dBm and −119.9 dBm) values, which is caused by the effects of multi-path propagation and external interference. Despite being above the sensitivity limit for each modulation, these values result in poor average PDR values (i.e., from 65.9% to 87.4%), indicating that additional schemes are needed to meet the link reliability requirements of industrial applications. Hence, the presented dataset will allow researchers and practitioners to propose novel mechanisms and evaluate their performance using realistic conditions, enabling the dependability vision of the RAW (Reliable and Available Wireless) WG (Working Group) at the IETF (Internet Engineering Task Force).

1. Introduction

Industrial and smart grid applications require dependable transmissions for monitoring and control operations, which means reliable, available, and timely delivery of packets [1]. In wired networks, this kind of dependability is achieved using a central path computation entity, which has a complete view of the network topology. However, this model cannot be simply adapted to wireless networks for two main reasons [2]. First, wireless communications are subject to propagation (i.e., the Fresnel zone is typically blocked) and interference (i.e., these networks operate in unlicensed bands) effects. Second, radio conditions may change faster (i.e., mobile nodes or a dynamic environment cause multi-path fading) compared to the adaptability and reprogramming ability of a centralized engine, in particular when the controller is distant or when connectivity is limited.

To guarantee dependability in such adverse conditions, the RAW (Reliable and Available Wireless) WG (Working Group) is currently being formed at the IETF (Internet Engineering Task Force) [3]. RAW enables parallel transmissions over heterogeneous media and multiple hops and leverages the PAREO functions [4], which stand for Packet Replication, ARQ (Automatic Repeat Request), Replication, Elimination, and re-Ordering. In addition, RAW separates the routing time scale, at which a complex path is recomputed, from the forwarding time scale, at which the forwarding decision is made for an individual packet. Hence, the RAW problem is to decide, within the redundant solutions that are proposed by the routing, which link will be used for each individual packet to provide a dependable service while minimizing the waste of resources. That is, a RAW solution would consist of a set of protocols that evaluate the media in real-time, and control the use of redundancy and diversity attributes that are available along the path.

Given the rising interest in dependable low-power wireless communications for industrial and smart grid applications, there are various studies that have recently focused on analyzing the performance of IEEE 802.15.4g SUN (Smart Utility Network) modulations, namely SUN-FSK (Frequency Shift Keying), SUN-OQPSK (Offset Quadrature Phase-Shift Keying), and SUN-OFDM (Orthogonal Frequency Division Multiplexing), as defined in the latest revision of the IEEE 802.15.4 standard [5]. For example, in [6,7,8], the authors studied the suitability of IEEE 802.15.4g SUN modulations for smart metering utility networks. More recently, in [9,10], the authors studied the suitability of IEEE 802.15.4g in smart building and environmental monitoring scenarios, respectively. However, none of these studies have conducted a deployment in an industrial scenario to evaluate the real-world performance of these technologies for a long period of time to demonstrate their benefits and drawbacks.

In this paper, we fill this gap by presenting a dataset obtained from the deployment of 11 nodes using the IEEE 802.15.4g SUN modulations for 99 days in a warehouse with an area of 110,044 m${}^2$. The dataset contains 10,710,868 measurements of RSSI (Received Signal Strength Indicator), CCA (Clear Channel Assessment), and PDR (Packet Delivery Ratio) values. The results show large variability in the average RSSI (i.e., between −82.1 dBm and −101.7 dBm) and CCA (i.e., between −111.2 dBm and −119.9 dBm) values caused by multi-path propagation and external interference effects, leading to poor PDR values (i.e., from 65.9% to 87.4%), despite the fact that nodes are well above the sensibility limit for each modulation. To the best of our knowledge, this is the first paper to present a large dataset using IEEE 802.15.4g SUN modulations in an industrial environment, and we expect that the data gathered will enable researchers and practitioners to study alternatives to enhance the dependability properties of low-power wireless communications in the context of the RAW WG at the IETF.

The remainder of this article is organized as follows. Section 2 provides an overview of the SUN modulations that are standardized in IEEE 802.15-2015. Section 3 describes the environment, the hardware, and the network operation used for the deployment. Section 4 presents the dataset and analyzes the results obtained from the measurement campaign. Section 5 discusses the results and proposes modulation diversity as a means to combat the effects of multi-path propagation and external interference, and to achieve the dependability required for industrial and smart grid applications. Finally, Section 6 concludes the article.

2. Related Work

2.1. Overview of IEEE 802.15.4g

The IEEE 802.15.4 standard [5] was first released in May 2003 and defined a PHY (PHYsical) layer and a MAC (Medium Access Control) layer for LR-WPAN (Low Rate Wireless Personal Area Networks). At the PHY layer, the standard employs the DSSS-OQPSK (Direct Sequence Spread Spectrum-Offset Quadrature Phase Shift Keying) modulation and provides data rates of 20 kbps and 40 kbps in the sub-GHz bands (868 MHz in Europe, 915 MHz in America), and of 250 kbps in the 2.4 GHz band (available worldwide). The selected modulation provides a good trade-off between radio transceiver complexity, robustness, energy consumption, and communication range [10]. At the MAC layer, the standard defines slotted/synchronized and unslotted/unsynchronized operation based on the CSMA/CA (Carrier Sense Multiple Access with Collision Avoidance) channel access mechanism to trade-off the bandwidth, latency, and energy consumption of the devices.

The adoption of the IEEE 802.15.4 standard by different low-power wireless technologies has promoted the revision of the standard three times (i.e., in 2006, 2011, and 2015) in order to clarify the operation and to add new features to both the PHY and MAC layers. For example, the 2015 standard revision adopted the MAC layer proposals defined in the IEEE 802.15.4e-2012 [11] amendment. Among others, this amendment defined the TSCH (Time Slotted Channel Hopping), a channel access mechanism that combines TDMA (Time Division Multiple Access) and FDMA (Frequency Division Multiple Access) to support industrial requirements, including reliable packet delivery (i.e., 99.999%) in adverse conditions, such as multi-path propagation and external interference. Similarly, the IEEE 802.15.4-2015 standard revision included three new physical layers targeted to SUN (Smart Utility Network) applications, as defined in the IEEE 802.15.4g-2012 [12] amendment. The SUN-FSK and SUN-OQPSK modulations focus on maintaining backwards compatibility with previous standards and commercially available transceivers, whereas the SUN-OFDM is focused on adding robustness and improving spectrum efficiency at the physical layer.

In the following subsections, we present the operation of the SUN-FSK, SUN-OQPSK, and SUN-OFDM physical layers, as these are used in the deployment described in this article.

2.1.1. SUN-FSK

The SUN-FSK modulation has two main advantages: the first one is the good power efficiency, due to the constant envelope of the signal, and the low implementation complexity, and the second one is the compatibility with legacy systems. For example, most of the deployed systems for smart metering in the United States are based on FSK modulation schemes, mainly the ones that use the 902–928 MHz frequency band.

SUN-FSK may be used in several frequency bands, which makes it suitable for different regions. In particular, the 2.4 GHz band is allowed worldwide, and the 863–870 MHz band is allowed in Europe. Three different operation modes are defined for each frequency band. The operation mode defines the modulation and channel parameters, such as the modulation type (2-FSK or 4-FSK), the channel spacing, and the modulation index.

In the 863–870 MHz band, three data rates are supported depending on the operation mode selected: 50 kbps, 100 kbps, and 200 kbps. The higher data rate is achieved when using 4-FSK. However, only Mode 1, which uses 2-FSK and provides a data rate of 50 kbps, is mandatory. In this mode, the total number of channels is 34, with a channel spacing of 200 kHz.

At the physical layer, the PSDU (PHY Service Data Unit) can be optionally processed by an FEC (Forward Error Correction) encoder. Two types of FEC may be applied: an RSC (Recursive and Systematic Code) or an NRNSC (Non-Recursive and Non-Systematic Code). FEC shall be employed on the PHR (PHY Header) and PSDU bits, applying a 1/2-rate coding with constraint length $K=4$. Interleaving shall be employed in conjunction with NRNSC coding and may also be employed with RSC coding. The use of data whitening is optional.

The SUN-FSK PPDU (Physical Protocol Data Unit) is composed of an SHR (Synchronization Header) with a preamble, a SFD (Start of Frame Delimiter), a 16 bit PHR, and a payload of up to 2047 bytes. In the PHR, the FCS (Frame Check Sequence) length is defined, as well as the frame length. There is one bit to indicate if data whitening is used, and another one to indicate the occurrence of mode switching. Mode switching is a mechanism that allows changing the symbol rate or modulation scheme used by the radios.

2.1.2. SUN-OQPSK

The OQPSK modulation was introduced in the first version of the IEEE 802.15.4 standard, in 2003, only for the 2.4 GHz band, and providing a bit rate of 250 kbps. In the IEEE 802.15.4g, other modes of use were defined, allowing the use of OQPSK modulation also in other frequency bands, and for bitrates varying from 6.25 to 500 kbps. The SUN-OQPSK employs DSSS (Direct Sequence Spread Spectrum), and different spreading factors may be used. The use of DSSS allows a better resistance to interference. For some bands, it is possible to use an alternative spreading mode, called MDSSS (Multiplexed DSSS).

The PPDU of the SUN-OQPSK frame is composed by a SHR, a PHR, and the PSDU. The symbol rate is defined as the bit rate of the SHR. Other important parameters are defined in the PHR, such as the spreading mode and the rate mode. Up to four rate modes are supported for each frequency band, but only Rate Mode 0 is mandatory. The bit-to-chip mapping used for the SHR and PHR fields depends on the frequency band, and the mapping used for the payload depends on the rate mode. In Mode 3 of the sub-GHz bands, DSSS is not used for the payload, and it is possible to achieve a higher data rate. For the 2.4 GHz band, DSSS is used for the payload in all rate modes.

The use of FEC is mandatory for the PHR field. When using DSSS as the spreading mode, FEC shall also be applied to the PSDU bits, with a 1/2-rate convolutional coding, with constraint length $K=7$. For spreading mode set to MDSSS, FEC is optional, depending on the rate mode. Interleaving shall be employed in conjunction with FEC, to improve robustness against burst errors and to break the correlation of consecutive bits.

2.1.3. SUN-OFDM

SUN-OFDM allows to provide high data rates and a long range, while dealing with multi-path fading and external interference problems. Unlike FSK and OQPSK, OFDM has not been widely used for low-power wireless communications, due to the stringent processing, memory, and energy consumption requirements.

SUN-OFDM may be used in different frequency bands (both sub-GHz and 2.4 GHz) and provides data rates from 50 kbps to 800 kbps. The sub-carrier spacing is constant and equal to 10416-2/3 Hz, while the symbol rate is equal to 8-1/3 ksymbol/s (a symbol period of 120 $\mathsf{\mu }$s). Four different options are defined, each one with a different number of active tones. For each option, a set of MCS (Modulation and Coding Schemes), numbered from 0 to 6, may be used. If a device supports a given option, it has to support all BPSK and QPSK modulations and all MCS for that option. The support for QAM (Quadrature Amplitude Modulation) is optional.

The MCS determines the scheme used to modulate the sub-carriers (BPSK, QPSK, or 16-QAM), whether frequency repetition is applied (i.e., 4×, 2×, or no frequency repetition), and the FEC coding rate (i.e., 1/2 or 3/4). Thus, the effective data rate depends on the MCS used. For example, for Option 2, which uses 52 active tones, the effective data rate may vary from 50 kbps, when using BPSK, a coding rate of 1/2, and 4× frequency repetition, to 800 kbps, when using QAM, with a coding rate of 1/2, and no frequency repetition.

The PPDU of SUN-OFDM is composed by a SHR, a PHR, and the payload. The SHR has a preamble, and the PHR contains the information about the configuration used to transmit the payload. Both SHR and PHR are transmitted using the lowest supported MSC for the option being used.

3. Deployment Overview

This section presents the environment, introduces the hardware, and describes the network operation of the deployment that has been used to obtain the dataset.

3.1. Overview

The network was deployed in a warehouse located in Guadalajara (Spain), as depicted in Figure 1. The warehouse is built of steel and concrete, and measures 451 m × 244 m (area 110,044 m${}^2$). As shown in Figure 2, a total of 11 nodes (orange) and one gateway (green) were placed in the warehouse. The gateway was located near the warehouse entrance (bottom, center) at a height of 2 m, whereas the nodes were placed at different distances and heights from the gateway.

Table 1. Node identifier (i.e., last 2 bytes of EUI-64), distance to gateway (meters) and installation height (meters). Notice that for evaluation purposes nodes are grouped into clusters according to their distance to the gateway (i.e., close, medium, and far), as labeled in the fourth column.

EUI-64 (Last 2 bytes)	Distance (m)	Height (m)	Cluster Group
56-53	34.0	12.0	Close
55-AD	63.0	2.0	Close
55-E4	80.0	6.0	Close
55-99	115.1	2.0	Medium
55-DD	115.1	6.0	Medium
55-65	115.1	10.0	Medium
56-0B	115.1	14.0	Medium
56-32	172.5	2.0	Far
55-B3	221.4	2.0	Far
55-63	224.4	2.0	Far
63-0A	273.5	2.0	Far

shows the node identifiers (i.e., last two bytes of the EUI-64 address) and the distance to the gateway, which ranges between 34.0 m and 273.5 m.

3.2. Hardware

Both the nodes and the gateway are implemented using the OpenMote-B board [13,14], as shown in Figure 3a. The OpenMote-B board, which has a TelosB form factor (64 × 33 mm), is equipped with a Texas Instruments CC2538 SoC (System on Chip) and a Microchip AT86RF215 radio transceiver. The CC2538 [15] includes an ARM Cortex-M3 micro-controller (32 MHz, 32 kB RAM, 512 kB Flash) and a radio transceiver compatible with the IEEE 802.15.4-2006 standard. The AT86RF215 [16] is a dual-band (sub-GHz and 2.4 GHz) radio transceiver compatible with the IEEE 802.15.4-2015 standard, which supports all multi-rate PHY options defined in the IEEE 802.15.4g-2012 amendment (i.e., SUN-FSK, SUN-OQPSK, and SUN-OFDM).

For the nodes, the OpenMote-B was coupled with the OpenMote-Sensors board, shown in Figure 3b. The OpenMote-Sensors is a carrier board equipped with the Bosch BME280 [17] and the Texas Instruments OPT3001 [18] sensors. The Bosch BME280 sensor can measure temperature, relative humidity, and pressure, whereas the OPT3001 can measure ambient light. The nodes were enclosed in a custom 3D-printed case that exposes the sensors and the SMA antenna connectors, but hides the USB connector for protection. Each node was powered by a pair of 2× AA batteries connected in series, providing 2.4 V and containing a nominal charge of 2000 mAh. The laboratory measurements previous to the deployment showed an average consumption of 0.2 mA during operation (described in Section 3.3), allowing the boards to remain operative for about one year.

The gateway, depicted in Figure 4, is composed of a Raspberry Pi 3B+ SBC (Single Board Computer) connected to three OpenMote-B boards that act as IEEE 802.15.4g SUN receivers and a Huawei E3531 HSPA+ modem to provide Internet access. Power to the gateway is supplied through an internal PSU (Power Supply Unit), which translates the 220 VAC to 5 VDC. The gateway runs GNU/Linux and executes a Python script that reads the packets from each OpenMote-B receiver, converts the raw payload to a JSON payload, and forwards it to the cloud back-end using the MQTT (Message Queue Telemetry Transport) protocol.

3.3. Network Operation

Figure 5 depicts the network setup, with the nodes on the left and the gateway with Internet access on the right.

Nodes operate on a 60 second period with an active sub-period of 1750 ms and a sleep sub-period of 58,250 ms, as depicted in Figure 6. During the active sub-period, the nodes sample the environmental sensors (100 ms) and then perform three transmit cycles. At each transmit cycle, the node transmits three different packets, one with each of the IEEE 802.15.4g SUN modulations (i.e., SUN-FSK, SUN-OQPSK, and SUN-OFDM). Between two consecutive packets in a transmit cycle, there is a 50 ms inter-packet delay. Furthermore, between the first and second cycle, there is a 100 ms delay, whereas between the second and third cycle there is a 200 ms delay.

Before each packet transmission, nodes execute a CSMA/CA (Carrier Sense Multiple Access/Collision Avoidance) protocol with CCA (Clear Channel Assessment) to determine if the channel is busy. That is, prior to transmitting, the radio samples the channel to determine the energy present. If the energy is below the CCA threshold for the current modulation, the CSMA/CA succeeds, and the packet is loaded to the radio transceiver and transmitted subsequently. Otherwise, if the energy is above the CCA threshold for the current modulation, the node draws a random number between zero and 100 (representing the number of slots with each slot being 100 $\mathsf{\mu }$s) and the transmission is delayed. The CCA threshold values for SUN-FSK, SUN-OQPSK, and SUN-OFDM have been determined experimentally and set to −94 dBm, −93 dBm, and −91 dBm, respectively.

The configuration parameters for the different IEEE 802.15-4 SUN modulations that are used to transmit the data packets are summarized in

Table 2. IEEE 802.15.4-2015 SUN (Smart Utility Network) modes used in the deployment. Notice that the presented values for the transmit power are the theoretical values according to the configuration of the Atmel AT86RF215 radio transceiver, but have been validated in the laboratory prior to the deployment. MCS, Modulation and Coding Schemes; SHR, Synchronization Header; PHR, PHY Header; DSSS, Direct Sequence Spread Spectrum; PSDU, PHY Service Data Unit; FEC, Forward Error Correction.

	SUN-FSK	SUN-OQPSK	SUN-OFDM
Name	Mode 1	Mode 3	Option 2 MCS0
Data rate (kbps)	50	50	50
Modulation type	BFSK	OQPSK	BPSK
Modulation index	1.0	N/A	N/A
Chip rate (kchips/s)	N/A	100	N/A
Spreading mode	N/A	SHR:(32,1)-DSSS PHR:(8,1)-DSSS PSDU: none	N/A
Pilots Total/Data/Pilot	N/A	N/A	52/48/4
Center frequency (MHz)	863.125	868.3	863.425
Channel spacing (kHz)	200	1300	800
Available channels	34	3	8
Transmit power (dBm)	15	15	9
Sensitivity (dBm)	−114	−116	−111
CCA Threshold (dBm)	−94	−93	−91
Notes	FEC: yes	Coding rate: 1/2 Interleaver: yes	Coding rate: 1/2 Freq.repetition: 4×

. As can be observed, all modulations provide an effective data rate of 50 kbps, but they trade off the link budget and occupied bandwidth. For example, SUN-FSK provides a good link budget (i.e., 129 dB) and the largest number of channels available (i.e., 34) thanks to its narrow bandwidth occupation (i.e., 200 kHz). In contrast, SUN-OQPSK provides the highest link budget (i.e, 131 dB) at the expense of bandwidth occupation (i.e., 1300 kHz). Finally, SUN-OFDM provides a trade-off between channel occupation (i.e., 800 kHz) and link budget (i.e., 120 dB).

Notice that for all the SUN modulations, the transmit power of the AT86RF215 radio transceiver was set to its maximum value. However, there is a 6 dB offset between the value of the transmit power and the actual transmit power of the radio transceiver depending on the selected modulation. Hence, for SUN-FSK and SUN-OQPSK, the packets are transmitted with 15 dBm, whereas for SUN-OFDM, they are transmitted with 9 dBm.

In terms of payload, each packet contains the fields depicted in Figure 7, as summarized next:

• The PAN_ID field is 2 bytes and is set to 0xCAFE for all packets being transmitted. The gateway filters all packets that are not received with such a value to avoid received packets from other networks;
• The EUI_16 field is 2 bytes and contains the last 2 bytes of the EUI-64 address of the nodes transmitting the packet;
• The TX_MODE field is 1 byte and refers to the modulation being used to transmit the packet: SUN-FSK (0), SUN-OQPSK (1), and SUN-OFDM (2);
• The TX_COUNTER field is 1 byte and indicates the packet repetition within the current period (i.e., 1, 2, or 3);
• The PACKET_COUNTER field is 4 bytes and is an absolute sequence number to allow for packet identification and eventual node resets;
• The TEMPERATURE, HUMIDITY, PRESSURE, and LIGHT fields are 2 bytes each and are devoted to storing the sensor measurements. Prior to transmitting, the raw values are converted to their respective physical values and multiplied by 10 to provide single-digit decimal resolution;
• The CSMA_RETRIES value is 1 byte and indicates the number of times that the CSMA/CA process has been repeated before the transmission is allowed;
• The CSMA_RSSI value is 1 byte and indicates the RSSI level (dBm) that has been measured during the CSMA/CA process.

Notice that, regardless of the modulation, the packet length is always 21 bytes. However, before transmission, packets are encrypted using AES-128 and, hence, the length of the PSDU transmitted by each node is always 32 bytes. Thus, considering that packets are transmitted at 50 kbps, each packet requires 5.12 ms to be transmitted. Since each node transmits nine packets every minute and there are 11 nodes in the network, the total channel occupation is 506.88 ms per minute (about 0.85%). Hence, considering the low channel occupation and the use of CSMA/CA prior to transmission, we did not expect a high impact on the results caused by self-interference.

4. Performance Evaluation

This section presents a description of the dataset and conducts a statistical analysis of the RSSI, the CCA, and the PDR values, which are defined as:

• RSSI (Received Signal Strength Indicator): average power (dBm) of a packet that has been received by the gateway;
• CCA (Clear Channel Assessment): average power (dBm) that a transmitter has sampled before transmitting a packet;
• PDR (Packet Delivery Ratio): percentage of packets that have been successfully received by the gateway within a given period of time.

4.1. Overview

The dataset, which can be found at a public GitHub repository (https://github.com/wine-uoc/wisun_traces) together with the Python scripts used to create the plots presented in this article, is composed of 11 files formatted in CSV (Coma Separated Value), one for each sensor that was deployed, and contains the data captured between 31 July and 10 November 2019 (103 days). However, due to external reasons (i.e., loss cellular connectivity), we do not have enough data for 4 days (20–21 August 2019 and 28–29 October 2019, respectively) and, thus, the data analyzed only encompasses 99 days of the total.

The files in the dataset are named data_x.csv, where x is the last two bytes of the EUI-64 node address, and each file contains 13 columns. The first 4 columns are independent of the IEEE 802.15.4g SUN modulation and contain the following fields:

• day: the day that the data packet was received in yyyy/mm/dd format;
• hour: the hour that the data packet was received in hh/mm/ss format;
• pkt_number: the number of the data packet received;
• pkt_retry: the number of the data packet retries (i.e., 0, 1, 2).

The next 9 columns in each file contain 3 different fields that are repeated for each IEEE 802.15.4g SUN modulation (i.e., SUN-FSK, SUN-OQPSK and SUN-OFDM):

• rssi: The RSSI value (dBm) for the current data packet received;
• cca: The CCA value (dBm) sampled before the current data packet was transmitted;
• cca_retries: The number of retries before the transmission passed the CCA check and the data packet was transmitted.

4.2. Data Analysis

First,

Table 3. Node identifier (EUI-64), received packets, RSSI, CCA, and PDR.

EUI-64 (2 bytes)	Received Packets	RSSI (dBm)	CCA (dBm)	PDR (%)
56-53	924574	−84.0	−112.4	72.1
55-AD	1024664	−83.7	−115.5	79.9
55-E4	872200	−82.1	−111.2	68.0
55-99	897718	−96.2	−117.5	70.0
55-DD	1091950	−92.5	−117.3	85.1
55-65	1058746	−85.3	−118.3	82.5
56-0B	871477	−91.3	−118.4	67.9
56-32	1121696	−96.1	−119.9	87.4
55-B3	1076572	−95.0	−119.3	83.9
55-63	926221	−101.7	−118.1	72.2
63-0A	845050	−101.6	−119.0	65.9
Mean	10710868	−91.8	−117.0	75.9

shows the basic statistics (i.e., total packets received, mean RSSI, mean CCA, and mean PDR) aggregated by node. As can be observed, a total of 10,710,868 packets were received out of 14,113,440 packets (i.e., 11 nodes, 99 days, 24 h, 60 min, 9 messages) transmitted by all the nodes during the deployment, representing an aggregate PDR (Packet Delivery Ratio) of 75.9%. However, nodes experienced different communication impairments (i.e., multi-path propagation and external interference), leading to a per-node PDR that ranged between 65.9% and 87.4% depending on the communication distance and the noise levels. In that regard, it is important to notice that even nodes that were close together experienced high RSSI and PDR variability. For example, nodes 55-99, 55-DD, 55-65, and 56-0B were deployed at the same distance from the gateway (i.e., 115.1 m), but have RSSI values ranging between 85.3 dBm and 96.2 dBm and average PDR ranging from 67.9% to 85.1%.

Given these results, we decided to cluster the nodes according to the distance (d) to the gateway, allowing us to make comparisons within similar groups. In particular, we defined three groups of nodes: (1) nodes that are close to the gateway (i.e., $d<80$ m: 56-53, 55-AD, and 55-E4), (2) nodes that are at a medium distance from the gateway (i.e., $80\le d<150$ m: 55-99, 55-DD, 55-65, and 56-0B), and (3) nodes that are far from the gateway (i.e., $d\ge 150$ m: 56-32, 55-B3, 55-63, and 63-0A).

As an example, in Table 3, we can observe that nodes of the far group present an average CCA value (−119.0 dBm) that is lower than those of the close group (−113.0 dBm). This can be explained by the fact that the gateway is located near the warehouse entrance, where other wireless technologies are known to be deployed and operating in the same sub-GHz band (i.e., 868 MHz), creating interference. Moreover, we observe that nodes of the close group with a higher CCA average have lower PDR values at the physical layer. For example, node 55-E4 has a CCA of −111.2 dBm and a PDR of 68.0%, whereas node 55-AD has a CCA of −115.5 dBm and a PDR of 79.9%. This can be explained by the fact that nodes with higher CCA average values would refrain from transmitting more often due to the CCA threshold value defined in Table 2.

Table 4. Per node RSSI, CCA, and PDR (physical layer) values separated by modulation type (i.e., SUN-FSK, SUN-OQPSK, and SUN-OFDM).

EUI-64 (2 bytes)	SUN-FSK			SUN-OQPSK			SUN-OFDM
	RSSI (dBm)	CCA (dBm)	PDR (%)	RSSI (dBm)	CCA (dBm)	PDR (%)	RSSI (dBm)	CCA (dBm)	PDR (%)
56-53	−78.5	−118.5	71.7	−85.0	−114.6	75.7	−88.4	−104.0	68.8
55-AD	−76.8	−120.0	79.8	−89.6	−117.4	78.5	−84.7	−109.2	81.2
55-E4	−75.3	−117.6	66.8	−84.9	−113.2	70.3	−86.1	−102.8	66.8
55-99	−91.9	−121.6	77.6	−99.0	−119.7	77.0	−97.7	−111.3	55.2
55-DD	−86.9	−122.2	88.9	−96.1	−119.5	88.1	−94.5	−111.1	78.4
55-65	−79.8	−122.6	81.5	−85.9	−119.8	88.1	−90.1	−112.4	78.0
56-0B	−86.2	−122.5	74.4	−94.4	−120.4	74.7	−93.4	−112.4	54.6
56-32	−92.0	−123.2	93.9	−101.0	−122.4	89.4	−95.5	−114.0	79.0
55-B3	−93.4	−122.8	88.3	−96.8	−122.0	89.0	−94.9	−113.1	74.4
55-63	−99.6	−122.9	86.7	−106.2	−119.3	74.7	−99.4	−112.0	55.1
63-0A	−97.9	−122.7	84.8	−104.1	−121.5	82.2	−102.8	−112.7	30.6
Mean	−87.1	−121.5	81.3	−94.8	−119.1	80.7	−93.4	−110.5	65.7
Std. Dev.	8.2	1.8	7.8	7.2	2.8	6.6	5.3	3.5	14.7

shows the per-node RSSI, CCA, and PDR values separated by modulation type. As can be observed, SUN-FSK provided the highest physical layer PDR (81.3%) compared to SUN-OQPSK (80.7%) and SUN-OFDM (65.7%). However, SUN-OQPSK provide the minimum standard deviation (6.6) compared to SUN-FSK (7.8) and SUN-OFDM (14.7). This can be explained because SUN-OQPSK has a higher sensitivity (−116 dBm) compared to SUN-FSK (−114 dBm) and SUN-OFDM (−109 dBm), allowing it to successfully receive packets that were received with a lower RSSI with higher probability.

In that regard, it is important to notice that the PDR at the physical layer is low despite the fact that the RSSI is well above the sensitivity limit for each modulation. For SUN-FSK, the average RSSI of −87.1 dBm is only able to provide a PDR of 81.3% despite that the sensitivity limit is −114 dBm. Similarly, for SUN-OQPSK, the average RSSI of −94.8 dBm can only provide a physical layer PDR of 80.7% with a sensitivity limit of −116 dBm. Finally, for SUN-OFDM, the average RSSI of −93.4 dBm can only provide a physical layer PDR of 65.7% with a sensitivity limit of −111 dBm.

Table 5. Per node PDR (physical layer) values separated by modulation type and packet repetition.

EUI-64 (2 bytes)	SUN-FSK			SUN-OQPSK			SUN-OFDM
	P${}_1$	P${}_2$	P${}_3$	P${}_1$	P${}_2$	P${}_3$	P${}_1$	P${}_2$	P${}_3$
56-53	68.2	77.1	69.8	70.6	79.8	76.6	70.0	71.9	64.7
55-AD	83.4	81.2	74.9	81.6	80.2	73.8	87.9	83.5	72.2
55-E4	75.4	70.9	54.2	78.0	75.2	57.7	69.7	66.4	64.4
55-99	80.4	71.8	80.6	79.7	71.1	80.2	68.3	59.2	38.2
55-DD	89.3	88.5	88.8	87.4	88.0	88.9	78.5	77.8	78.8
55-65	68.3	88.3	87.9	80.0	92.4	91.8	63.1	85.1	85.7
56-0B	68.4	80.1	74.7	69.6	79.6	75.1	64.5	63.4	35.9
56-32	94.7	92.0	95.0	90.3	87.4	90.4	80.3	77.3	79.5
55-B3	94.4	94.0	76.7	94.8	94.2	77.9	78.0	84.7	60.6
55-63	82.4	88.7	89.1	71.7	75.5	77.1	55.8	55.9	53.7
63-0A	86.6	83.5	84.3	84.2	80.5	81.9	31.7	28.1	31.9
Mean	81.0	83.3	79.6	80.7	82.2	79.2	68.0	68.5	60.5
Std. Dev.	9.5	7.5	10.9	7.8	7.0	9.1	14.3	16.1	17.7

shows the per-node reception probability for each independent transmission attempt (i.e., P${}_1$, P${}_2$, P${}_3$) using each modulation (i.e., SUN-FSK, SUN-OQPSK, and SUN-OFDM). From the data, it is clear that there is no strong relationship between the modulation, the distance, and the transmission attempt, except for the fact that SUN-OFDM presents the lowest mean and the highest standard deviation of all modulations. As detailed earlier, this can be explained by the fact that the SUN-OFDM transmission power is 6 dBm lower than SUN-FSK and SUN-OQPSK.

As an example, we pick nodes of the medium group (i.e., distance from the gateway equal to 115 m). Within that group, we observe that node 56-0B presents a high variability for SUN-OFDM transmissions, where the PDR ranges from 64.5% in the first transmission attempt to 35.9% in the third transmission attempt. On the contrary, the PDR for node 55-65 ranges from 63.1% in the first transmission attempt to 85.7% in the third transmission attempt. Hence, despite transmission attempts having a large degree of correlation for a given node and modulation given the short time interval between them, variations over 30 percentage points on average can still occur due to the effects of multi-path propagation.

Figure 8 shows the daily average RSSI values for each modulation and node. The shadowed region for each modulation represents the maximum and minimum values observed for each given day, and the grey shadowed area represents the days for which we did not have data. In general, the daily averages were within 10 dB, but the minimum values had drops that were over 30 dB. Since the gateway and the nodes were both fixed, such large drops in the minimal RSSI values could only be explained by the effect of moving objects within the factory.

Figure 9 shows the RSSI histogram grouped by clusters (i.e, close, medium, and far on each row). In general, the RSSI followed a normal distribution with per-node and per-modulation mean and standard deviation values depicted in Table 5.

Figure 10 shows the per-day hourly average PDR values for each modulation and node. The shadowed region for each modulation represents the maximum and minimum values observed for each given day, and the grey shadowed area represents the days for which we did not have data. In general, we observe that the SUN-FSK and SUN-OQPSK modulations provided better PDR than SUN-OFDM. As mentioned above, this was owed to the fact that the link budget was better for these modulations (i.e., both higher transmit power and lower sensitivity). However, we also observed that for all modulations and nodes, there were sporadic drops in the average, maximum, and minimum PDR values, leading to moments where nodes could experience complete loss of connectivity.

Figure 11 shows the daily average CCA values for each modulation and node. The shadowed region for each modulation represents the standard deviation, and the grey shadowed area represents the days for which we did not have data. In general, we observed that the average CCA values for nodes in the close group were higher than the average CCA values of the nodes in the medium and far groups. This could be explained because the nodes were closer to the building entrance, where other wireless technologies were known to operate in the same sub-GHz band. In addition, we observed that there were certain days where nodes experienced sporadic interference peaks of up to 5–10 dB for nodes in the medium and far groups and 20 dB for nodes in the close group. Interestingly enough, such peaks did not seem to be correlated among nodes and seemed to affect the SUN-FSK and SUN-OQPSK modulations more, which could be attributed to concurrent transmissions from other networks operating in the same sub-GHz band.

Figure 12 shows the CCA histogram grouped by clusters (i.e, close, medium, and far in each row). In general, it is interesting to observe that for nodes in the close group the CCA values for SUN-OFDM are spread, whereas for nodes in the medium and far groups, they are more concentrated. As mentioned earlier, this confirms the fact that nodes in the close group operate near the warehouse office, where other wireless technologies transmitting in the 868 MHz band are known to be operating.

Finally, Figure 13 shows the RSSI vs. PDR scatter plot for each modulation and node, where each point is the eight hour average of both parameters. Contrary to common wisdom, where nodes experience connected, transition, and disconnected zones according to their communication distance, our measurements showed that the PDR is not only related to distance, but largely impacted by multi-path propagation and interference. In particular, for nodes in the close and medium groups, where the average RSSI is well above the modulation sensitivity, we observe that the PDR ranges from 0% to 100% in a random pattern. In contrast, for nodes in the far group, we observe that the connected, transition and disconnected zones start to appear in the plots. This confirms the idea that nodes in the close and medium groups are limited by multi-path propagation and external interference, whereas nodes in the far group are limited by propagation effects.

5. Discussion

Traditional low-power wireless communications in the industrial domain have been based on the IEEE 802.15.4 standard, using the OQPSK-DSSS modulation and operating in the 2.4 GHz ISM (Industrial, Scientific, and Medical) band. Given the expected communication range (i.e., typically below 50 m) caused by multi-path propagation and external interference effects, networks based on such a standard have typically relied on mesh technologies, which allow extending the range by forwarding neighbor packets towards the gateway in a multi-hop fashion. However, deploying and maintaining mesh networks has proven to be challenging due to the changing nature of the environment.

Thanks to the deployment in a real-world industrial setting, in this paper, we have shown that the use of the sub-GHz band combined with the robust IEEE 802.15.4-2015 SUN modulations provides good communication characteristics in an industrial setting, with links up to 273.5 m and effective data rates of 50 kbps. This allows to simplify the network deployment and operation compared to mesh networks based on the traditional IEEE 802.15.4 standard.

However, despite the propagation and robustness advantage compared to traditional IEEE 802.15.4 communications, we have observed that communications between the nodes and the gateway are still severely impacted by both multi-path propagation and external interference effects, limiting network performance in terms of reliability and availability. In fact, the average PDR is only 75.9%, which is well below the typical 99.9% requirement for industrial settings.

To cope with packet losses caused by such propagation and interference effects, we used blind packet repetitions, a common technique used to increase the reliability of LR-WPAN (e.g., WirelessHART, ISA 100.11a, 6TiSCH) [19] and LP-WAN (e.g., LoRaWAN, Sigfox, NB-IoT, etc.) [20] networks. However, the results from our deployment show that even three transmissions cannot guarantee a PDR > 99.9%. In fact, with three transmissions, it is only possible to reach an average of 94.2% with SUN-FSK, 94.1% with SUN-OQPSK, and 86.0% with SUN-OFDM, which is still far from the target PDR requirement. Further increasing delivery success adding more transmissions is possible, but it has a high impact on the battery life and is limited by the duty-cycle regulations, as it causes interference to the remaining nodes, as well as to other networks operating in the same area.

A widely used technique to further mitigate the propagation and interference effects is frequency diversity (i.e., channel hopping). However, while frequency diversity is suitable in the 2.4 GHz band given that the available bandwidth is large and the channels are uncorrelated, the benefits in the sub-GHz band (i.e., 868 MHz in Europe and 915 MHz in United States) are mostly limited to mitigating interference effects. In addition, adding frequency diversity requires synchronization between the nodes and the gateway, which adds complexity to the solution and limits certain use cases where there are multiple gateways and/or nodes are mobile.

Interestingly, modulation diversity (i.e., changing modulations for every transmitted packet) has the potential of providing similar effects as frequency diversity, since each modulation has its own properties with regard to robustness against both propagation and interference effects. For example, narrow-band modulations (i.e., SUN-FSK) are robust and provide a large number of channels, but are prone to frequency drifts caused by temperature changes and component aging. In contrast, wide-band modulations (i.e., SUN-OQPSK) are inherently robust to narrow-band interference and can cope better with frequency drift, but have a limited number of channels available. Finally, multi-carrier modulations (i.e., SUN-OFDM) can use time and frequency repetition to provide robustness against both multi-path fading and external interference effects, while providing a trade-off in terms of channels available. Hence, using different modulations for packet re-transmissions can be used as a mechanism to address scenarios where nodes are subject to multi-path propagation and where multiple wireless technologies coexist in the same band.

Until today, modulation diversity has been limited in low-power wireless communications due to the lack of standards and the availability of commercial transceivers. However, the IEEE 802.15.4-2015 standard included the definitions of the IEEE 802.15.4g amendment, which defines a variety of physical layers covering a wide variety of frequency bands, allowing to meet the requirements of different types of applications. In addition, off-the-shelf transceivers that implement all the physical layer definitions of the IEEE 802.15.4-2015 standard are now available, which allows the nodes to use different modulation schemes using only one radio transceiver. Thus, modulation diversity may now be used to deal with the different challenges of low-power wireless communications that need to be faced in industrial environments.

6. Conclusions

In this article, we have presented a long-term deployment in a large real-world industrial environment using the SUN (Smart Utility Network) modulations, as introduced in the IEEE 802.15.4-2015 standard. The resulting dataset shows that despite the increased communication range (i.e., links up to 273.5 m), the use of the SUN modulations alone cannot guarantee the PDR (Packet Delivery Ratio) requirements of industrial and smart grid applications (i.e., PDR > 99%) due to multi-path propagation and external interference effects present in such a challenging scenario. Taking that into account, in the future, we aim to use the presented dataset as a tool to explore and validate the suitability of novel diversity mechanisms, such as combining packet replication with modulation diversity, to increase the PDR of low-power wireless communications while meeting its energy and regulatory constraints (i.e., duty cycling). This goal is aligned with the objectives defined by the RAW (Reliable and Available Wireless) WG (Working Group) at the IETF (Internet Engineering Task Force), in which protocols to manage diversity will be developed as a tool to ensure dependable wireless communications.