Comparative Evaluation of the Performance of ZigBee and LoRa Wireless Networks in Building Environment

Abstract

ZigBee and LoRa are communication technologies widely used in the application of the Internet of things (IoT), especially in the field of smart building environmental monitoring. The main purpose of this study is to compare and analyze the transmission performance of ZigBee and LoRa wireless communication networks in the building. Through two indicators of packet loss rate (PLR) and round-trip time (RTT), this paper discusses in detail the transmission performance of ZigBee and LoRa technologies in whole buildings under the same working conditions. We set up three experimental scenarios of line-of-sight, horizontal and vertical to evaluate the communication performance of these two networks by changing the baud rate and packet length, and cost and power consumption were considered. Experiments have shown that LoRa networks outperform ZigBee networks in most cases and are the best choice for building communication networks. The experimental results provide basic data support and engineering reference for the application of these two technologies in buildings, especially for the deployment of communication networks throughout buildings. The innovation and contribution of this paper are to discuss the effect of packet length, baud rate, distance, and different locations within a building on the performance of ZigBee and LoRa transmissions, using RTT and PLR as metrics through three experimental scenarios.

1. Introduction

Communication network technology plays an important role in the success of building energy and environmental monitoring systems. As a bridge, the communication network realizes the transmission of data from the physical world to the Internet and becomes an important guarantee for improving data quality [1]. Many previous studies have explored the application of communication technology in the field of building monitoring and made great contributions [2,3,4,5,6,7,8,9]. These monitoring systems use cable communication and wireless communication technology, and the monitoring scenarios include schools, hospitals, laboratories and residences. Ten years ago, wired communication networks were mainly used in the monitoring systems of building energy consumption and environmental parameters, such as RS485 bus and CAN bus for data transmission. Kolokotsa et al. [10] conducted an experiment in a university office and used data from one office to draw conclusions about the whole building. Zeiler et al. [11] chose a classroom in the teaching building for wired measurement. Dili et al. [12] conducted research on residential buildings and showed pictures of the installation layout, and the wiring was complicated. The common feature of wired communication technology is that there is only one sensing node connected. It features a stable and reliable, high data-transmission rate, but cumbersome wiring. When people choose to use a wired communication network, they usually plan a fixed cable deployment when a building is first built. For an already built building, once the network structure changes, a large additional cost will be incurred.

Furthermore, the development of the Internet of Things (IoT) makes it possible for wireless communication networks to be applied to indoor environment monitoring in smart buildings [13]. A wireless communication network overcomes the shortcomings of cumbersome wiring and difficult construction of the wired network, and improves the long-distance transmission capability of the monitoring system. At the same time, the emergence of wireless communication networks solves the problem that wired networks can only store data locally in most cases. Data can be sent to remote servers or IoT cloud platforms for remote monitoring. At present, a large number of studies have used wireless communication networks to monitor the indoor environment in smart buildings. From Bluetooth and radio frequency technologies with relatively short transmission distances to 3G, ZigBee and LoRa technologies for medium and long distances, wireless communication technologies have gradually shown good application effects in indoor environment monitoring systems [14]. Smith et al. [15] and Cho [16] applied Bluetooth to the indoor environment monitoring of residences but required handheld devices to be close to the sensing nodes to obtain data. Tran et al. designed a radio frequency identification (RFID) tag device that can collect available data within a maximum distance of 250 cm from the reader [17]. Martín-Garín et al. [18] deployed a sensing node in the main room of the apartment and transmitted data via WiFi. Moreno-Range et al. [19] placed five instruments with WiFi communication functions in a bedroom for environmental monitoring. Yang et al. [20] conducted experiments in the office using WiFi networks. Carre et al. [21] placed a sensing device with a 3G communication function in a lounge for experiments, but the SIM card needed to be inserted into the device to generate traffic to connect to the Internet. Through the analysis of the above studies, most researchers currently choose to use wireless communication networks for monitoring, which can reduce the complexity of wiring. Due to the distance limitation of Bluetooth and radio frequency transmission, researchers often only use them for fixed-point monitoring. Although WiFi and 3G networks can be deployed at multiple points, factors such as access traffic and whether the building supports these two networks should be considered in actual deployment.

With the emergence of ZigBee and LoRa wireless communication technology, new solutions appear in the network deployment of monitoring systems in buildings. ZigBee networks are favored by researchers because of their characteristics of free frequency bands and self-organizing networks, and LoRa networks are also suitable for monitoring systems with long-distance transmission and low power consumption. Karami et al. [22] deployed two ZigBee module routing nodes and a coordinator in the laboratory to collect indoor environmental data, continuously collecting data for 10 days, and no missing data were found during the data-collection process. Benammar et al. [23] deployed 14 ZigBee nodes on one floor of the library and tested the impact of wall obstruction on the PLR. Yang et al. [24] deployed 10 sensing nodes in a university to measure several buildings and used the ZigBee network to transmit environmental data. Vcelak et al. [4] arranged a LoRa sensor node in a high school to monitor indoor environmental data. Zhao et al. [25] deployed eight LoRa sensing nodes in the building and measured the PLR of LoRa on different floors. Liang et al. [26] deployed multiple LoRa sensing nodes on the same floor and different floors in the building and measured the LoRa transmission performance from multiple perspectives.

Through the above analysis, ZigBee and LoRa networks can be used for multi-point monitoring, which is suitable for the application scenarios of the whole building or group of buildings. However, most researchers only use one of the two in the experiment, lacking the situation of deploying ZigBee and LoRa networks simultaneously for performance comparison, and the experimental scenario is relatively simple, which cannot fully describe the network deployment of the whole building. In this way, for end users and subsequent researchers of the IoT, there are limitations in the choice of communication networks when deploying a monitoring system. Additionally, with the development of the IoT, companies, enterprises, hospitals, schools and other groups have become the largest source of orders for monitoring systems. They often focus on the communication network deployment of the whole building or building group. More importantly, users can choose the appropriate wireless communication network to deploy the system according to their requirements for the monitoring system and refer to the performance comparison results of the communication network, which is one of the original intentions of the IoT application research.

The main purpose of this study is to compare and analyze the transmission performance of ZigBee and LoRa wireless communication networks in the building. Through three experimental scenarios of line-of-sight, horizontal (same floor) and vertical (different floors), several sensor nodes are deployed in the building, and the monitoring range runs through the whole building. The round-trip time (RTT) and packet loss rate (PLR) of ZigBee and LoRa networks under the same working conditions are measured, and the transmission performance of the two networks is compared and analyzed. The experimental results provide basic data support and engineering reference for the application of ZigBee and LoRa technology in buildings, especially the communication network deployment of the whole building, and provide the choice and optimization of a communication network for users to implement the indoor environment monitoring system. The innovation and contribution of this article are to use RTT and PLR as indicators to discuss the impact of packet length (PL), baud rate (Baud), distance and different locations in buildings on the transmission performance of ZigBee and LoRa through three experimental scenarios.

The remainder of this study is organized as follows. Section 2 describes the related works. In Section 3, three experimental scenarios and measurement devices are introduced in detail, and the deployment and parameter setting of ZigBee and LoRa wireless modules are discussed. Section 4 analyzes the experimental results and provides a discussion. Finally, the study is concluded in Section 5.

2. Related Works

2.1. LoRa Technology

LoRa is a physical layer technology with the ability to connect multiple devices in a wide area of coverage, with low power consumption and with interference robustness [27]. Understanding the performance and characterization of the LoRa technology in indoor buildings is imperative for its deployment and application. Xu et al. [28] investigate the large-scale fading characteristic, temporal fading characteristic, coverage and energy consumption of the LoRa technology in four different types of buildings. The choice of the best location for the installation of gateways, as well as a robust network server configuration, is key to the deployment of a LoRa [29]. Souza et al.’s [30] study indicates that one must take into account the environment temperature on the deployment plan of LoRa, mainly because of its influence on the battery capabilities, the propagation conditions and the noise behavior. Silva et al. [31] describes the tools for simulating LoRa networks in the ns-3 network simulator. The performance of LoRa networks is evaluated using the simulators. Compared with the actual deployment experiments, the use of network simulation tools can greatly reduce the cost. Furthermore, simulators are easily extensible, and they allow us to study scenarios that are difficult or expensive to investigate in real systems. However, it is difficult for the simulator to fully consider and simulate the multiple complex and unexpected problems in the real environment.

2.2. ZigBee Technology

ZigBee is a low-cost, low-power, reliable, multi-hop wireless network technology that provides high reliability, larger coverage and easy integration into new and existing home control products. Due to these properties, ZigBee technology is considered as a potential solution for home automation. Vo et al. [32] describes a practical design and implementation of a multi-story building automation system using a ZigBee wireless sensor network. Gezer et al. [33] developed a prototype using the ZigBee smart energy profile. The network performance in different monitoring scenarios has been evaluated in terms of packet error rate and latency. Dash et al. [34] present the results of an experimental study on the performance of ZigBee networks in the environment. They measured the packet drop rate of unidirectional transmission and the throughput of bidirectional transmission in the ZigBee network. Adi et al. [35] examines the ability and Performance or Quality of Services of the IEEE 802.15.4 or ZigBee Radio Frequency module on the ZigBee-based sensor node. At present, there have been many experimental studies on LoRa or ZigBee network performance. However, little has been done to compare and analyze the transmission performance of ZigBee and LoRa wireless communication networks for the same operating conditions in different experimental scenarios in buildings.

3. Experimental Deployment

3.1. Test Scenario and Node Placement

The comparative experiment of communication performance between ZigBee and LoRa was carried out in a building in Dalian, China. The division of the building area is shown in Figure 1, which is divided into blocks A and B, and the middle is connected. Among them, there are 12 floors in block A, and the effectively usable floors with the same internal structure are floors 4–12. The height of each floor is 4 m. The walls of the rooms on each floor are made of cement, and the walls between floors are made of reinforced concrete. Block B includes 16 floors, and the effectively usable floors are floors 3–16. The height and wall materials are the same as those of block A, but the house layout of each floor is inconsistent. In order to eliminate other interference, most of the experiments in this paper are completed in block A. The plan structure of the floors that contain both blocks A and B is shown in Figure 2.

This paper divides the performance comparison test of ZigBee and LoRa networks into three cases. The three cases are independent experiments and there is no interference between them. The specific description is as follows:

Case I: Line-of-sight experiment without obstacles. Select the farthest line-of-sight inside the seventh floor, as shown in Figure 2; the position is marked in red, and the labels are E and F.

Case II: Horizontal experiment with the obstruction of cement walls. As shown in Figure 2, we deployed a coordinator (also known as a transmitter, labeled C), a ZigBee router (labeled R), and five receivers (labeled numbers 1–5) at the position marked green on the seventh floor. The five receivers are located on the southwest, northwest, southeast, northeast and opposite sides of the coordinator.

Case III: Vertical experiment with the obstruction of reinforced concrete walls. As shown in Figure 2, the position is marked in orange on floors 4–12. In this case, there are two types of experiment. One is the penetration performance test in the reinforced concrete wall, which is recorded as the central position experiment, and the label is Z. The other is the transmission performance test at the window position, which is located on the balcony of room 3 with the label K.

3.2. Hardware Module and Configuration Software

3.2.1. ZigBee Module

The ZigBee wireless communication module is shown in Figure 3a. It is a ZigBee wireless transceiver developed based on the CC2630 chip of the 32-bit ARM Cortex-M3 core. It can be connected to a host computer through an RS485 serial port to form the transmitter (coordinator) and connect with a data acquisition instrument to form a receiver. In addition to the transmitter, each ZigBee wireless communication module can be used as a router to perform data-forwarding functions, as a coordinator or router is achieved through configuration software. In this paper, the DTK-ZigBee configuration software of the EBYTE company is used to set parameters such as node type, communication channel and baud rate, as shown in Figure 3b. The basic parameters of the ZigBee module are shown in

Table 1. Basic parameters of ZigBee module.

Interface	Node Type	Baud (bps)	Receive Sensitivity (dBm)	Supply Voltage (V)	Work Current (mA)	Carrier Frequency (MHz)	Price
RS485	C */R *	1200–115,200	−100	3.3	T *-200 Re *-20	2400	$6.8

* C: coordinator; R: router; T: transmission; Re: receive.

.

3.2.2. LoRa Module

The LoRa wireless communication module is shown in Figure 4a. It uses the E32-433T30D integrated chip of the EBYTE company, and the core is the SX1278 LoRa spread spectrum chip of the SEMTECH company with the transmission power of 21–30 dbm. It can also connect with a host computer through the RS485 serial port to form a transmitter (coordinator) and with a data acquisition instrument to form a receiver. The configuration software used in this paper is the RF-Setting software of the EBYTE company, which is mainly used to modify the baud rate of the LoRa module, as shown in Figure 4b. The basic parameters of the LoRa module are shown in

Table 2. Basic parameters of LoRa module.

Interface	Baud (bps)	Receive Sensitivity (dBm)	Supply Voltage (V)	Work Current (mA)	Carrier Frequency (MHz)	Price
RS485	1200–115,200	−130	3.3	T-600 Re-20	433	$8.4

.

3.2.3. Data-Acquisition Instrument

The data-acquisition instrument is a sensor module with an RS485 interface. At the receiving end, the data-acquisition instrument can be connected with the ZigBee and LoRa modules, respectively, to form the corresponding network receiver. It can receive the Modbus inquiry message sent by the host computer through the communication network, and then return a series of Modbus response messages. This process can be displayed on the measurement software.

3.2.4. Measurement Software

The function of the measurement software is to send and receive data packets, as shown in Figure 5. In this figure, Mark 1 is the parameter setting area, which is mainly used to select the serial port and set the baud rate and cycle period of data transmission. Mark 2 is the editing area; the main function is to input the Modbus inquiry message, and the baud rate of the data acquisition instrument is also modified here. Mark 3 is the status display area, in which the blue number is the query message sent by the transmitter, and the green number is the response message returned by the receiver. Moreover, the sending and receiving time of the message can be displayed here. Mark 4 is the counting area, which can automatically record the number of packets sent and received and the byte length of corresponding packets. The data of the status display area and counting area provide the source for the communication performance metrics analysis in the following chapters.

3.3. Performance Metrics

There are many indicators to evaluate the performance of wireless networks, such as RSSI, SNR, PDR, LQI, etc. Most of the indicators in the previous studies focused on RSSI, SNR and PDR [36,37,38,39,40,41,42], and there are few studies on RTT through experiments. However, end-users do not care about the RSSI and SNR of wireless signals. What they can obviously feel are low latency and more effective data. Jang et al. [43] mentioned that from the perspective of communication quality, the most important factors affecting network transmission performance are round-trip time and packet loss rate.

3.3.1. Round-Trip Time (RTT)

RTT refers to the end-to-end transmission delay. It is the time interval from the transmitter sending the query message to receiving the response message sent by the receiver and reflects the real-time and follow-up ability of the data transmitted by the communication network. In the application of IoT, RTT is considered to be one of the important parameters of communication network reliability. Not limited to the indoor environment monitoring system in the building, as long as the application scenarios that require low latency response include alarm push, automatic control/manual control, the RTT needs to be considered, because it involves the timeliness of monitoring and control.

3.3.2. Packet Loss Rate (PLR)

PLR refers to the ratio of the number of lost data packets to the sent data packets during data transmission, that is, tracking the number of messages sent by the transmitter and monitoring the number of messages successfully received by the receiver. Then, the percentage of successfully received packets in the total number of transmissions is calculated. The value of the packet loss rate is 1 minus this percentage. It can reflect the stable performance of the communication network. In the indoor environment monitoring system, the lower PLR can ensure that the data of each collection cycle are stably transmitted to the application layer, so that the system can accumulate more continuous data, thereby providing valuable services for other tasks such as the evaluation, prediction and improvement of the indoor environment.

In addition, in the deployment of wireless networks, it is generally necessary to pay attention to the scale, cost and battery life of sensor nodes. These indicators can be used as a supplement to performance. Since the application scenario focuses on the building, there is generally a power supply in the building. We do not pay much attention to the battery life, but to transmission performance, and only through the scale of experimental node deployment. The power and cost are discussed in Section 4.4. Therefore, when studying the comparison of the communication network transmission performance of ZigBee and LoRa in the whole building, this paper mainly considers two performance indicators: RTT and PLR. The recording source of these two indicators is the measurement software shown in Figure 5.

3.4. Preparations for Experiment

Before the implementation of the experiment, in order to reduce the impact of environmental and working conditions on the two communication networks and ensure the accuracy of the experiment, this paper made the following preparations:

1. Monitor the communication channels of ZigBee and LoRa and select the unoccupied channel for communication to avoid interference;

2. LoRa and ZigBee modules are located in the same position, and tests are carried out at the same time to ensure the uniformity of test conditions;

3. LoRa and ZigBee modules use the same power supply;

4. Use the same measurement software to record RTT and PLR;

5. During the test, the antennas of all communication modules are vertically upward.

3.5. Experiment Implementation

The experimental measurement is divided into three groups: line-of-sight, horizontal and vertical experiments. The vertical experiments include two groups: the central position and the window position. The experimental parameters are shown in

Table 3. Experimental parameters.

Scenarios		Baud (bps)	PL (Bytes)	Router	Sensor Node	Floor
Line-of-sight		change	change	0	1	7th
Horizontal		change	change	1	5	7th
Vertical	Cen *	9600	change	2	1	4–12th
	Win *	9600	change	3	1	4–12th

* Cen: central; Win: window.

. In each group of experiments, we obtained the two outputs of RTT and PLR under different conditions by changing the two inputs of baud rate and packet length (PL). The value of the RTT is the result of the tester manually sending 10 sets of data packets continuously through the measurement software and calculating the average value. The PLR is obtained by cyclically sending 500 data packets and calculating the ratio of the lost packets to the total packets. In addition, when configuring the parameters of the LoRa module, its transmission power and air rate are not the subject of experimental research and only need to be set to a fixed value. The LoRa transmit power is set to 30 dBm and the air rate is 19.2 kbps.

3.5.1. Line-of-Sight Scenario

The purpose of this experiment is to measure the communication performance of ZigBee and LoRa at the farthest line-of-sight on the same floor. On the seventh floor, at the farthest line-of-sight, a coordinator is placed at point E and a receiver is placed at point F. The equipment placement is shown in Figure 6. According to the measurement software shown in Figure 5, the cycle period is set as 3 s. Under different baud rates and PLs, including (Baud = 1200 bps; PL = 7 B, 37 B, 69 B, 133 B), (Baud = 9600 bps; PL = 7 B, 37 B, 69 B, 133 B) and (Baud = 115,200 bps; PL = 7 B, 37 B, 69 B, 133 B), the RTT and PLR of the two networks in the line-of-sight scenario are measured.

3.5.2. Horizontal Scenario

The experiment measured the penetration performance of ZigBee and LoRa on the same floor due to the shielding of cement walls. The network topology is shown in Figure 7. On the seventh floor, the coordinator is deployed at label C. The purpose of this test is to research the penetration performance of the two networks under the same conditions. The selection of the coordinator is not the location where the overall network transmission performance is the best. Five receivers are deployed in all directions farthest from the coordinator, with labels of 1–5, and the height from the ground is consistent. The equipment placement is shown in Figure 8. In addition, a ZigBee router is placed near the elevator, labeled R. The reason is that when using the DTK-ZigBee configuration software to read the ZigBee network topology of label 1 and label 2, the signal strength status of both is disconnected, which means that they cannot communicate with the coordinator. The solution is to use the ZigBee self-organizing network feature to place a ZigBee router between label 1, label 2 and the coordinator to realize the fully connected state of the network. Label 4 can automatically establish a connection with the coordinator through label 3, which also reflects the characteristics of the ZigBee self-organizing network.

Through the measurement software, the cycle period is set as 4 s. Under different baud rates and PLs, including (Baud = 1200 bps; PL = 7 B, 37 B, 69 B, 133 B), (Baud = 9600 bps; PL = 7 B, 37 B, 69 B, 133 B) and (Baud = 115,200 bps; PL = 7 B, 37 B, 69 B, 133 B), the RTT and PLR of the two networks in different directions are measured.

3.5.3. Vertical Scenario

This experiment selects the 4th–12th floors with the same internal structure in block A of the building. In the case of a baud rate of 9600 bps, by changing the packet length, including PL = 7 B, 37 B, 69 B, 133 B, the penetration performance through the reinforced concrete wall in the central position and the transmission performance in the window position of ZigBee and LoRa communication networks were measured.

The central position is a passageway connecting the corridors on the north and south sides, and the east and west sides are all windowless walls, as shown in the point labeled Z in Figure 2. Data transmission between the coordinator and receiver needs to penetrate the reinforced concrete wall between the upper and lower floors. We placed one coordinator on the fourth floor and one receiver on the floor to be measured, which is directly above the coordinator. With the increase in the floors to be measured, there was no packet loss in the ZigBee network when measuring the sixth floor, and the PLR of the ZigBee network reached about 10% when measuring the seventh floor. Furthermore, the ZigBee receiver on the eighth floor could no longer establish a stable connection with the coordinator. In view of the good performance of ZigBee penetrating two floors, we placed a ZigBee router every two floors, so that the ZigBee network could continue data transmission on higher floors. The routing nodes were placed on the sixth, eighth and tenth floors, respectively. The routing cascade of the ZigBee network is shown in

Table 4. Description of ZigBee cascade.

Coordinator Location	Receiver Location	Router Location	Network Topology
4th	5th	NA	4th→5th
4th	6th	NA	4th→6th
4th	7th	6th	4th→6th→7th
4th	8th	6th	4th→6th→8th
4th	9th	6th and 8th	4th→6th→8th→9th
4th	10th	6th and 8th	4th→6th→8th→10th
4th	11th	6th and 8th and 10th	4th→6th→8th→10th→11th
4th	12th	6th and 8th and 10th	4th→6th→8th→10th→12th

.

For the implementation of the window position experiment, one coordinator was placed on the balcony on the fourth floor, and one receiver was placed on the balcony on the floor to be measured, as shown in the point labeled K in Figure 2. With the increase in the floors to be measured, the ZigBee network still showed no packet loss on the sixth floor. The PLR on the seventh floor was about 5%, while that on the eighth floor was more than 50%. Although the performance of the network was better than that of the central position, it also caused the phenomenon that the ZigBee network could not establish a connection when measuring higher floors. We adopted the same deployment situation as the above central position experiment and added ZigBee routing nodes to the balcony of every two floors. In addition to the eleventh floor, the routing cascade of the ZigBee network is also shown in Table 4. The cascade of the eleventh floor is 4th-6th-8th-11th. The routing node of the eighth floor skips the routing node of the 10th floor and directly connects with the receiving node of the eleventh floor.

4. Experimental Results and Analysis

4.1. Line-of-Sight Scenario Experiment

Under the line-of-sight condition, there is no packet loss phenomenon in the two networks; both of them show excellent communication stability, while the RTT of the LoRa module is lower in most cases, so it is more suitable for the barrier-free scenarios that require low latency. The maximum RTT value of the two networks is shown in

Table 5. The Maximum RTT value in line-of-sight scenario.

PL (Byte)	Baud (bps)	Network	RTT (ms)
133	1200	ZigBee	2380
		LoRa	1925
	9600	ZigBee	523
		LoRa	452.5
	115,200	ZigBee	249
		LoRa	312

.

4.2. Horizontal Experiment

4.2.1. Analysis of RTT

Figure 9a–c shows the RTT of receiving nodes in different directions on the same floor at Baud = 1200 bps, 9600 bps and 115,200 bps, respectively. It can be found that the RTT at Baud = 1200 bps is between 500 and 3000 ms; at Baud = 9600 bps between 200 and 700 ms; and at Baud = 115,200 bps between 150 and 500 ms. The comparison shows that the RTT decreases with the increase in baud rate, and both ZigBee and LoRa follow this rule. Regardless of the baud rate and PL, the RTT of LoRa at positions 1 and 2 is always lower than that of ZigBee, which reflects that the LoRa network can better adapt to the complex conditions inside the floor. ZigBee at positions 2 and 4 are forwarded by routing. Although the addition of routing nodes enables ZigBee to connect to nodes farther away, multi-level routing increases the latency, making the RTT higher than LoRa. The two networks at positions 3 and 5 are directly connected to the coordinator; the only difference is that position 3 penetrates the concrete wall more, so its RTT is higher than that at position 5. Since the ZigBee modules at locations 3 and 5 are directly connected to the coordinator, the communication time between the modules is substantially reduced. For this reason, the communication latency of the ZigBee modules at these two locations is not very different from that of the LoRa modules, even to the extent that the latency data for individual scenarios are slightly lower than that of LoRa.

4.2.2. Analysis of PLR

Figure 10a–c shows the PLR of receiving nodes in different directions on the same floor at Baud = 1200 bps, 9600 bps and 115,200 bps, respectively. It can be seen from the comparison of the three graphs that as the baud rate increases, the PLR also increases, indicating that the communication stability worsens. At each baud rate, the maximum packet loss of the LoRa module occurs at position 3, which is separated by nine cement walls from the coordinator, and the maximum PLR is 5%. Although the situations of positions 1 and 2 are relatively complex and there are elevator shafts, the PLR is not the maximum, which may be related to LoRa’s CSS technology to actively find the optimal path. The minimum packet loss occurs at position 5, which is closest to the coordinator, and the PLR is less than 1%.

The ZigBee module has a poor cement wall penetration effect. The maximum packet loss occurs at position 2, and its PLR is 10% and 27% at Baud = 1200 bps and 9600 bps, respectively. What is more serious is that the PLR exceeds 35% at Baud = 115,200 bps, which cannot meet the stability requirements of the monitoring system. Moreover, the PLR of the receiver passing through the router is higher than that which is directly connected with the coordinator, which indicates that the routing cascade reduces communication stability. The smallest packet loss is also at position 5, with a PLR of 3%.

In addition, when someone moves into the room where the coordinator is located, the door opening or closing will also affect the stability of the network. For example, when Baud = 115,200 bps, the PLR of the two networks at position 5 dropped twice, and the door was open at that time.

4.3. Vertical Experiment

4.3.1. Analysis of PLR in Central Scenario

The test results of the PLR through the reinforced concrete wall in the central position are shown in Figure 11. The ZigBee module has a router placed on every two floors. Due to the cascading effect of the self-organizing network, the receiving node of the eighth floor is the farthest measured, which penetrates four layers of reinforced concrete walls. However, the PLR of the eighth floor exceeds 30%, which no longer satisfies systems with higher accuracy requirements. The ZigBee receiving node directly connected to the coordinator can penetrate two layers of reinforced concrete walls without packet loss; then, there will be packet loss when passing through the routing node. From 7% to 10% PLR of penetrating three layers to 30–60% of penetrating four layers, and then to 100% of PLR behind, the cascading performance increasingly worsens.

The farthest measurement distance of the LoRa module Is the tenth floor, which penetrates six layers of reinforced concrete walls. When PL = 7 B, 37 B, 69 B, 133 B, the corresponding PLR is 1%, 1%, 1%, 2%, and there is no packet loss in other lower floors, showing better penetration performance in reinforced concrete walls. The PLR of the eleventh floor directly changes to 100%, which is a large drop compared to the tenth floor. It seems that the LoRa module does not show the gradual change trend of PLR as for the ZigBee module, but changes abruptly.

4.3.2. Analysis of RTT in Central Scenario

Figure 12 shows the RTT test results of penetrating the reinforced concrete wall in the central position. From the perspective of the PL, the RTT of the two networks on each floor increases with the increase in the PL. Within the range allowed by ZigBee’s farthest transmission capability, that is, eighth and below, the RTT of ZigBee is lower when PL = 7 B, 37 B, 69 B, while LoRa has the advantage of a lower RTT than ZigBee when PL = 133 B. This may mean that when a small number of reinforced concrete walls are penetrated longitudinally, the ZigBee network is more suitable when lower latency and less PL are required, while the LoRa network is suitable for the case of larger PL.

From the perspective of the floor, regardless of the PL, the RTT of the ZigBee network increases with the rise of the floor. The LoRa network has the same rule when PL = 7 B, 37 B, but when PL = 69 B, 133 B, it shows the characteristics of a lower delay on the high floor than on the low floor, which may be related to the phenomenon of partial packet loss when the larger data packet is transmitted over a longer distance, resulting in the packet with missing messages returning faster.

4.3.3. Analysis of PLR in Windows Scenario

The test results of the PLR in the window position are shown in Figure 13. Overall, both ZigBee and LoRa can transmit data to the highest floor. Most obviously, there is no packet loss in the LoRa modules on all floors, which means that the LoRa network has excellent transmission performance near the window. For the ZigBee module, no packet loss occurs when it penetrates three floors. After that, the eighth, ninth, and tenth floors all have large packet loss, and the PLR increases with the rise of the floor, which shows that the cascade effect is increasingly worsens. However, the PLR of the eleventh floor (labeled as the yellow line) is lower than the previous floors. After the investigation, it was found that the owner of the room on the eleventh floor had opened the balcony windows, while the windows on other floors were closed. The opening of the window changes the ZigBee routing path and reduces packet loss. Due to the improvement of network performance on the eleventh floor, the PLR of the twelfth floor is also lower than that of the ninth and tenth floors.

4.3.4. Analysis of RTT in Windows Scenario

Figure 14 shows the RTT test results in the window position. Generally, the RTT of each floor increases with the increase in the PL, and LoRa shows the advantage of lower latency. In the same packet length, the RTT of the ZigBee module has an increasing trend with the rise of the floor, and there is a downward inflection point on the eleventh floor, which is also related to the opening of windows on the eleventh floor. However, the RTT of the LoRa module does not change significantly with the rise of the floor. The RTT of the LoRa network on each floor is basically the same when the same-size packet is sent. Compared with the central position, the delay consistency shows that there is no message loss in the packet when the larger packet is transmitted to the higher floor near the window. The RTT of the LoRa module on the eleventh floor is also reduced.

4.3.5. Comparison between Center Position and Window Position

Comparison of PLR

The comparison results of the PLR of the ZigBee network at two positions in the vertical direction are shown in Figure 15a. Generally, the PLR of the ZigBee network at both locations increases as the floor rises. On the seventh floor and below, the PLR is less than 10%, but the packet loss on the other floors is serious. For the central position, the minimum PLR (that is, the case of PL = 7 B) of the 5–12th floors is 0, 0, 3%, 30%, 100%, 100%, 100% and 100%, and for the window position 0, 0, 0, 25%, 39%, 45%, 10% and 35%.

The comparison results of the PLR of the LoRa network are shown in Figure 15b. Overall, the LoRa network basically has no packet loss. For the central position, the LoRa can penetrate to the tenth floor at most, and there is no packet loss in the range of the tenth floor and below. The LoRa near the window position still has excellent transmission capacity at the highest layer, and there is no packet loss on the twelfth floor and below.

Comparison of RTT

The comparison results of the RTT of the ZigBee network at two positions in the vertical direction are shown in Figure 16a. In most cases, the small ball is contained in the large ball, indicating that the RTT between the central position and the window position is equivalent. There are a few cases in which the RTT is different between the two positions, which may be affected by working conditions and the environment. In the case of PL = 7 B, 37 B, 69 B, the RTT changes slowly with the increase in floors, and the change is more obvious when PL = 133 B. In addition, the maximum value of RTT appears at the highest layer of the corresponding transmission range. Under the condition of PL = 7 B, 37 B, 69 B, 133 B, for all floors, the maximum RTT in the central position is 255.2 ms, 335 ms, 395 ms and 590.7 ms, and in the window position 287.2 ms, 333.2 ms, 421.5 ms and 660 ms, respectively.

The comparison results of the RTT of the LoRa network are shown in Figure 16b. Generally, the RTT near the window is lower. Under the condition of PL = 7 B, 37 B, 69 B, 133 B, for all floors, the maximum delay in the central position is 275.2 ms, 361 ms, 387.2 ms and 492 ms, and in the window position 250.2 ms, 330 ms, 390.2 ms and 489.2 ms, respectively.

4.4. Discussion

ZigBee and LoRa are widely used communication technologies in the application of the Internet of Things, and, especially in the field of smart building environmental monitoring, each has its own characteristics. The performance of ZigBee and LoRa technology in the whole building is compared and analyzed from five different dimensions. In specific applications, including but not limited to buildings, the appropriate communication technology can be selected from these five aspects according to the actual situation.

• Penetrability: for the whole building, a single LoRa module can basically meet the transmission requirements of line-of-sight, horizontal and vertical scenarios. However, the penetration capability of a single ZigBee module is weak, and it can increase the penetration distance by adding routing nodes, but the overall effect after cascading is still not as good as LoRa;
• PLR: The LoRa network has lower a PLR, while the ZigBee network has a higher PLR. Especially as the wall obstruction increases, LoRa reflects better transmission stability in a complex environment;
• RTT: LoRa adopts CSS communication technology, which can optimize the transmission path when encountering obstacles. Therefore, in some cases, the RTT will be higher than that of ZigBee, but in most cases, the RTT of LoRa is lower;
• Cost: The cost of a single ZigBee module is lower, but with the addition of routing nodes, the cost will increase. The total cost of ZigBee and LoRa in actual deployment depends on the number of devices;
• Power consumption (power-cons.): The power consumption of a single LoRa may be three times that of ZigBee, and ZigBee’s power consumption is lower when deployed to a small scale and short distance. The power consumption of ZigBee and LoRa networks in actual deployment also depends on the number of devices.

In our experiments, we found that the transmission performance of these two networks in the window position is better than in the closed central position. Moreover, opening the window or the door has a positive impact on the transmission performance.

In addition, some interesting experimental phenomena were discovered during the experiment, which are expressed in the following two aspects.

(1) The continuous packet loss of the ZigBee network: When the coordinator sends query data packets in a loop, no response packets are returned for a continuous period of time, while response packets continue to return during another period of time. This may be related to the change in network signal strength and the instability of network connections.

(2) LoRa’s performance is affected by the surrounding environment of the transmitter: Specifically, when the researcher is close to the LoRa transmitter, the PLR of the LoRa network decreases significantly. This phenomenon appears in the closed vertical-central position experiment; the PLR of the highest layer will change from 100% when no one approaches to less than 10% when someone approaches, which means that the proximity of the human body enhances the penetration ability and transmission stability of LoRa.

5. Conclusions

ZigBee and LoRa communication technologies provide available transmission networks for energy and environmental monitoring in buildings. In this paper, we focus on the performance of the two networks in the whole building, rather than a single floor or single communication network. Through three experimental scenarios of line-of-sight, horizontal and vertical, the baud rate and message length are changed to quantify the PLR and RTT of the ZigBee and LoRa networks under the same working conditions. The LoRa network has a better penetration ability for cement walls and reinforced concrete walls and can reach a longer communication distance under obstacles. Although the ZigBee network can be forwarded through the cascade of routing nodes for long-distance communication, the loss of data packets is serious. In most cases, the RTT of LoRa is lower, but due to its own characteristics of optimizing the transmission path, the RTT is sometimes higher than that of ZigBee. For the measurement of different floors, placing the equipment near the window has a better transmission effect. The experimental data can provide engineering reference for users to carry out environmental monitoring or other data transmission tasks of the whole building, and they can select ZigBee and LoRa according to their own needs. Overall, the performance advantages of LoRa are more obvious, and it is a good choice as a network deployment for the entire building. In terms of cost and power consumption, ZigBee networks can also be considered by users.

In future work, we will study the selection of node locations that can maximize the performance of ZigBee and LoRa networks in the entire building. In addition, corresponding multi-sensor integrated monitoring equipment will be developed, combined with the IoT cloud platform, in order to monitor the indoor environment for a long time and realize the prediction of environmental data and the comprehensive evaluation of environmental quality through intelligent algorithms. We are also considering more in-depth work comparing experimental and simulation results for the same scenarios. This could be a good way to evaluate the accuracy of simulation models and will provide a reference for future research work.