Embracing IoT? Wait!! A Test Report from Allion Smart Life Program

The idea of IoT (Internet of Things) and the innovation of technology have created a new lifestyle to people. Today, individuals, when stepping into a house, are able to turn on light, air conditioner, or television by simply speaking a word. However, the larger scale applications of smart technology also created more complex issues.

In this article, we argue that, since the communications of devices have been extended from “one-to-one” to “one-to-many,” product makers need whole new methodologies to validate the functionality, performance, and interoperability of their products. We will share the test results from Allion Smart Life Program and explain issues that caused by cross-cloud communication, inconsistent user interfaces, and wireless signal interference.


IoT Technology & Public Safety  

Modern people tend to equip smart devices (or sensors) to monitor the condition of the environment or the status of the facility, enhancing public safety. For example, teachers can link different smart devices to make the school security system more dynamically. As the illustration below, the sensor will be triggered once the invader opens the smart window during the “Security on” mode. The sensor will then activate the smart alarm and camera, calling them to start ringing and recording. It will send messages to the teacher’s smart phone, too.

max-width:_______px ; height: 100%; width: 100% ; background-image: none; border-width: 0px; display: block; margin: 0px auto; padding: 0px; text-align: justify;

Unlike traditional security system merely allow users to passively maintain the safety of their properties, the dynamic system provides instant response, enabling individuals to react immediately.

Not just about security, smart technology also links other living devices, such as smart light bulbs, smart doors and locks, and smart smoke detectors, to the security system. In order to validate the reliability of these devices in real-world scenarios, we built “Smart Campus” in Allion IoT Innovation Center and designed several situation models.


The Latencies between Cross-Cloud Communications 

In one of our situation models, “There’s Smoke!” the smoke detector was connected with smart light bulb and users’ mobiles. When there is smoke in the classroom, the alarm should announce evacuation order, and the light bulb should start flashing, showing students where the exits are. Other faculties and students on campus should also be able to receive the evacuation message via their mobiles. The system is built to ensure that each individual will receive notifications once emergency occurred. The figure below shows the transmission path of the data package.

First, the smoke detector sends the data package, “There’s Smoke,” to “Cloud 1” via Wi-Fi. “Cloud 1” then sends emergency message to the connected mobiles directly via Wi-Fi or 4G network; it also, at the same time, communicates with “Cloud 2,” seeking to activate the smart light bulb. However, we found 10-40 secs latencies during the communication between Cloud 1 and Cloud 2. This period of latency, if in the reality, could cause serious loss of lives, bringing concerns to autonomous home security systems. 


IoT Technology & Smart Living

Besides public safety, IoT technology smartens up people’s own living spaces. Since the diversity of IoT products (i.e. different brands/types) and the distinctness of user behaviors can affect devices’ functionality, performance, and interoperability, we also structure different kinds of family scenarios in Allion IoT Innovation Center, including single, nuclear, and extended family. We aim to help our clients discover and resolve issues before they release products to the market.

Each scenario has its test focus. For example, we built “House Scenario” to replicate an extended family lives in a big house. In this kind of family, different members might adapt to different brands/types of smart devices. However, the various devices might use various systems, clouds, or controlled by various hubs, causing the interference of signal and the degradation of performance. Therefore, we designed several situation models in House Scenario to validate whether IoT devices can react (or be controlled) properly in a complex ecosystem.


Inconsistent User Interfaces 

To enhance product interoperability, manufacturers often design multiple user interfaces (UI), allowing their products to support other operating systems (e.g. iOS, Android). Nevertheless, we found numbers of bugs in a product’s different UI, and those issues might decrease user satisfaction.

The illustration above shows three kinds of UI of a smart control hub. For iOS users, manufacturers design “integrated app,” which allows them to control other devices by using only one app. Android users, on the other hand, need to download multiple apps to control different devices, complicating its manipulation. What is worse is that some of the settings can only be operated via legacy control. This means users have to connect keyboard and mouse to the smart control hub to fully activate all the functions.  Although, in this case, the product supports different user interfaces, its inconsistency might still reduce user satisfaction.


Wireless Signal Interference 

IoT technology has sophisticated the applications of wireless technology, constructing a more complex wireless ecosystem. However, co-channel and adjacent-channel interferences were often occurred when these signals (e.g. Wi-Fi, Zigbee, Thread, etc.) coexist under the same roof.

For example, one of our smart assistance had not been able to activate the air purifier via voice control. In the beginning we thought it was, again, an issue that caused by cross-cloud communication. Yet, we later found that it was co-channel interference of wireless APs by analyzing the wireless signal heat map report. The heat map report allows us to obtain an immediate visual summary for various wireless signals, perceiving Received Signal Strength Indicator (RSSI) to validate the connectivity and performance of smart devices.

As the figure shown above (on the right side), it is the strong RSSI rate in the environment that decreased the performance of smart assistance. The issue was then solved after we changed the AP distribution (the left side figure).



In conclusion, we have shared the test results of Allion Smart Life Program and discussed the issues that caused by cross-cloud communication, inconsistent user interfaces, and wireless signal interference. We argue that, since smart technology has sophisticated the applications of digital devices, IoT product makers need real-world scenarios and situation models to fully validate the functionality, performance, and interoperability of their products.

In Allion IoT Innovation Center, we structured several situation models in the five user scenarios (i.e. single, family, and school lives), aiming to help our clients discover and resolve issues before they release products to the market. Of course, we will never stop our steps. In the near future, Allion will develop more IoT product validation services, such as utilzing the wireless signal heat map and our RF test capability to guild homebuilders to allocate wireless network while designing homes.

Let us know or like our Page if you are interested in exploring our world-class testing service.