Application of Affordance Factors for User-Centered Smart Homes: A Case Study Approach

Abstract

Smart homes improve quality of life by providing various services based on information and communication technologies. However, smart home systems are complicated and user interfaces for the interaction between smart home and user are often not user-friendly, causing potential difficulties and inconvenience for the user. Therefore, in order for smart homes to become user-centered, usability needs to be improved. This study aims to present guidelines for improving the usability of smart homes based on the concept of affordance, which is highly meaningful in user-centered design. To do this, firstly, the affordance factors that could be applied to improve the usability of active devices and user interfaces in smart homes were extracted, secondly, a case study was conducted to analyze the application of affordance factors, focusing on active devices (e.g., control devices and smart appliances) and user interfaces that directly interact with users in smart homes. Lastly, guidelines on the application of affordance factors were presented by combining case analysis results with relevant guidelines. Active devices and user interfaces should provide users with appropriate cognitive, physical, functional, and sensory affordances so that users can use the smart home services easily and conveniently.

1. Introduction

As high-tech information and communication technologies (ICTs), such as the Internet of Things (IoT) and artificial intelligence (AI), are applied to the residential environment, the residential space is evolving. Based on sensors, actuators, and the base system that supports the collection and exchange of information, devices and appliances in the home have become ‘smart’ [1,2] and can provide various services to residents. The ultimate aim of smart home technologies is to improve residents’ quality of life [3,4,5]. However, smart home systems are complicated and user interfaces for the interaction between smart home and user are often not user-friendly, causing potential difficulties and inconvenience for the user. Hargreaves et al. [6] noted that many users have difficulty operating smart home systems because they are not intuitive. Haines et al. [7] stated that smart devices can be misused or underused due to their complexity and the interconnection of diverse devices. Finally, Makonin et al. [8] noted that users have been overwhelmed by the poor usability of smart home systems. Therefore, in order for smart homes to become user-centered, they should be designed for easy and convenient use.

Many studies on smart homes have focused on technologies that can be applied to provide more intelligent services. Studies have been conducted on recognizing users’ movements, locations, and behavioral patterns using sensors and cameras [9,10,11,12,13], or using voice control or remote control for smart homes based on the IoT and cloud computing [14,15,16,17]. Research has also been conducted on methods for predicting and providing services to users by collecting information about context, using methods such as machine learning [18,19,20,21]. Furthermore, a number of studies have proposed energy use management systems to increase energy efficiency in smart homes [22,23,24,25,26] or to provide healthcare services remotely by collecting and monitoring residents’ movements and physiological signals [27,28,29,30,31]. As discussed above, it is important to examine how various technologies can be applied to provide more advanced smart home services, but if smart homes are to be user-centered, it is also necessary to study user-centered rather than technology-based approaches [32].

Therefore, this study examines the smart home based on the concept of affordance, which is highly meaningful in user-centered design. James J. Gibson coined the term affordance, referring to the relationship between an object and an actor, specifically the actor’s possible actions on the object [33]. After Gibson presented the affordance concept, Donald Norman applied it to the field of design, and it is now considered important for enhancing the usability of objects and promoting interaction between objects and users in human–computer interaction (HCI) and user experience (UX) [34,35,36,37]. In the HCI field, most research has focused on usability evaluation and design guidelines for websites, smartphones, and mobile applications (apps). In comparison, few studies have been conducted on the usability of smart homes, and user evaluation studies on the user interfaces of smart homes [7,38,39] or installation of smart home devices [40] have been carried out. We have not had any study suggesting method to promote user’s action in terms of cognitive, physical, functional, and sensory aspects by applying the concept of affordance to improve the usability of smart homes. User-centered smart homes should be designed so that users can operate smart home systems without difficulty and obtain the desired services conveniently; for this purpose, guidelines for improving the usability of smart homes are also required. Smart homes are evolving into spaces that can automatically provide a user’s desired service without user intervention, but at the current stage of technology, the user must input commands to operate smart home devices. Thus, to improve the usability of smart homes, it is essential to improve the usability of the active devices (e.g., control devices and smart appliances) and user interfaces with which the user directly interacts in the command input and output process. To this end, this study aims to present guidelines on how to apply affordances to improve the usability of active devices and user interfaces in smart homes. As a first stage to present guidelines for the application of affordances to smart homes, this study presents the guidelines not for specific users with unique characteristics (e.g., elderly or disabled individuals), but for general users. To do this, two methods were used. For the first, the affordance factors that could be applied to improve the usability of active devices and user interfaces were extracted from previous studies, and for the second, a case study was conducted to analyze the application of affordance factors, focusing on active devices and user interfaces. Guidelines on the application of affordance factors were then composed, by combining case analysis results with relevant guidelines.

The remainder of this paper is organized as follows. Section 2 discusses smart home services and the interaction between users and smart home components. In Section 3, the concepts of affordance in general and affordance in smart homes are discussed. Affordance factors for improving the usability of active devices and user interfaces are then derived. In Section 4, the application status of the affordance factors is analyzed, focusing on active devices and user interfaces. Section 5 summarizes the results of the case analysis and proposes guidelines for the application of affordance factors to improve the usability of smart homes. Section 6 presents the conclusion.

2. Smart Homes

2.1. Smart Home Services

A smart home can be defined as a residence that improves quality of life by providing services that respond to the needs of its residents based on network technology linking sensors, home appliances, and devices that can be accessed, controlled, and monitored remotely [3,41,42]. Diverse networked sensors, devices, and appliances automate residential spaces and support the remote control of home appliances, thereby reducing the burden imposed by household tasks [43] and making it easier and more convenient to carry out everyday household activities. In addition, the application of the IoT and AI to smart homes enables capabilities that were impossible before the application of smart technology [32].

Smart homes provide various kinds of services using intelligent information technology devices to enhance users’ convenience and welfare. Aldrich [44] stated that smart homes should enhance residents’ comfort, convenience, security, and entertainment by using ICTs that connect inside and outside residential spaces. Lee et al. [45] classified smart home services as security, environmental control, energy saving, entertainment, healthcare, and convenience. Security includes visitor monitoring, gas leak detection, and emergency calls; environmental control includes temperature control, ventilation systems, and lighting control; and energy saving includes standby power blocking and water-saving systems. Entertainment includes smart TV and smart audio/video; healthcare includes health management and medical examination; and convenience provides remote control and scheduling. Balta-Ozkan et al. [41] classified smart home services as safety, life support, and energy management. Safety services are provided to cope with crime and negligent accidents, while life support services provide a pleasant indoor environment by conveniently controlling lighting, heating, and ventilation; energy management refers to a service that enables energy saving by efficiently controlling and monitoring home appliances and devices. In addition, Kim et al. [46] divided smart home services into four areas: security, convenience, entertainment, and healthcare. Of these services, convenience includes energy management for managing power and water, environmental services for controlling temperature and lighting in the home, and appliance control services for the remote control of home appliances and devices. As such, smart homes provide services related to safety, convenience, comfort, enjoyment, healthcare, and energy management using many kinds of sensors, devices, and user interfaces to improve residents’ quality of life.

2.2. Interaction between Users and Smart Home Components

Diverse smart devices constituting a smart home are connected via networks, and each device provides services through communication between them. The components that make up the smart home are sensors, actuators, controllers, home gateway, network infrastructure, smart appliances, user interfaces, etc. [47]. These components provide users with services related to safety, convenience, comfort, enjoyment, healthcare, and energy management, and users interact directly or indirectly with them. The interaction between smart home components and users can be divided into two types.

The first type of interaction occurs when data on users and surroundings are collected through sensors. Unnoticed by users, necessary services are then provided to users based on these collected data (see Figure 1). In other words, in this type of interaction, there is no direct input instruction for user commands, but the service is provided as output so that passive interaction occurs between the user and smart home control system [48]. For example, if the occupancy sensor determines that there is no user inside, the system automatically turns off lights or cuts off standby power. CO2 sensors measure indoor air quality and automatically trigger ventilation when the CO2 concentration is high to provide a pleasant indoor environment.

The second type of interaction provides the user with services if the user inputs a command in the smart home control system. That is, the interaction between the user and smart home system occurs through the process of input by the user and output according to the user’s manipulation. As shown in Figure 2, users enter commands using control devices (e.g., switches, wall pads, smartphones, tablet PCs, and smart speakers) as well as control panels on smart appliances. As such, devices with which the user directly interacts in the smart home are called active devices [48], and active interaction occurs between the user and active devices in the process of using the services. In addition, user interfaces act as intermediaries to enable communication and interaction between active devices and users [47,49]. Active devices are applied with a physical user interface, a graphical user interface, a voice user interface, and a gesture user interface [38,50], which enables users to interact in various ways. For example, if users wish to know the expiration date of food stored in a smart refrigerator, they can tap the menu on the refrigerator’s built-in display to show up the expiration date of the food. Users can also adjust the brightness or color of the light and room temperature by using buttons on wall pads, or by giving voice commands to smart speakers.

3. Affordances

3.1. Concept of Affordance

Gibson defined affordance as the possibility for action that the environment provides to the actor in the relationship between the environment and the actor [33]. Actors perceive the various types of affordances that the environment provides, and based on these affordances, they determine their action in the environment. In other words, an affordance is a potential feature of objects that allows actors to use objects in any way in the environment.

Norman decomposed the affordance concept into real and perceived affordances [35]. Real affordances refer to the physical characteristics of an object that help a user perform an action, while perceived affordances provide external clues that help a user recognize an object and determine its action. In particular, Norman emphasized the importance of providing perceived affordances in design, and presented seven design principles that enable users to complete tasks easily and simply [51]. He stated that it is necessary to convey information using clear and concise text to make the system easier to understand and use and that concise design is necessary to minimize unnecessary information and choices. He also suggested that feedback should be provided so that users can recognize the progress and results of work as well as designing icons and graphics as metaphors so that the method and result of operation can be predicted. It is also important to provide constraints to prevent user misbehavior. Error messages and ‘undo’ functions should be provided to help users respond to errors, and the design should be such that users can solve problems based on the information they have already acquired. McGreener and Ho [36] argued that when trying to improve design by applying affordances, two axes should be maximized: ‘easy-to-undertake affordance’ and ‘clear information’. ‘Easy-to-undertake affordance’ increases the convenience of a task, for example, by increasing accessibility to frequently used functions and using multiple keys to reduce the number of clicks; ‘clear information’ involves the provision of information specifying an affordance.

In addition, H. Rex Hartson decomposed the affordance concept into cognitive, physical, functional, and sensory so that the affordance concept could be more effectively applied to interaction design [37]. Cognitive affordances refer to design features that help users recognize and think about something, while physical affordances refer to design features that help users physically act. For example, a label with a clear meaning enhances cognitive affordance by enabling prediction of the function of the button, and a button large enough to allow the user to click accurately promotes physical affordance. Functional affordances are design features that help users accomplish tasks, meaning the functional usefulness of the system to allow the user to accomplish the task effectively. Finally, sensory affordances refer to design features that help users sense something and that have characteristics that assist in cognitive and physical affordance. For example, an easily readable text size on a label can facilitate sensory affordance. Hartson emphasizes the role of affordances in design and asserts that affordances facilitate the user’s perception, understanding, and use of something.

3.2. Affordances in a Smart Home

Users interact directly or indirectly with active devices or passive devices in the process of using a smart home system. In particular, users interact directly with active devices such as control devices, smart appliances, and the user interfaces applied to them, by inputting commands and receiving output for the commands. Thus, the affordances that active devices provide can directly affect the usability of smart homes and user satisfaction. Active devices in smart homes have a variety of user interfaces, such as physical, graphical, voice, and gesture user interfaces. Therefore, to improve the usability of smart homes, active devices and their user interfaces should provide appropriate affordance in terms of physical, visual, auditory, and tactile aspects.

Hartson’s four types of affordances [37], cognitive, physical, functional, and sensory, which are discussed in Section 3.1, can be applied to promote various forms of interaction that users experience in smart homes. As Hartson argues, applying his concept of affordance to smart homes can help users manipulate smart devices more easily and obtain the desired services, thereby leading to a positive user experience. Therefore, based on Harton’s four types of affordances [37], this study derived the affordance factors by type of affordances to improve the usability of active devices and user interfaces with which users directly interact. To derive the affordance factors, we referred not only to Hartson’s research but also to the application method of affordances for user-centered design proposed by McGreener and Ho [36] and Norman [51], which are also discussed in Section 3.1. We also examined research that presented design principles for improving user-interface usability [52,53,54]. Nielsen [52] presented 10 usability heuristics for evaluating the usability of user interface design, and Nielsen’s heuristics have been the basis for many studies on the usability of user interface design. Mandel [53] suggested the three golden rules of user interface design and details for each rule to provide users with a positive user experience. Blair-Early and Zender [54] presented 10 user interface design principles for effective interaction design. Haines et al. [7] proposed the usability principles of the human–computer interface, emphasizing that user interfaces in smart homes should be user-friendly, intuitive, and easy to use.

The previous studies mentioned above presented components or design principles to be considered for improving the usability of computerized devices and software based on the concept of affordance [36,37,51], or suggested design principles that enhance the usability of graphical user interfaces for effective interaction between humans and computers [7,52,53,54]. To derive affordance factors applied to active devices and user interfaces in smart homes, this study classified the items of design principles suggested in the previous studies into four categories based on Harton’s four types of affordances [37], and then reorganized them into 16 affordance factors by grouping items of similar concepts. Figure 3 shows the design principles of previous studies grouped together and the affordance factors derived from each group. The affordance factors consist of a total of 16 items, four for each affordance type.

Cognitive affordances in smart homes involve design features that provide visual clues or information to enable prediction of how a task is performed or the results. Users should be able to intuitively understand the current state of smart systems in smart homes and how to operate such systems to control the indoor environment [55]. Hence, a simple design that eliminates unnecessary complexity is applied, and buttons or menus on the control devices and user interfaces use predictable names for functions. The design also helps users understand what functions are provided and how to perform tasks by using easy-to-understand icons and texts. The concept of physical affordances refers to a design that helps users perform physical actions so that they can perform tasks easily. It is recommended that active devices support multimodal interfaces so that they can be selected according to the user’s convenience, and the operation method of user interfaces should enable users to perform tasks with simple actions. In addition, the design of active devices should be convenient to operate in terms of size, shape, and location and should minimize repetitive actions by users when performing tasks. The concept of functional affordances refers to a design that helps users achieve the desired results effectively. Therefore, the design increases accessibility to frequently used functions and provides customized settings to help users effectively accomplish the desired tasks. It is also important to provide feedback on the results of user manipulation and to remove or disable elements that can cause mistakes or risks. The concept of sensory affordances refers to a design that helps or promotes the action of users to see, hear, and feel something. The text, buttons, and icons of the user interface should be clearly distinguishable and noticeable from the background and highlight elements that require attention to prevent risks and user mistakes. Additionally, services should be provided in an appropriate form by using visual, auditory, or tactile elements.

4. Case Study

4.1. Research Methodology

The objective of this study is to present guidelines for improving the usability of smart homes, focusing on active devices and user interfaces that interact directly with users. To present guidelines, an analytical framework was derived through literature review, and case study was conducted.

At the literature review stage, based on a review of previous studies on affordance, usability, and user interfaces, we derived affordance factors to improve the usability of active devices and user interfaces. To derive affordance factors, each item of the design principles of affordance and user interface was classified into four categories based on the concept of cognitive, physical, functional, and sensory affordances presented by Hartson [37]. Then, items of similar content were grouped together to derive the subdivided affordance factors for each category. As a result of deriving an affordance factors summarizing the characteristics of the items of each group, a total of 16 affordance factors, four for each category, were derived (see Figure 3).

Next, we conducted a case study using the derived 16 affordance factors as an analytical framework. The case study was conducted on four cases in South Korea where diverse types of active devices were installed in smart homes. The application status of the affordance factors was analyzed for active devices and user interfaces installed by each case.

Lastly, guidelines on the application of affordance factors to improve the usability of active devices and user interface were presented by combining case analysis results with relevant guidelines. Flow chart of research methodology are shown in Figure 4.

4.2. Overview of the Cases

In this study, a total of four cases in South Korea were selected for investigation, including one exhibition hall and three sample houses with smart technology. An overview of the cases is shown in

Table 1. Overview of the cases to be investigated.

Category	S Exhibition Hall	P Model House	L Model House	H Model House
Location	Jagok-dong, Gangnam-gu, Seoul	Daechi-dong, Gangnam-gu, Seoul	Dongcheon-dong, Gyeonggi-do, Yongin	Osan-dong, Gyeonggi-do, Hwaseong
Spatial configuration	Kitchen, living room, bedroom, dressing room, balcony	Entrance, living room, kitchen/dining room, bedroom, bathroom, powder room, balcony	Entrance, living room, kitchen/dining room, bedroom, bathroom, dressing room, balcony	Entrance, living room, kitchen/dining room, bedroom, bathroom, balcony
Image

. The ‘S exhibition hall’ was established to promote and provide an experience of smart homes. It was set up like a residential space, and various smart devices and smart home appliances were installed to experience the changes in living life with smart technology. The ‘P model house’, the ’L model house’, and the ‘H model house’ are model houses in a complex that incorporate ICT technology into residential spaces for convenient and safe living. Each model house uses home IoT system. In each case, we analyzed how the affordance factors were applied to active devices and their user interfaces by dividing them into cognitive, physical, functional, and sensory affordances.

4.3. Case Analysis Results

4.3.1. S Exhibition Hall

The ‘S exhibition hall’ is a space for smart home experience, and includes various control devices such as wall pad, tablet PC, and smart speaker, as well as smart appliances such as smart door lock, smart mirror, smart refrigerator, smart range hood, and smart TV. In terms of cognitive affordances, the mobile app for home appliance control uses icons that depict images of each home appliance as well as text labels with the name of the given appliance. It also provides information on the registered appliances so that users can check the current state of each at a glance with a tablet PC (see Figure 5). The menu names of the wall pad use predictable names for functions, such as energy information, vehicle arrival, and CCTV. In the case of the energy information menu, information on the current month’s estimated electricity bill and a comparison with last month’s rate is provided to help users understand changes in electricity consumption. In terms of physical affordances, the smart door lock supports facial recognition and has a card key to accommodate the user’s preferred method (see Figure 5). In the ‘S exhibition hall’, the 3D motion sensor and smart speaker are connected to the home appliances, so that they can be controlled by the user’s motion and voice. However, the command recognition rate is poor, making it necessary to repeat commands several times before the home appliance recognizes them. Home appliances can be controlled via a mobile app by tapping the control button on the list of registered devices, and the control button is sufficiently large to be pressed with a finger without inconvenience.

In terms of functional affordances, if a user saves the desired mode (e.g., out/return/sleep/wake up mode) in the mobile app, s/he can conveniently control the appliance by tapping the icon for each mode on a mobile device. For the gas circuit breaker, visual and audible feedback is provided when the valve is locked, and users can set the gas valve locking time using the mobile app. The smart range hood connected to the sensor senses odors and discharges them outdoors. When the hood function is selected, the corresponding icon lights up, and feedback is provided. Each menu on the wall pad also provides feedback that changes the color of the menu name so that the selected menu is easily noticeable when the user selects it. Additionally, users can set the energy usage goal value on the wall pad for customized energy management and monitor the usage compared to the goal value. In terms of sensory affordances, the menu names on the wall pad’s touchscreen use Gothic bold font to increase readability, but the button names on the wall pad frame (e.g., emergency, call, and door open) are less readable because the text is small, narrow, and the similar color as the background. The energy information function of the wall pad provides energy usage both graphically and numerically, allowing users to compare the amount of energy used at a glance (see Figure 5). Additionally, the smart mirror uses motion recognition and augmented reality technology to match the attire selected by the user to the user’s image in real time (see Figure 5). The smart refrigerator displays text and image information on the LED display about the management of ingredients as well as recipes.

4.3.2. P model House

The wall pad is installed in the living room, enabling lighting control, standby power cut-off, temperature control, gas valve shut-off, security setting, and energy monitoring. In addition, a smart switch is installed in the hallway for lighting and gas valve control, elevator call, and parking location check. Each room is also equipped with a switch for temperature and lighting control and standby power cut-off, and a speakerphone is installed in the bathroom. In terms of cognitive affordances, the menu names are displayed on the wall pad with predictable names for functions, such as cooling, heating, scheduling, and timing. The menu named Energy on the wall pad is categorized by item (e.g., electricity, water, gas, and heating) and is configured to monitor the energy consumption for each item by month and year. The smart switch in the hallway uses familiar icons reminiscent of their associated functions, such as light bulbs, gas valves, and padlocks (see Figure 6). In terms of physical affordances, the door lock supports fingerprint recognition, password input, and a card key, depending on the user’s preferred method. In addition to the wall pad in the living room, each room is equipped with a touch-based smart switch with lighting and heating control and standby power cut-off, enabling convenient indoor environment control. By tapping the brightness control button on the smart switch, users can incrementally adjust the brightness of the light (see Figure 6), and can also control the outlet individually or collectively by tapping the standby power cut-off icon (see Figure 6). The home network systems are also connected to mobile apps, enabling remote control of heating, gas, and lighting.

In terms of functional affordances, an emergency call function is provided on the living room wall pad and bathroom speakerphone to facilitate response to an accident. Frequently used functions such as door open, call, and emergency call are not only provided as menus on the wall pad but also have shortcut buttons to increase accessibility. In the case of the lighting control function of the smart switch, when a user inputs and saves a frequently used level of brightness, it is possible to realize the stored brightness with one touch. For personalized energy management, users can check the energy usage of the current month, previous month, and previous year on the wall pad, compare it with the average usage in other households with the same area, set energy usage goal values, and set notifications according to usage. In terms of sensory affordances, when the parking location check function is executed, the parking location of vehicles is indicated by a red dot on the drawing, and the parking area name is indicated by text to increase visibility (see Figure 6). The room temperature and power consumption on the smart switch are displayed in white on a black display, and the font size is also large, improving readability. Energy usage is provided graphically and numerically on the wall pad to help users understand changes in usage, and the energy usage goal value is marked with an attention-drawing color. In addition, in the bathroom speakerphone, the icons for emergency call and guidance light for late at night are illuminated to increase attention.

4.3.3. L Model House

The ‘L model house’ also has a wall pad installed in the living room, and a switch is installed in the hallway for lighting control, elevator call, energy monitoring, and parking location verification. In addition, each room is equipped with switches for standby power cut-off, temperature control, and ventilation, and the switch in the living room includes these functions as well as emergency call. The bathroom is equipped with a speakerphone that provides door open, call, and emergency call functions. In terms of cognitive affordances, the temperature and ventilation control switch facilitate recognition of the method of operation by applying a simple design consisting only of the necessary buttons and displays. The bathroom speakerphone uses icons with text labels to help users recognize the function of each button. However, the buttons on the standby power cut-off switch are labeled ‘Outlet 1’, ‘Outlet 2’, etc., and it is difficult to recognize the position of the outlet connected to each button. In the case of icons, simple icons are displayed on the wall pad along with text labels that provide information on functions such as door open, parking location verification, and elevator call. In terms of physical affordances, the door lock has a built-in proximity sensor that automatically detects and captures suspicious persons and allows users to check the status through the wall pad. A switch installed in the hallway allows users to turn off all indoor lighting with one touch (see Figure 7), and can also remotely control heating, gas, and lighting using mobile apps. Each room is equipped with a switch for temperature and ventilation control, which can be set to the desired level by pressing the dial or turning it to the left or right by applying the jog dial (see Figure 7).

In terms of functional affordances, if the user saves the desired mode (e.g., out/return mode) in the mobile application and selects the desired mode, the selected mode is executed and the push alarm is sent to the user’s smartphone. When a user presses the emergency button on the bathroom speakerphone, immediate feedback is provided by alerting the security guard and the wall pad. In addition, the standby power cut-off switch installed in each room lights up only the selected button, making it easy to see whether the button has been selected or not. The switches to control temperature adopt tact switches to reduce misoperation by users unfamiliar with the touch screen, and they make a click sound when the switches are pressed. In terms of sensory affordances, among the various kinds of buttons on the switch in the living room, only the emergency call button is red, thereby increasing its visibility. Additionally, of the several buttons on the standby power cut-off switch, the entire outlet control switch is in a different color from the individual outlet control switches, making it stand out visually (see Figure 7). When the temperature is adjusted with the jog dial, the set temperature is displayed as text with high readability. However, for the bathroom speakerphone, the color of the icon is similar to the background color, and the size of the text label is also small, making it less visible (see Figure 7).

4.3.4. H Model House

The living room is equipped with a wall pad, and the hallway is equipped with a switch for lighting and gas control, and elevator call. An integrated controller is installed in each room for standby power cut-off, lighting control, and temperature control. In addition, the home network system can be linked with mobile apps, enabling remote control of the indoor environment using the apps. In terms of cognitive affordances, the temperature control buttons on the integrated controller used concise text labels (e.g., temperature up/down, heating schedule) describing the function of each button (see Figure 8). However, the light control buttons on the integrated controller are labeled ‘light1’, ‘light2’, and it is difficult to recognize the position of the light connected to each button. In the case of the menu names on the wall pad, the emergency and call buttons use universally accepted icons (e.g., alarm lamp and telephone receiver) to enable understanding of the functions of each button without text labels. However, other menus use abstract icons and the menu names are not concrete (e.g., control, addition, and information), making it difficult to understand their respective functions. In addition, the menu name of the energy monitoring function is marked as HEMS; thus, it is difficult to understand the function solely by the menu name. In terms of physical affordances, the wall pad is equipped with push buttons and a touch screen (see Figure 8). Lighting control and temperature control are not only included in the control menu of the touch screen but can also be controlled using push buttons, which enhances the convenience of operation. The buttons on the integrated controller are sufficiently large to be pressed without inconvenience, and the temperature can be adjusted incrementally by pressing a button. Additionally, lights and outlets in each room can be controlled individually or collectively using the buttons on the integrated controller (see Figure 8). The switch installed in the hallway also makes it possible to turn off all lights conveniently with one touch (see Figure 8). Bathroom LED sensor lights detect user movements, and the light is softly illuminated without switch operation.

In terms of functional affordances, the energy management function of the wall pad enables personalized energy management by setting the electricity usage goal value and monitoring the amount of electricity, water, and gas used and the amount of CO₂ generated for each type of energy usage. In addition, the push buttons on the wall pad and the buttons on the switches and integrated controllers help to recognize operation results by lighting up when pressed. To minimize risks and prevent negligent accidents, when the gas valve is controlled by the mobile application, the gas valve opening function is not available. Additionally, the time delay switch turns off the lights after a few seconds, to consider the safety of the user when moving. In terms of sensory affordances, among the various types of buttons on the wall pad, only the emergency button is red, which increases its visibility. The security function of the wall pad includes the fire/gas/crime alarm function, which not only sounds an alarm when an abnormal situation occurs but also informs the user of the situation by text message. In addition, if users press the elevator call button when they go out, an arrival reminder is provided by voice message, and the location of the vehicle is indicated on the display when the user inquires about the parking location.

5. Discussion

In this section, the results of the case studies are summarized, and guidelines are then proposed for applying the affordance factors to active devices and user interfaces to improve the usability of smart homes (see Appendix A).

Table 2. A synthesis of the application status of affordance factors.

		Application Status of Affordance Factors	S	P	L	H
CA	CA1	Simple appearance consisting of buttons of main functions	✓	✓	✓	✓
		Grouping of related contents	✓	✓	✓	✓
	CA2	Button names and menu names using familiar terms	✓	✓	✓
		Button names and menu names containing information about the function	✓	✓	✓	✓
	CA3	Use of familiar terms in providing information	✓	✓	✓	✓
		Key information provided concisely	✓	✓	✓	✓
	CA4	Use of familiar icons reminiscent of functions	✓	✓	✓	✓
		Use of icons with text labels	✓	✓	✓	✓
PA	PA1	Indoor environment control using fixed and mobile devices	✓	✓	✓	✓
		Application of multimodal interfaces	✓	✓		✓
	PA2	Control of devices via simple touch gesture	✓	✓	✓	✓
		Application of consistent operation method	✓	✓	✓	✓
	PA3	Touchscreen wall pad installation in the living room	✓	✓	✓	✓
		Wall mounted switch installation in each room		✓	✓	✓
		Speakerphone installation in bathroom		✓	✓
		Appropriate button spacing for operation	✓	✓	✓	✓
		Appropriate button size for operation	✓	✓	✓	✓
	PA4	Providing buttons to move to main menu on the wall pads	✓	✓	✓	✓
		Providing ‘All Off’ function on the wall pads or control switches		✓	✓	✓
FA	FA1	Providing shortcut buttons on the wall pad for frequently used functions	✓	✓	✓	✓
		Installation of switches for functions required when going out near the front door		✓	✓	✓
	FA2	Providing various mode setting functions	✓		✓	✓
		Providing user’s preferred indoor environment setting function		✓	✓	✓
		Providing personalized energy management service	✓	✓	✓	✓
	FA3	Providing immediate feedback on user’s action	✓	✓	✓	✓
		Providing multiple feedback	✓	✓	✓	✓
	FA4	Disabling a function that can cause risk	✓	✓	✓	✓
		Providing ‘Undo’ or ‘Cancel’ functions for the user’s action	✓	✓	✓	✓
		Providing help		✓	✓
SA	SA1	Use of easy-to-read size text	✓	✓
		Use of highly readable fonts	✓	✓	✓	✓
	SA2	Clear contrast between text color and background color	✓	✓
		Clear contrast between button color and background color		✓
	SA3	Emphasis on important functions or information using color	✓	✓	✓	✓
		Emphasis on important information by differentiating thickness or size of font	✓	✓	✓	✓
	SA4	Providing information using graphic elements	✓	✓	✓	✓
		Notification of situation using appropriate medium in case of danger	✓	✓		✓

Note: CA: cognitive affordances; PA: physical affordances; FA: functional affordances; and SA: sensory affordances.

shows the results of analyzing the application of affordance factors for active devices and their user interfaces by each case. Based on Table 2, the results of applying the four types of affordances are summarized as follows. Firstly, to provide cognitive affordances, active devices and user interfaces were designed simply for users to recognize main functions at a glance; and familiar icons, concise text labels, and easy terms were used to help users easily understand the functions and information. Secondly, to provide physical affordances, active devices provided easy-accessible environment, mobile applications control for remote access, and multimodal interfaces. To avoid inconvenience, appropriate button size and button spacing were implemented, and simple and consistent operation method was applied for improving learnability. Thirdly, to provide functional affordances, shortcut buttons and user-customized settings were provided for improving the efficiency of the task, and helped users complete tasks efficiently by providing immediate and multiple feedback on user’s operation, as well as ‘Undo’ or ‘Cancel’ functions, and ‘Help’. Lastly, to provide sensory affordances, the color/size of the text and buttons of active devices and user interfaces were designed to be distinct from the background, and appropriate visual, auditory, and tactile medium were used for effective delivery of important information or risk alert to users.

5.1. Cognitive Affordances

Cognitive affordances refer to design features that provide visual clues to help manipulate control devices and user interfaces. Most control devices have been designed with a simple appearance consisting of displays and buttons for main functions, or only buttons. However, some switches have more than 10 icons and text buttons on the display, causing complexity. For devices that provide various functions such as wall pads, the complexity is eliminated by including the submenus in the relevant top menus. Additionally, most buttons and menus use simple names (e.g., security, emergency, gas, and lighting) that allow their functions to be predicted. However, in the case of the ‘H model house’, an abbreviation is used in the menu name of the energy management function, making it difficult for the user to understand the function intuitively. According to the study of Chi et al. [56], there is no systematic relationship between words and their abbreviations, which makes the intuitive understanding of function difficult, so users have less preference for abbreviated menu names. Therefore, it is recommended to avoid using abbreviations in menu names. The button names for lighting and standby power cut-off (e.g., outlet 1, outlet 2, lighting 1, and lighting 2) fail to specify the target controlled by each button. Therefore, the button names should allow the user to intuitively recognize the position of the target controlled by each button. Wall pads or switches provide brief information on each menu or condition of the indoor environment, and it is important to briefly provide the necessary information using easy terms for intuitive understanding.

In addition, wall pads installed in the four cases use icons with text labels. Most of the icons symbolically represent their functions (e.g., security is symbolized with a padlock, emergency with an alarm lamp, and lighting with a light bulb), but some apply designs that make it difficult to understand the functions without text labels. In the case of switch buttons, familiar icon buttons reminiscent of functions without text labels are used (P model house), or familiar icons and appropriate text labels are applied (L model house). However, in some cases, it is difficult to predict the functions without text labels due to non-intuitive icons (H model house). According to a study by Hannah [57], for labeled icons, 88% of users were able to accurately predict functions. However, for common unlabeled icons, 60% of users predicted functions, and for unique icons without labels, only 34% of users were able to predict functions. Therefore, by including text labels that clarify the meaning of icons in universal icons, the user can predict the function before selecting the icon. Guidelines for improving cognitive affordances are as follows:

• Apply a simple design consisting of buttons and displays of main functions;
• Hide infrequently used functions;
• Organize menus hierarchically;
• Group related contents together;
• Use familiar and concise button names without abbreviations or jargon;
• Provide key information concisely using familiar terms;
• Provide information that matches the menu names;
• Use button names that enable prediction of the function;
• Use button names that enable clear recognition of the operation target;
• Use universal icons with text labels on buttons.

5.2. Physical Affordances

Physical affordances are features that promote physically doing something using active devices and user interfaces, and include control methods and physical designs that enable convenient manipulation. All four cases enable convenient indoor environment control using mobile apps connected to home network systems as well as fixed control devices such as wall pads and wall-mounted switches. Additionally, touch screen wall pads are installed in all cases, while wall pads with touch screens and push buttons are installed in the ‘H model house’. For switches, touch screen switches (P model house), tact switches (L model house), and push button switches (H model house) are installed. In most cases, one type of user interface is used for wall pads and switches. However, when multimodal user interfaces are applied (e.g., touchscreen wall pads with voice recognition support), the user can select an input and output method, thus increasing convenience. In addition, the ‘S exhibition hall’ is equipped with a smart speaker and a 3D motion sensor to control the home appliances with voice and gestures, but improvement is needed due to poor recognition rate. Voice user interfaces can improve the recognition rate by applying noise reduction and acoustic echo cancelation [58], as voice commands may not be recognized well in noisy surroundings. Gesture user interfaces may be less intuitive because the user must learn gestures for device control [59], and recognition errors may occur depending on the user’s location or surroundings [60]. Therefore, gestures that can be learned naturally are applied, and a gesture user interface capable of quick and accurate command recognition in various situations is required.

Touch user interfaces, which are applied to most control devices, enable users to perform desired functions through simple touch gestures such as tapping and swiping. The push buttons or dial buttons of the ‘H model house’ and the ‘L model house’ are designed to help the user understand how to operate them (e.g., using up and down arrow buttons). In all cases, a 10-inch touchscreen wall pad is installed in the living room, which is the most preferred location for environmental control and living activity monitoring devices [61], and wall mounted switches are installed in each room. A bathroom speakerphone is installed next to the toilet bowl of the ‘P model house’ and the ‘L model house’. The wall pad should be installed at a height of approximately 1400 mm, the user’s eye level, so that the user can easily check the screen [45]. In all cases, fixed wall pads are installed, but angle-adjusted wall pads may be more convenient for elderly individuals and wheelchair users [62]. The wall mounted switches should be installed at a height of 750–1200 mm [63], and the bathroom speakerphone is easy to reach when installed at a height of 800 mm from the bottom of the wall around the toilet bowl or bathtub [62]. In addition, the size and spacing of buttons in touch user interfaces and physical user interfaces should be designed for convenient operation, and touch targets should be at least 10 mm × 10 mm in size and spaced at 2 mm to prevent accidental mis-taps [64,65,66]. To reduce the user’s task steps and minimize repetitive actions, all wall pads provide buttons that can be moved to the main menu. Additionally, wall pad and wall-mounted switches provide the ‘All Off’ function for the user to control the lighting, heating, and cooling with one touch. Guidelines for improving physical affordances are as follows:

• Apply a multimodal interface to vary the input and output methods;
• Allow control of the device with a few simple actions;
• Apply easy-to-learn and easy-to-remember device operating methods;
• Keep the interface consistent so that the same interactions lead to the same results;
• Provide the same menu item in the same location on every screen of the wall pad;
• Install the control devices in each room to enable environmental control and monitoring;
• Install the control devices at a height that is convenient for operation (e.g., it is recommended to install the wall pads approximately 1400 mm from the floor, the switches 750–1200 mm from the floor, and the bathroom speakerphone approximately 800 mm from the floor);
• In the touch interface, touch targets should be at least 10 mm by 10 mm in size, with 2 mm gap between targets to prevent mis-taps;
• Provide a ‘Home’ button on the wall pad to reduce work steps;
• Provide an ‘All Off’ function to control the indoor environment with one touch.

5.3. Functional Affordances

Functional affordances refer to design features that help users accomplish tasks efficiently and achieve the desired results. All wall pads enable access using a shortcut button in addition to access through the main menu to improve accessibility to frequently used functions (e.g., call, door open) and emergency call functions. The three model houses are equipped with switches that support turning off all lights, shutting off the gas, calling an elevator, and checking the parking location near the front door. This allows the user to conveniently control the indoor environment and reduce waiting time when going out. In addition, the mobile apps allow the user to set various modes (e.g., going out, returning home, and sleeping) according to the life pattern, and to control the indoor environment easily and quickly for each mode. It is also possible to control the room temperature and humidity via wall pads, switches, and mobile apps, and in some cases, the user can set their preferred illumination. All cases provide a customized energy management function as well as control of the indoor environment according to the user’s lifestyle.

When the user sets the energy usage goal, they can check the energy consumption compared to their goal value and average usage in other households of the same area by using the wall pad or mobile app. Setting goals and comparing with other households, not only enables personalized energy management but also helps motivate users to conserve energy [67]. When operating the control devices, immediate feedback is provided, through, for example, a change in the color of the selected menu name, icon, or button, or a light coming on. In the case of temperature and illuminance control, the result of operation can be confirmed instantly by the change in numerical value. Some functions provide feedback that informs of the result of the operation in text, or through changes in button color. In case of incorrect operation, informative feedback is also provided to show the solution. All cases provide immediate feedback on the user’s operation, but mainly in the form of visual feedback. Safety-related functions (e.g., gas valve shut-off, security) or functions that take time to complete a task (e.g., elevator call) provide both visual as well as auditory feedback such as sound or voice messages, enabling users to recognize the results more clearly and easily. To minimize error or risk, all cases provide only the shut-off function for the gas valve to prevent accidents arising from negligence when controlling the gas valve using the wall pad or mobile app. In addition, the ‘H model house’ includes a delayed ‘off’ function in the lighting control switch, considering the safety of the user when extinguished, and the ‘P model house’ provides a night safety light function to prevent bathroom safety accidents. In the case of incorrect operation, the previous button allows the user to return to the previous stage, but only some cases provide help on the wall pad or mobile app. Guidelines for improving functional affordances are as follows:

• Increase accessibility by providing shortcut buttons for frequently used or emergency functions;
• Allow users to set customized modes and easily control the indoor environment for each mode;
• Allow users to customize the interface for their preference (e.g., screen brightness, volume, font, and font size adjustment);
• Allow users to store and easily implement their preferred indoor environmental conditions;
• For energy management functions, enable customized energy management by self-monitoring, setting goals, and comparing usage with neighbors;
• Provide feedback that clearly indicates the results of actions for every manipulation;
• Provide visual feedback that clearly indicates the selected function;
• Provide informative feedback that provides information on how to resolve an error;
• For safety-related functions and functions that take time to complete, provide multiple types of feedback (e.g., visual, auditory, and tactile feedback) to ensure that the feedback is clearly recognized;
• Provide ‘Back’ and ‘Cancel’ functions to easily reverse actions;
• Provide help where it is easily accessible.

5.4. Sensory Affordances

Sensory affordances are design features related to the noticeability and discernibility of the user interface applied to active devices. Providing appropriate sensory affordances enhances usability by assisting with cognitive and physical affordances. The text of the user interface should be legible in terms of text font, size, and color, and layout. Decorative fonts should be avoided and no more than three fonts are used on one screen [7]. In the user interface, text is mainly used for menu names and information. In most cases, the text size is appropriate, and the font is familiar and highly readable Gothic. However, the text labels of some buttons are less legible because the text is small or the colors are not contrasted significantly with the background color. To make it easier for everyone, including low vision and color-blind users, to distinguish text against a background, the brightness contrast between the text and background should be at least 4.5:1, but a brightness of 3:1 can be used when the text font is 18 pt or larger or when a bold font of 14 pt or larger is used [68]. To ensure visibility in the user interface, the contents should be easily noticeable, with sufficient color contrast between the contents and background and at most four different colors on each screen [7]. Icons and buttons should be designed to contrast clearly with the background color, enabling the user to clearly recognize the tappable areas. Red and green, blue and orange are color combinations that are difficult for color-blind users to perceive, and should be avoided if no other clues are provided [69].

Regarding important functions or information, the ‘H model house’ has increased noticeability of emergency buttons by applying a red button on a gray background, while the ‘P model house’ uses red text on the smart switch to highlight help for incorrect operation. In most cases, information on the current indoor conditions (e.g., temperature, power consumption), frequently-checked information such as date and time, and menu names are emphasized using bold text or larger text. Additionally, smart home services should be provided through an appropriate presentation medium (e.g., text, graphic, sound, voice message, and vibration) so that the contents of the service can be effectively delivered to the user. For the energy management function, presenting usage trends through visualization of energy usage is an effective way to improve the user’s understanding of power consumption patterns [70,71]. Therefore, displaying energy usage both numerically and graphically helps users understand the usage more easily. In the case of the parking location check function, both the parking location and parking area name are indicated on the drawing (P model house). When a dangerous situation occurs (e.g., fire, intruder), not only does an alarm sound but a message is sent to the user so that the user can respond appropriately. Guidelines for improving sensory affordances are as follows:

• Avoid using decorative fonts;
• Do not use more than three fonts on one screen;
• The brightness contrast between the text and the background should be at least 4.5:1;
• If text can be magnified, the contrast between the text and the background should be at least 3:1;
• Apply a color with high visibility to text that notifies of caution, error, or danger;
• Do not use more than four colors on one screen;
• The buttons for emergency functions (e.g., emergency call) should be distinguished from other buttons by color to be noticeable;
• Differentiate size, color, and thickness of the font according to the hierarchy of information;
• Place frequently checked information or important function in a prominent position,

6. Conclusions

This case study was conducted to suggest guidelines for the application of affordance factors to improve the usability of smart homes. For the first, the affordance factors that could be applied to improve the usability of active devices and user interfaces in smart homes were extracted from previous studies. In the case study, the application status of affordance factors was analyzed, focusing on the active devices with which users directly interact, such as control devices, smart appliances, and their user interfaces. Then, based on the results of the case analysis and extracted affordance factors, guidelines were presented for the application of affordance factors in terms of cognitive, physical, functional, and sensory aspects.

Users interact with active devices or passive devices directly or indirectly in the process of using smart home services. In particular, active devices and their user interfaces, which are used when a user enters commands to obtain the desired services, are closely related to smart home usability because the user interacts directly with them. Therefore, active devices and user interfaces should provide users with appropriate cognitive, physical, functional, and sensory affordances so that users can use the service easily and conveniently. For example, in terms of cognitive affordances, sufficient and appropriate visual clues and information should be provided to help users recognize how to operate active devices and user interfaces. Additionally, physical, functional, and sensory affordances should be provided so that users can perform tasks conveniently and effectively to achieve the desired results.

This study contributes to providing the basis for user-centered smart home design by presenting specific guidelines for improving the usability of active devices and user interfaces in terms of cognitive, physical, functional, and sensory. Unlike studies that presented general design principles for the graphical user interfaces of websites or applications that have been mainly researched in the HCI field, this study presented guidelines considering the characteristics of various kinds and types of active devices and user interfaces in smart homes. Therefore, depending on the type of active devices, the necessary guidelines can be easily applied.

Despite these contributions, this study has some limitations and requires further study based on them. This case study research was conducted on cases where smart home devices had been fully installed. Therefore, the guidelines on the application of affordance factors for improving usability were suggested by focusing on the process of using smart home devices. However, the installation process for smart home devices should also be improved to promote smart home usability. Therefore, user-friendly design of active devices and user interfaces should be accompanied by technological advances such as improved compatibility between devices constituting smart homes and standardization of networks to secure connectivity between devices. Additionally, our suggested guidelines can be applied to general users but do not specify particular types of smart home user. Therefore, for users with specific characteristics (e.g., elderly or disabled individuals), special considerations should be included in the factors of cognitive, physical, functional, and sensory affordance required for the use of smart home services compared to general users. Future research should present more detailed guidelines that consider the characteristics of specific users. In addition, this study presented guidelines for the application of affordances to improve the usability of smart homes based on case study and theoretical considerations. However, further research is necessary that includes usability evaluation by smart home users. By synthesizing the guidelines presented in this study and the research results on the usability evaluation of smart home users, it will be possible to provide more sophisticated guidelines for user-centered smart home design.