Development of a User-Centric Bridge Visual Defect Quality Control Assisted Mobile Application: A Case of Thailand’s Department of Highways

Abstract

Digital innovations have changed the way many industries operate, but the construction industry has been slow to adopt these technologies. However, challenges such as low productivity, project overruns, labor shortages, and inefficient performance management have motivated Thailand’s Department of Highways to adopt digital innovations to build a competitive advantage. Because this industry requires a large work force, obstacles to collaboration can result in ineffective project management. We aimed to improve collaboration on bridge inspections that typically requires the involvement of many people, personal judgement, and extensive travel to survey bridges across the country. One major challenge is to standardize human judgement. To address these challenges, we developed a user-centric bridge visual defect quality control mobile application to improve collaboration and assist field technicians to conduct visual defect inspection. Our results can be used as a case study for other construction firms to embrace digital transformation technologies. This research also demonstrates the new-product development process using the new technology in known markets innovation development and technology acceptance model. We offer several recommendations for future research, including other infrastructure applications.

1. Introduction

Recently, many industries in Thailand have begun to adopt digital transformation to stay competitive. For example, traditional car rental is transitioning into a car sharing business using Internet of Things (IoT) and mobile phone application technology [1]. The construction industry is also undergoing digital transformation to improve its ability to address challenges such as low productivity, project overruns, labor shortages, and inefficient performance management [2] Suboptimal collaboration among project managers, administrative staff, and on-site laborers is a common problem in large construction firms. Ineffective collaboration leads to inefficient project management, delays, and increased costs. Miscommunication, human error, poorly standardized work processes, and data loss, also contribute to inefficient project management. Bridge inspection requires many staff operating in different locations to perform visual inspection, and to update bridge information. This routine process involves inspection planning, problem identification, and considerable human judgment that is difficult to standardize [3]. The analysis of large digital data sets may improve project management, safety, energy management, decision-making design, resource management, and control costs [4].

Bridge substructure defect inspection is ideally suited for automated or semi-automated digital applications due to frequent human errors and a chronic shortage of skilled inspectors as shown by the study of V. Gattulli and L. Chiaramote [5]. A 2021 study by P. Kruachottikul et al. described the use of mobile phone applications to improve quality control, reduce miscommunication, increase transparency, and improve collaboration in Thailand’s major construction organizations. Such applications enable users to remotely monitor their projects in real-time, support multimedia attachments, and reduce communication time [6]. In addition, our previous study described how artificial intelligence (AI) can assist users with bridge substructure defect inspection and severity prediction [7]. Recently, more study on the various bridge defect detection with AI have been performed in the laboratory, but not in real environment [8]. Here, we continue this work with the development of a mobile phone application with a built-in AI tool to improve quality control (QC) for bridge defect inspection quality control assessment. This tool can also be used to assist QC field technicians to build bridge information databases, detect defects, perform severity ranking, communicate with team members, and assign new tasks. In addition to technical development, we also describe the product development process from concept to implementation and propose factors that influence intention to use under the framework of the technology acceptance model.

The overarching objective is to create value to Thailand’s construction industry through digital transformation technology. In doing so, in collaboration with Thailand’s Department of Highways, our study can improve project management, reduce workload for its limited experts, and improve collaboration among team members. Importantly, by helping to improve road and bridge infrastructure, these technologies may also reduce accidents, a serious problem in Thailand which ranked second in deaths from traffic accidents in a 2018 World Health Organization report [9]. Therefore, we investigated the use of digital technology to conduct nondestructive bridge defect inspection to evaluate the condition of reinforced concrete [10].

Improving project management through better collaboration and quality control can increase productivity and efficiency, leading to cost savings and competitive advantages. Collaboration is about teamwork and communication. Teamwork is individuals working together to achieve a common goal and is developed through continuous communication among team members. Each team member seeks to clarify their roles and responsibilities and reduce ambiguity. Team members then concentrate on their individual tasks and coordinate in sub-groups where mutual expectations are developed. Therefore, the published literature on teamwork, project management, and visual inspection were reviewed and summarized.

First, teamwork is achieved through cooperation and coordination facilitated by effective communication. When team members understand each other and are able to synchronize their activities on a given schedule, the team can avoid delays, budget overruns, and reduce worksite accidents. Together, communication, coordination, cooperation, and synchronicity are key elements of team performance [11]. Communication is also an essential part of project management. The balance of frequent informal and formal communication affects the performance on the project, by influencing the degree of collaboration and the level of trust within the team [12]. Consequently, communication is a major determinant for project success or failure [13]. Instant messaging is a form of computer-mediated communication (CMC) that involves sending digital messages through computer networks [14].

Second, project management has become an important topic for an improvement in construction industry, where communication as information exchange in the broader organization is key [15]. As project teams face challenges from increasing competition and higher productivity targets, it is essential that team members are capable of utilizing CMC effectively. General forms of CMC are, for example, email, instant messaging, collaboration tools, and social networking. Instant messaging, or chat, is the application for sending and receiving short text messages, images, or files between people through a smart phone or computer. Instant messaging, through time, has evolved from teen sensation to an important modern communication tool in daily business. Although email can technically do the same function, instant messaging has some advantages that are team members can send and receive information instantaneously. Moreover, cloud services can be applied as a central repository for construction data as an end-to-end solution for timely decision-making, [16] because construction projects usually involve several project teams, handling different project locations, and dealing with various business report models.

Finally, automated optical inspection offers significant advantages over human eyes in terms of fatigue, completeness, speed, and accuracy. Technological advances in sensors, hardware, software, and data transfer speeds have improved automated visual inspection and enabled high standard quality control product inspection and certification. These digital technologies also generate data that can be used for statistical analysis. These tools can be adopted by the construction industry to automate visual inspection processes to improve competitiveness in road pavement, bridge construction, and other infrastructure projects. Pavement maintenance requires the updated information on road conditions. Because humans are subjective, have different levels of expertise and experience, and can suffer from eye fatigue, problems with inter-rater variability are common. Semi-automated pavement inspection systems have been used since 2000, but still rely heavily on human inspection [17,18]. Additionally, pavement inspections are not only concerned with acquiring images and designing image processing algorithms, [19] but also require integrating data related to the surface stages of structures from sensors to analyze the visual quality of the pavement. For example, a mobile phone to provide information of the location and way to communicate between involved parties. Large concrete structures such as bridges differ in age, performance, and condition. Various infrastructure monitoring programs and systems have been integrated to monitor the structural safety and serviceability to provide information for decision making and the allocation of resources [20].

This study explores the use of a mobile application for bridge substructure inspection, a task that requires visual inspection as an initial step that is usually performed manually according to the inspector’s judgment and an evaluation manual. However, this process is subjective, time consuming, and involves safety risks; limitations that can be addressed by the application of digital technology [21].

2. Materials and Methods

2.1. New Product Development Process

The next-generation stage-gate development system involves the conception, development, and launch of new products [22]. It comes with the “Triple A” system concept that promotes an innovation-driven product development process to be adaptive, agile, and accelerated according to each development phase. This is also similar to that of the lean startup concept that encourages development of a minimal viable product (MVP) that is sufficient to generate user feedback to confirm the concept or prompt a pivot [23]. This study combines the stage-gate development system with a human-centered design innovation [24] for a new technology-known market (NT-KM) to discover unmet needs useful for restructuring product processes and to help inform the direction of the late design process. NT-KM starts with identifying the design scope and direction of an innovative product and/or its functionality in existing markets. Moreover, the research method tends to be semi-structured with many open-ended questions and sometimes through direct observation or physical models. The design direction is developed through a detailed understanding of the existing processes to clearly define the desired outcome or capability.

2.2. Technology Acceptance Model

The unified theory of acceptance and use of technology (UTAUT) [25] is applicable for identifying the key factors that influence behavioral intention to use. The objective of the UTAUT is to support management by providing a useful tool to assess the success of new technology introductions and understand the drivers of user acceptance, thereby enabling them to design interventions targeted at populations of users that may be less inclined to adopt and use new systems. The UTAUT framework was modified, as shown in Figure 1, by adding a bridge visual defect quality inspection (VIS) factor that was designed though group interviews with ten lead users from Thailand’s Department of Highways. VIS influences the performance expectancy (PER), effort expectancy (EFF), and social influence (SOC) of UTAUT, and leads to a behavioral intention to use (BEH). Additionally, VIS consists of an image acquisition to capture the bridge and identify defects (VIS1), environment setup (VIS2), bridge defect information assistance (VIS3), and bridge history information (VIS4). After the user acceptance framework was developed, a correlation analysis based on a quantitative analysis involving 400 questionnaires was performed to confirm the validity of previously identified factors, and to gather information to build a user-driven mobile application. This analysis identified only VIS-PER-BEH and VIS-EFF-BEH to be significant (

Table 1. Technology acceptance model framework results.

Factor	Hypothesis	df	Mean Square	SD	F	Sig.
VIS > PER	H1	39	0.473	0.464	2.529	0.000*
VIS > EFF	H2	39	0.337	0.462	1.688	0.008*
VIS > SOC	H3	39	0.416	0.525	1.598	0.016*
PER > BEH	H4	8	0.593	0.520	5.036	0.000*
EFF > BEH	H5	10	0.466	0.520	3.960	0.000*
SOC > BEH	H6	11	0.113	0.520	0.962	0.482

). VIS-SOC-BEH was not significant, suggesting that a user’s behavioral intention relies on personal factors and not from social influence. The analysis also determined that VIS affects PER the most and that PER has the biggest impact on BEH. In other words, user intention occurs once users try the product themselves, and not when other people in an organization convince them to use it. This suggests that allowing the user to test the product in a free trial could be an effective strategy. Additionally, user intention to trial the product is more likely to occur if they recognize that the product helps them improve their work performance. This also suggests that product features that include image acquisition to capture the bridge and identify defects (VIS1), environment setup assistance (VIS2), bridge defect information assistance (VIS3), and bridge history information (VIS4), can assist user collaboration and inspection issues explicitly. After all factors are confirmed, they are taken into account to design the user experience and user interface (UXUI) of the mobile application.

2.3. Functional and Non-Functional Requirements

This section explains user requirements and key findings for functional and non-functional requirements. The initial requirements were derived from our previous study [3] as described in items 1 to 6 in

Table 2. User requirements and key findings for bridge inspection mobile application.

User Requirements and Key Findings
1. The application can communicate in the same way as instant messaging.
2. The application can report to project managers in real-time.
3. The application can upload multimedia files, such as images or videos.
4. The application contains a progress status for each subtask.
5. The application is able to identify the responsible person for each task.
6. The application can send the report via email.
7. The application can be trialed for free before committing to use.
8. The application trial convinces the user that it has the potential to improve the user’s work performance. This suggests that product features such as image acquisition to capture the bridge (VIS1), identify defects (VIS2), environment setup assistance (VIS3), and bridge defect information and bridge history information (VIS4), will support user collaboration and inspection issues.

, and then combined with the key findings (items 7 and 8) from our present study of the technology acceptance model.

In addition, Thailand’s Department of Highways’ inspection procedure flow chart for bridge substructure was modified by adding a mobile application with assisted technology for the visual inspection process. (Figure 2) [25].

Next, the system is designed to automatically synchronize data with the cloud server system via a mobile phone network (Figure 3).

2.4. Use Case Diagram

After in-depth interviews, document analysis, and on-site monitoring, a use case diagram was created (Figure 4). Users were divided into project managers (expert engineers) and users (bridge inspectors). Each group can access different functions according to their roles. Project managers can manage all functions in the application, including adding projects and bridges that need to be inspected, and assign users specific tasks during the inspection processes. Project managers have authority to confirm or reject the AI-assisted inspection result. All data from each inspection is stored on a cloud server and can be used to make maintenance plans and to improve AI accuracy. Users are limited to task functions. They can add and edit the bridge and its damage information and upload an image for AI analysis. A comment system for each task facilitates communication between users and project managers.

2.5. Hierarchy of Application Screen

The application starts with the default log-in page, where a user can either log in or create a new account. After logging in, three screens can be selected as All Project, Notification, and Setting screen. Then, under the All Project screen, the user can add or edit a project, view all bridge details, and view assigned task lists from the Task Screen. Finally, under Task Screen, the user can add task, assign, or edit accordingly. Hierarchy of application screen is shown in Figure 5.

2.6. Backend Design

This mobile application backend design is based on microservice architecture because it is a collection of small services, making the service management straightforward and easier to add services in the future. (Figure 6) A variety of Amazon web services (AWS) are employed to develop the application (API Gateway, Cognito, Cloudwatch, AWS Lambda, DynamoDB, S3, and AWS IAM) [25]. The user interface (UI) of the application was developed using the react native framework [26]. The UI communicates with backend software using AWS Amplify via Amazon API Gateway, involving API method data management such as GET, POST, PUT, and DELETE that read, write, update, and delete data via APIs. There are four functions (user, project, task, and notification) with their corresponding APIs, created on AWS Lambda for managing project collaboration information. The Expo [27] backend server was chosen to handle the notification part on account of multi-platform message transmission via Firebase Cloud Messaging for Android and Apple push notification service for iOS [28]. The developed image processing software, CNN and ANN, are also executed on AWS Lambdas. For data storage, Amazon DynamoDB, which is the NoSQL database, creates five tables to store the text data and Amazon S3 Public Bucket stores images. Any procedures related to the authentication system, such as log-in, register, credential request, and status checking, are required to connect with Amazon Cognito. Lambda Trigger function is used as post-confirmation for creating the user DynamoDB database.

2.7. Microservice Development

The development of a collection of small services in the microservice architecture can best be explained with a sequence diagram. The sequence diagram of the AI assisted bridge visual defect inspection microservice collection was derived from our prior study [7], which starts from the user uploading an image and defect information through the system. This original image is stored in Amazon S3 bucket whereas the defect information is recorded on the image table. Image processing function processes the original image by enhancing and filtering to increase contrast and remove noise. The processed image is stored in Amazon S3 bucket. The defect inspection function analyzes a processed image to identify the defect section and send these to defect classification function to categorize the types of defects. The severity prediction function evaluates the seriousness of the defect to help the project manager make decisions. This study proposes an alternative sequence diagram of the user collaboration as shown in Figure 7b. It indicates that when users upload or modify the task information, the application credentials, paths, and parameters are sent to the API gateway. The system will call a task function with the task path to store the task info in the task table and return these parameters back to the application UI.

3. Results

3.1. Mobile Application

First, a user communication requirement was analyzed, the web board feature was chosen, and then implemented at the task detail page. This feature allows the user and project manager to communicate based on each task topic, so it is easy to track the progress and see the historical conversation. After the new project is created, the user can add a new task in a web board function. When creating a new task, the user must add an image and task detail, and can insert a new comment or reply to the existing topic. When a real-time report is required, a notification feature can be added to inform the project manager via multiple channels such as in-app notification or email. At the task summary and detail page, the user can open, read, or edit issues including progress status, detail, and web board. A person in-charge of a specific task can be assigned by the user. Furthermore, a task status shown in the task detail page is used to indicate whether the task is on-going or finished. Visual inspection assistance was implemented to support users to make more accurate decisions, reduce inspection time, and provide an improved inspection experience. Thus, the image acquisition tool supports users to acquire the bridge images easily in different environments and the vision AI assisted feature automatically identifies defects and predicts the severity result (severe or non-severe). The result is sent to users to verify and confirm the results, saving time and enabling prescreening of results. Users trialed the product and suggested the addition of self-learning tutorial tools, a more intuitive UXUI, and using the mobile application as a core platform. Next, the mobile application wireframe (Figure 8) and the application page schematic as a visual blueprint were developed using the earlier UXUI to improve visual understanding of the application during the research development phase. Its advantages are to optimize the layout and content placement easily, and solve navigation and functionality problems in an adjustable format.

3.2. Application Employment

The system is designed as a communication tool with AI embedded to assist the project managers and bridge inspectors. Following systematic instruction of the application as shown in Figure 9 was demonstrated to inspection crews. Project managers creates new projects and assign bridges to be inspected according to the inspection plan from the headquarter responding to their local divisions. Bridge inspectors are assigned. People involved in the process are notified by the application. Bridge defects are examined by taking defect images and recording the defect details. These data are sent through Cloud via 4G mobile network to be stored and analyzed by the assist AI. Project managers review the defect information and can communicate with bridge inspectors to discuss in-depth details on each defect with application web board system. With these organized and AI assisted data, maintenance plan can be created efficiently and optimally.

3.3. User Satisfaction Results

After development was completed, a prototype was delivered for a hands-on trial in a controlled environment by 14 project managers and bridge inspectors from the Thailand Department of Highways. After the trial, satisfaction surveys and in-depth interviews were conducted to elicit and measure feedback. Four domains were assessed: functionality, user interface design, user experience, and overall performance. User satisfaction in each domain was scored from 0 (unsatisfactory) to 5, indicating highly satisfied.

The average satisfaction result of each domain was 4.00, 3.714, 4.00, and 4.142, respectively, and the total score was 4.024 (80.48%). (

Table 3. User satisfaction for bridge inspection mobile application with 14 trial users.

	Score (Max. 5.00)	Percentage
Functionality	4.00	80%
User interface design	3.714	74.28%
User Experience	4.00	80%
Overall Performance	4.142	82.84%

) This means that the implemented features in this application that are required for behavioral intention to use, were built and tested by another group of users who were highly satisfied. However, the lowest score of user interface design (

Table 4. User Interface.

Log in	Project Overview	List of Bridge under Project	Notification
Add/Edit Project	Add/Edit Bridge	Add New Task with Image	Add Task Detail
Task Detail for Inspector with comment	Assigned Person	Edit Task	User Setting

) can be improved in the future. According to the concept of stage-gate process, the improvement will be performed after collecting and analyzing more user validation feedback.

4. Discussion

This study was conducted in collaboration with the Thailand Department of Highways and is an example of high-impact digital transformation that can improve public service and safety. We developed a prototype mobile application to assist field technicians to conduct visual bridge defect inspection and improved collaboration among team members. To develop a successful user driven application, we applied the unified theory of acceptance and use of technology (UTAUT) to identify the key factors that influence behavioral intention to use of the mobile application. The fundamental specifications were derived from our previous research on mobile applications for construction process quality control [6], AI for visual bridge defect-inspection system [7], and related literature reviews in the area of teamwork, project management, and visual inspection. Building on this body of work, the current study proposes a modified technology acceptance model UTAUT research framework, which identifies parameters that influence user intention to use the product by adding a new VIS factor that leads to either performance expectancy or effort efficacy.

As a result, two valuable insights were obtained. First, the user expects the technology to improve their routine task performance. Second, the intention to use will occur when prospective users trial the product and see the potential benefit of the technology. This suggests that allowing the user to test the product in a free trial could be an effective strategy. Next, the product concept was developed including the system architecture, use case diagram, user interface, and backend design. This process was repeated several times according to the stage-gate process “Triple A” concept. After the prototype was constructed, it was used for a field trial in a controlled environment and a user satisfaction survey was conducted. The overall satisfaction score of 14 users was 4.024 of 5.00, reflecting the high satisfaction for user behavioral intention to use the product. However, satisfaction scores were lowest (3.714 of 5.00) with the user interface design section and so the UXUI will be improved with fine-tuning after the analyses of additional user feedback.

Although, few aspects of the new system need to be improved before replacing traditional methods, there is an agreement that this new system can significantly enhance the performance of the overall inspection processes. It might be slowly replaced or parallelly implemented with the traditional methods but eventually will be a main tool for bridge inspection as a government digital transformation policy.

5. Conclusions

In summary, trial users were highly satisfied with the first version of the Bridge Visual Defect Quality Control Assisted Mobile Application. Although there are details that need to be improved, engineers at Thailand’s Department of Highways expressed a strong interest in using this application to reduce workload and improve quality. There are more than 10,000 bridges in Thailand and their maintenance is a priority. This mobile application could be the first in a series of steps transforming bridge maintenance systems to full digitalization, uniting multiple operations into a single system for better project management. There are many other potential opportunities for research using a collaborative platform and visual inspection assisted technology in the construction industry. With the continued cooperation of the Thailand Department of Highways, future applications may be applied in areas such as road pavement, highway infrastructure inspection, or even overall transportation maintenance system.