Research

13 pages, 2090 KiB

Open AccessArticle

Use of IMU in Differential Analysis of the Reverse Punch Temporal Structure in Relation to the Achieved Maximal Hand Velocity

by Stefan Marković, Anton Kos, Vesna Vuković, Milivoj Dopsaj, Nenad Koropanovski and Anton Umek

Sensors 2021, 21(12), 4148; https://0-doi-org.brum.beds.ac.uk/10.3390/s21124148 - 17 Jun 2021

Cited by 6 | Viewed by 2272

Abstract

To achieve good performance, athletes need to synchronize a series of movements in an optimal manner. One of the indicators used to monitor this is the order of occurrence of relevant events in the movement timeline. However, monitoring of this characteristic of rapid [...] Read more.

To achieve good performance, athletes need to synchronize a series of movements in an optimal manner. One of the indicators used to monitor this is the order of occurrence of relevant events in the movement timeline. However, monitoring of this characteristic of rapid movement is practically limited to the laboratory settings, in which motion tracking systems can be used to acquire relevant data. Our motivation is to implement a simple-to-use and robust IMU-based solution suitable for everyday praxis. In this way, repetitive execution of technique can be constantly monitored. This provides augmented feedback to coaches and athletes and is relevant in the context of prevention of stabilization of errors, as well as monitoring for the effects of fatigue. In this research, acceleration and rotational speed signal acquired from a pair of IMUs (Inertial Measurement Unit) is used for detection of the time of occurrence of events. The research included 165 individual strikes performed by 14 elite and national-level karate competitors. All strikes were classified as slow, average, or fast based on the achieved maximal velocity of the hand. A Kruskal–Wallis test revealed significant general differences in the order of occurrence of hand acceleration start, maximal hand velocity, maximal body velocity, maximal hand acceleration, maximal body acceleration, and vertical movement onset between the groups. Partial differences were determined using a Mann–Whitney test. This paper determines the differences in the temporal structure of the reverse punch in relation to the achieved maximal velocity of the hand as a performance indicator. Detecting the time of occurrence of events using IMUs is a new method for measuring motion synchronization that provides a new insight into the coordination of articulated human movements. Such application of IMU can provide additional information about the studied structure of rapid discrete movements in various sporting activities that are otherwise imperceptible to human senses. Full article

(This article belongs to the Special Issue New Frontiers in Sensor-Based Activity Recognition)

► Show Figures

Figure 1

19 pages, 689 KiB

Open AccessArticle

Comparison between Recurrent Networks and Temporal Convolutional Networks Approaches for Skeleton-Based Action Recognition

by Mihai Nan, Mihai Trăscău, Adina Magda Florea and Cezar Cătălin Iacob

Sensors 2021, 21(6), 2051; https://0-doi-org.brum.beds.ac.uk/10.3390/s21062051 - 15 Mar 2021

Cited by 21 | Viewed by 7791

Abstract

Action recognition plays an important role in various applications such as video monitoring, automatic video indexing, crowd analysis, human-machine interaction, smart homes and personal assistive robotics. In this paper, we propose improvements to some methods for human action recognition from videos that work [...] Read more.

Action recognition plays an important role in various applications such as video monitoring, automatic video indexing, crowd analysis, human-machine interaction, smart homes and personal assistive robotics. In this paper, we propose improvements to some methods for human action recognition from videos that work with data represented in the form of skeleton poses. These methods are based on the most widely used techniques for this problem—Graph Convolutional Networks (GCNs), Temporal Convolutional Networks (TCNs) and Recurrent Neural Networks (RNNs). Initially, the paper explores and compares different ways to extract the most relevant spatial and temporal characteristics for a sequence of frames describing an action. Based on this comparative analysis, we show how a TCN type unit can be extended to work even on the characteristics extracted from the spatial domain. To validate our approach, we test it against a benchmark often used for human action recognition problems and we show that our solution obtains comparable results to the state-of-the-art, but with a significant increase in the inference speed. Full article

(This article belongs to the Special Issue New Frontiers in Sensor-Based Activity Recognition)

► Show Figures

Figure 1

25 pages, 4008 KiB

Open AccessArticle

Smartwatch-Based Eating Detection: Data Selection for Machine Learning from Imbalanced Data with Imperfect Labels

by Simon Stankoski, Marko Jordan, Hristijan Gjoreski and Mitja Luštrek

Sensors 2021, 21(5), 1902; https://0-doi-org.brum.beds.ac.uk/10.3390/s21051902 - 09 Mar 2021

Cited by 11 | Viewed by 5418

Abstract

Understanding people’s eating habits plays a crucial role in interventions promoting a healthy lifestyle. This requires objective measurement of the time at which a meal takes place, the duration of the meal, and what the individual eats. Smartwatches and similar wrist-worn devices are [...] Read more.

Understanding people’s eating habits plays a crucial role in interventions promoting a healthy lifestyle. This requires objective measurement of the time at which a meal takes place, the duration of the meal, and what the individual eats. Smartwatches and similar wrist-worn devices are an emerging technology that offers the possibility of practical and real-time eating monitoring in an unobtrusive, accessible, and affordable way. To this end, we present a novel approach for the detection of eating segments with a wrist-worn device and fusion of deep and classical machine learning. It integrates a novel data selection method to create the training dataset, and a method that incorporates knowledge from raw and virtual sensor modalities for training with highly imbalanced datasets. The proposed method was evaluated using data from 12 subjects recorded in the wild, without any restriction about the type of meals that could be consumed, the cutlery used for the meal, or the location where the meal took place. The recordings consist of data from accelerometer and gyroscope sensors. The experiments show that our method for detection of eating segments achieves precision of 0.85, recall of 0.81, and F1-score of 0.82 in a person-independent manner. The results obtained in this study indicate that reliable eating detection using in the wild recorded data is possible with the use of wearable sensors on the wrist. Full article

(This article belongs to the Special Issue New Frontiers in Sensor-Based Activity Recognition)

► Show Figures

Figure 1

31 pages, 1409 KiB

Open AccessArticle

A General Framework for Making Context-Recognition Systems More Energy Efficient

by Vito Janko and Mitja Luštrek

Sensors 2021, 21(3), 766; https://0-doi-org.brum.beds.ac.uk/10.3390/s21030766 - 24 Jan 2021

Cited by 2 | Viewed by 1665

Abstract

Context recognition using wearable devices is a mature research area, but one of the biggest issues it faces is the high energy consumption of the device that is sensing and processing the data. In this work we propose three different methods for optimizing [...] Read more.

Context recognition using wearable devices is a mature research area, but one of the biggest issues it faces is the high energy consumption of the device that is sensing and processing the data. In this work we propose three different methods for optimizing its energy use. We also show how to combine all three methods to further increase the energy savings. The methods work by adapting system settings (sensors used, sampling frequency, duty cycling, etc.) to both the detected context and directly to the sensor data. This is done by mathematically modeling the influence of different system settings and using multiobjective optimization to find the best ones. The proposed methodology is tested on four different context-recognition tasks where we show that it can generate accurate energy-efficient solutions—in one case reducing energy consumption by 95% in exchange for only four percentage points of accuracy. We also show that the method is general, requires next to no expert knowledge about the domain being optimized, and that it outperforms two approaches from the related work. Full article

(This article belongs to the Special Issue New Frontiers in Sensor-Based Activity Recognition)

► Show Figures

Figure 1

18 pages, 3832 KiB

Open AccessArticle

Analyzing the Importance of Sensors for Mode of Transportation Classification

by Björn Friedrich, Carolin Lübbe and Andreas Hein

Sensors 2021, 21(1), 176; https://0-doi-org.brum.beds.ac.uk/10.3390/s21010176 - 29 Dec 2020

Cited by 6 | Viewed by 2350

Abstract

The broad availability of smartphones and Inertial Measurement Units in particular brings them into the focus of recent research. Inertial Measurement Unit data is used for a variety of tasks. One important task is the classification of the mode of transportation. In the [...] Read more.

The broad availability of smartphones and Inertial Measurement Units in particular brings them into the focus of recent research. Inertial Measurement Unit data is used for a variety of tasks. One important task is the classification of the mode of transportation. In the first step, we present a deep-learning-based algorithm that combines long-short-term-memory (LSTM) layer and convolutional layer to classify eight different modes of transportation on the Sussex–Huawei Locomotion-Transportation (SHL) dataset. The inputs of our model are the accelerometer, gyroscope, linear acceleration, magnetometer, gravity and pressure values as well as the orientation information. In the second step, we analyze the contribution of each sensor modality to the classification score and to the different modes of transportation. For this analysis, we subtract the baseline confusion matrix from a confusion matrix of a network trained with a left-out sensor modality (difference confusion matrix) and we visualize the low-level features from the LSTM layers. This approach provides useful insights into the properties of the deep-learning algorithm and indicates the presence of redundant sensor modalities. Full article

(This article belongs to the Special Issue New Frontiers in Sensor-Based Activity Recognition)

► Show Figures

Figure 1

15 pages, 2602 KiB

Open AccessArticle

IMU-Based Movement Trajectory Heatmaps for Human Activity Recognition

by Orhan Konak, Pit Wegner and Bert Arnrich

Sensors 2020, 20(24), 7179; https://0-doi-org.brum.beds.ac.uk/10.3390/s20247179 - 15 Dec 2020

Cited by 9 | Viewed by 4473

Abstract

Recent trends in ubiquitous computing have led to a proliferation of studies that focus on human activity recognition (HAR) utilizing inertial sensor data that consist of acceleration, orientation and angular velocity. However, the performances of such approaches are limited by the amount of [...] Read more.

Recent trends in ubiquitous computing have led to a proliferation of studies that focus on human activity recognition (HAR) utilizing inertial sensor data that consist of acceleration, orientation and angular velocity. However, the performances of such approaches are limited by the amount of annotated training data, especially in fields where annotating data is highly time-consuming and requires specialized professionals, such as in healthcare. In image classification, this limitation has been mitigated by powerful oversampling techniques such as data augmentation. Using this technique, this work evaluates to what extent transforming inertial sensor data into movement trajectories and into 2D heatmap images can be advantageous for HAR when data are scarce. A convolutional long short-term memory (ConvLSTM) network that incorporates spatiotemporal correlations was used to classify the heatmap images. Evaluation was carried out on Deep Inertial Poser (DIP), a known dataset composed of inertial sensor data. The results obtained suggest that for datasets with large numbers of subjects, using state-of-the-art methods remains the best alternative. However, a performance advantage was achieved for small datasets, which is usually the case in healthcare. Moreover, movement trajectories provide a visual representation of human activities, which can help researchers to better interpret and analyze motion patterns. Full article

(This article belongs to the Special Issue New Frontiers in Sensor-Based Activity Recognition)

► Show Figures

Figure 1

19 pages, 2974 KiB

Open AccessArticle

A Framework of Combining Short-Term Spatial/Frequency Feature Extraction and Long-Term IndRNN for Activity Recognition

by Beidi Zhao, Shuai Li, Yanbo Gao, Chuankun Li and Wanqing Li

Sensors 2020, 20(23), 6984; https://doi.org/10.3390/s20236984 - 07 Dec 2020

Cited by 9 | Viewed by 2303

Abstract

Smartphone-sensors-based human activity recognition is attracting increasing interest due to the popularization of smartphones. It is a difficult long-range temporal recognition problem, especially with large intraclass distances such as carrying smartphones at different locations and small interclass distances such as taking a train [...] Read more.

Smartphone-sensors-based human activity recognition is attracting increasing interest due to the popularization of smartphones. It is a difficult long-range temporal recognition problem, especially with large intraclass distances such as carrying smartphones at different locations and small interclass distances such as taking a train or subway. To address this problem, we propose a new framework of combining short-term spatial/frequency feature extraction and a long-term independently recurrent neural network (IndRNN) for activity recognition. Considering the periodic characteristics of the sensor data, short-term temporal features are first extracted in the spatial and frequency domains. Then, the IndRNN, which can capture long-term patterns, is used to further obtain the long-term features for classification. Given the large differences when the smartphone is carried at different locations, a group-based location recognition is first developed to pinpoint the location of the smartphone. The Sussex-Huawei Locomotion (SHL) dataset from the SHL Challenge is used for evaluation. An earlier version of the proposed method won the second place award in the SHL Challenge 2020 (first place if not considering the multiple models fusion approach). The proposed method is further improved in this paper and achieves 80.72% accuracy, better than the existing methods using a single model. Full article

(This article belongs to the Special Issue New Frontiers in Sensor-Based Activity Recognition)

► Show Figures

Figure 1

16 pages, 5591 KiB

Open AccessArticle

A Random Forest-Based Accuracy Prediction Model for Augmented Biofeedback in a Precision Shooting Training System

by Junqi Guo, Lan Yang, Anton Umek, Rongfang Bie, Sašo Tomažič and Anton Kos

Sensors 2020, 20(16), 4512; https://0-doi-org.brum.beds.ac.uk/10.3390/s20164512 - 12 Aug 2020

Cited by 2 | Viewed by 3144

Abstract

In the military, police, security companies, and shooting sports, precision shooting training is of the outmost importance. In order to achieve high shooting accuracy, a lot of training is needed. As a result, trainees use a large number of cartridges and a considerable [...] Read more.

In the military, police, security companies, and shooting sports, precision shooting training is of the outmost importance. In order to achieve high shooting accuracy, a lot of training is needed. As a result, trainees use a large number of cartridges and a considerable amount of time of professional trainers, which can cost a lot. Our motivation is to reduce costs and shorten training time by introducing an augmented biofeedback system based on machine learning techniques. We are designing a system that can detect and provide feedback on three types of errors that regularly occur during a precision shooting practice: excessive hand movement error, aiming error and triggering error. The system is designed to provide concurrent feedback on the hand movement error and terminal feedback on the other two errors. Machine learning techniques are used innovatively to identify hand movement errors; the other two errors are identified by the threshold approach. To correct the excessive hand movement error, a precision shot accuracy prediction model based on Random Forest has proven to be the most suitable. The experimental results show that: (1) the proposed Random Forest (RF) model achieves the prediction accuracy of 91.27%, higher than any of the other reference models, and (2) hand movement is strongly related to the accuracy of precision shooting. Appropriate use of the proposed augmented biofeedback system will result in a lower number of rounds used and shorten the precision shooting training process. Full article

(This article belongs to the Special Issue New Frontiers in Sensor-Based Activity Recognition)

► Show Figures

Figure 1

27 pages, 14826 KiB

Open AccessArticle

Towards Breathing as a Sensing Modality in Depth-Based Activity Recognition

by Jochen Kempfle and Kristof Van Laerhoven

Sensors 2020, 20(14), 3884; https://0-doi-org.brum.beds.ac.uk/10.3390/s20143884 - 13 Jul 2020

Cited by 9 | Viewed by 3134

Abstract

Depth imaging has, through recent technological advances, become ubiquitous as products become smaller, more affordable, and more precise. Depth cameras have also emerged as a promising modality for activity recognition as they allow detection of users’ body joints and postures. Increased resolutions have [...] Read more.

Depth imaging has, through recent technological advances, become ubiquitous as products become smaller, more affordable, and more precise. Depth cameras have also emerged as a promising modality for activity recognition as they allow detection of users’ body joints and postures. Increased resolutions have now enabled a novel use of depth cameras that facilitate more fine-grained activity descriptors: The remote detection of a person’s breathing by picking up the small distance changes from the user’s chest over time. We propose in this work a novel method to model chest elevation to robustly monitor a user’s respiration, whenever users are sitting or standing, and facing the camera. The method is robust to users occasionally blocking their torso region and is able to provide meaningful breathing features to allow classification in activity recognition tasks. We illustrate that with this method, with specific activities such as paced-breathing meditating, performing breathing exercises, or post-exercise recovery, our model delivers a breathing accuracy that matches that of a commercial respiration chest monitor belt. Results show that the breathing rate can be detected with our method at an accuracy of 92 to 97% from a distance of two metres, outperforming state-of-the-art depth imagining methods especially for non-sedentary persons, and allowing separation of activities in respiration-derived features space. Full article

(This article belongs to the Special Issue New Frontiers in Sensor-Based Activity Recognition)

► Show Figures

Figure 1

Journal Menu

Journal Browser

New Frontiers in Sensor-Based Activity Recognition

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (9 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI