Loading...

The Effect of Temporal Alignment in 3D Action Recognition Using Recurrent Neural Network

Akyash, Mohammad Hossein | 2021

364 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 53910 (05)
  4. University: Sharif University of Technology
  5. Department: Electrical Engineering
  6. Advisor(s): Behroozi, Hamid; Mohammadzadeh, Hoda
  7. Abstract:
  8. Action recognition has a lot of applications in everyday human life. In the past, the researchers concentrated on using RGB frames, but since the advent of 3-dimensional sensors such as Kinect, 3D action recognition drew researchers' attention. Kinect can extract the joints of the body in action as time series. One of the main challenges of action recognition is that different individuals perform an action with various styles and speeds. Hence, the conventional methods such as calculating Euclidean distance seem inappropriate for this task. One solution is to use the techniques such as DTW, which aims to temporal aligning of the sequences. The DTW is not a metric distance; hence, in this project, we proposed a DTW-based kernel to access DTW space.The proposed kernel is based on calculating the area generated by the DTW aligned path between two time series. To generate the feature vectors, one action is randomly chosen from each class as a reference sample. The proposed kernel function is calculated between a sample and the reference ones. Then the feature vectors are classified with SVM. Through multiple experiments, it is shown that the proposed method can effectively address the issues of noise and frame loss.We proposed an RNN-based structure for aligning times series. In this network, we utilized the concept of DTW for temporal alignment. The main problem of training a neural network is the lack of sufficient numbers of training data which cause overfitting. For solving this issue, we introduced two novel data augmentation methods for time series. The first one is based on random movement in the distance matrix of the DTW algorithm and the second, augment data by merging sequences according to the aligned path of DTW. We use these methods for the classification of action sequences, and the results manifest better performance than when no data augmentation was used. We also experiment the augmentation methods for other applications and the state-of-the-art networks for time series classification, and the results have been improved.The proposed methods are evaluated on TST Fall detection, UTKinect, UCFKinect, and UCR time series classification archive
  9. Keywords:
  10. Temporal Alignment ; Data Augmentation ; Dynamic Time Warping ; Recurrent Neural Networks ; Kernel Trick ; Three Dimention Action Recognition ; Time Series Classification

 Digital Object List

 Bookmark

No TOC