A Hybrid RNN-HMM Approach for Weakly Supervised Temporal Action Segmentation (English)

In: IEEE Transactions on Pattern Analysis and Machine Intelligence   ;  42 ,  4  ;  765-779  ;  2020

How to get this document?

Free access

Action recognition has become a rapidly developing research field within the last decade. But with the increasing demand for large scale data, the need of hand annotated data for the training becomes more and more impractical. One way to avoid frame-based human annotation is the use of action order information to learn the respective action classes. In this context, we propose a hierarchical approach to address the problem of weakly supervised learning of human actions from ordered action labels by structuring recognition in a coarse-to-fine manner. Given a set of videos and an ordered list of the occurring actions, the task is to infer start and end frames of the related action classes within the video and to train the respective action classifiers without any need for hand labeled frame boundaries. We address this problem by combining a framewise RNN model with a coarse probabilistic inference. This combination allows for the temporal alignment of long sequences and thus, for an iterative training of both elements. While this system alone already generates good results, we show that the performance can be further improved by approximating the number of subactions to the characteristics of the different action classes as well as by the introduction of a regularizing length prior. The proposed system is evaluated on two benchmark datasets, the Breakfast and the Hollywood extended dataset, showing a competitive performance on various weak learning tasks such as temporal action segmentation and action alignment.

Table of contents – Volume 42, Issue 4

Show all volumes and issues

The tables of contents are generated automatically and are based on the data records of the individual contributions available in the index of the TIB portal. The display of the Tables of Contents may therefore be incomplete.

765
A Hybrid RNN-HMM Approach for Weakly Supervised Temporal Action Segmentation
Kuehne, Hilde / Richard, Alexander / Gall, Juergen | 2020
780
Automated Video Face Labelling for Films and TV Material
Parkhi, Omkar M. / Rahtu, Esa / Cao, Qiong / Zisserman, Andrew | 2020
793
Baselines Extraction from Curved Document Images via Slope Fields Recovery
Meng, Gaofeng / Pan, Chunhong / Xiang, Shiming / Wu, Ying | 2020
809
Deep Self-Evolution Clustering
Chang, Jianlong / Meng, Gaofeng / Wang, Lingfeng / Xiang, Shiming / Pan, Chunhong | 2020
824
Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs
Malkov, Yu A. / Yashunin, D. A. | 2020
837
Extracting Geometric Structures in Images with Delaunay Point Processes
Favreau, Jean-Dominique / Lafarge, Florent / Bousseau, Adrien / Auvolat, Alex | 2020
851
Group Maximum Differentiation Competition: Model Comparison with Few Samples
Ma, Kede / Duanmu, Zhengfang / Wang, Zhou / Wu, Qingbo / Liu, Wentao / Yong, Hongwei / Li, Hongliang / Zhang, Lei | 2020
865
Hierarchical Bayesian Inverse Lighting of Portraits with a Virtual Light Stage
Shahlaei, Davoud / Blanz, Volker | 2020
880
Hierarchical Fully Convolutional Network for Joint Atrophy Localization and Alzheimer's Disease Diagnosis Using Structural MRI
Lian, Chunfeng / Liu, Mingxia / Zhang, Jun / Shen, Dinggang | 2020
894
On Detection of Faint Edges in Noisy Images
Ofir, Nati / Galun, Meirav / Alpert, Sharon / Brandt, Achi / Nadler, Boaz / Basri, Ronen | 2020
909
Perspective-Adaptive Convolutions for Scene Parsing
Zhang, Rui / Tang, Sheng / Zhang, Yongdong / Li, Jintao / Yan, Shuicheng | 2020
925
Tensor Robust Principal Component Analysis with a New Tensor Nuclear Norm
Lu, Canyi / Feng, Jiashi / Chen, Yudong / Liu, Wei / Lin, Zhouchen / Yan, Shuicheng | 2020
939
Tracking-by-Fusion via Gaussian Process Regression Extended to Transfer Learning
Gao, Jin / Wang, Qiang / Xing, Junliang / Ling, Haibin / Hu, Weiming / Maybank, Stephen | 2020
956
Unsupervised Person Re-Identification by Deep Asymmetric Metric Embedding
Yu, Hong-Xing / Wu, Ancong / Zheng, Wei-Shi | 2020
974
Visibility Graphs for Image Processing
Iacovacci, Jacopo / Lacasa, Lucas | 2020
988
Weighted Manifold Alignment using Wave Kernel Signatures for Aligning Medical Image Datasets
Clough, James R. / Balfour, Daniel R. / Cruz, Gastao / Marsden, Paul K. / Prieto, Claudia / Reader, Andrew J. / King, Andrew P. | 2020
998
Denoising Autoencoders for Overgeneralization in Neural Networks
Spigler, Giacomo | 2020
1005
Efficient Graph Cut Optimization for Full CRFs with Quantized Edges
Veksler, Olga | 2020
1013
Learning Raw Image Reconstruction-Aware Deep Image Compressors
Punnappurath, Abhijith / Brown, Michael S. | 2020
1020
2020 COMPSAC CFP
| 2020
C1
Table of Contents
| 2020
C2
Cover
| 2020