Automated Video Face Labelling for Films and TV Material (English)

In: IEEE Transactions on Pattern Analysis and Machine Intelligence   ;  42 ,  4  ;  780-792  ;  2020

How to get this document?

Commercial Copyright fee: €28.50 Basic fee: €4.00 Total price: €32.50
Academic Copyright fee: €28.50 Basic fee: €2.00 Total price: €30.50

The objective of this work is automatic labelling of characters in TV video and movies, given weak supervisory information provided by an aligned transcript. We make five contributions: (i) a new strategy for obtaining stronger supervisory information from aligned transcripts; (ii) an explicit model for classifying background characters, based on their face-tracks; (iii) employing new ConvNet based face features, and (iv) a novel approach for labelling all face tracks jointly using linear programming. Each of these contributions delivers a boost in performance, and we demonstrate this on standard benchmarks using tracks provided by authors of prior work. As a fifth contribution, we also investigate the generalisation and strength of the features and classifiers by applying them “in the raw” on new video material where no supervisory information is used. In particular, to provide high quality tracks on those material, we propose efficient track classifiers to remove false positive tracks by the face tracker. Overall we achieve a dramatic improvement over the state of the art on both TV series and film datasets, and almost saturate performance on some benchmarks.

Table of contents – Volume 42, Issue 4

Show all volumes and issues

The tables of contents are generated automatically and are based on the data records of the individual contributions available in the index of the TIB portal. The display of the Tables of Contents may therefore be incomplete.

A Hybrid RNN-HMM Approach for Weakly Supervised Temporal Action Segmentation
Kuehne, Hilde / Richard, Alexander / Gall, Juergen | 2020
Automated Video Face Labelling for Films and TV Material
Parkhi, Omkar M. / Rahtu, Esa / Cao, Qiong / Zisserman, Andrew | 2020
Baselines Extraction from Curved Document Images via Slope Fields Recovery
Meng, Gaofeng / Pan, Chunhong / Xiang, Shiming / Wu, Ying | 2020
Deep Self-Evolution Clustering
Chang, Jianlong / Meng, Gaofeng / Wang, Lingfeng / Xiang, Shiming / Pan, Chunhong | 2020
Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs
Malkov, Yu A. / Yashunin, D. A. | 2020
Extracting Geometric Structures in Images with Delaunay Point Processes
Favreau, Jean-Dominique / Lafarge, Florent / Bousseau, Adrien / Auvolat, Alex | 2020
Group Maximum Differentiation Competition: Model Comparison with Few Samples
Ma, Kede / Duanmu, Zhengfang / Wang, Zhou / Wu, Qingbo / Liu, Wentao / Yong, Hongwei / Li, Hongliang / Zhang, Lei | 2020
Hierarchical Bayesian Inverse Lighting of Portraits with a Virtual Light Stage
Shahlaei, Davoud / Blanz, Volker | 2020
Hierarchical Fully Convolutional Network for Joint Atrophy Localization and Alzheimer's Disease Diagnosis Using Structural MRI
Lian, Chunfeng / Liu, Mingxia / Zhang, Jun / Shen, Dinggang | 2020
On Detection of Faint Edges in Noisy Images
Ofir, Nati / Galun, Meirav / Alpert, Sharon / Brandt, Achi / Nadler, Boaz / Basri, Ronen | 2020
Perspective-Adaptive Convolutions for Scene Parsing
Zhang, Rui / Tang, Sheng / Zhang, Yongdong / Li, Jintao / Yan, Shuicheng | 2020
Tensor Robust Principal Component Analysis with a New Tensor Nuclear Norm
Lu, Canyi / Feng, Jiashi / Chen, Yudong / Liu, Wei / Lin, Zhouchen / Yan, Shuicheng | 2020
Tracking-by-Fusion via Gaussian Process Regression Extended to Transfer Learning
Gao, Jin / Wang, Qiang / Xing, Junliang / Ling, Haibin / Hu, Weiming / Maybank, Stephen | 2020
Unsupervised Person Re-Identification by Deep Asymmetric Metric Embedding
Yu, Hong-Xing / Wu, Ancong / Zheng, Wei-Shi | 2020
Visibility Graphs for Image Processing
Iacovacci, Jacopo / Lacasa, Lucas | 2020
Weighted Manifold Alignment using Wave Kernel Signatures for Aligning Medical Image Datasets
Clough, James R. / Balfour, Daniel R. / Cruz, Gastao / Marsden, Paul K. / Prieto, Claudia / Reader, Andrew J. / King, Andrew P. | 2020
Denoising Autoencoders for Overgeneralization in Neural Networks
Spigler, Giacomo | 2020
Efficient Graph Cut Optimization for Full CRFs with Quantized Edges
Veksler, Olga | 2020
Learning Raw Image Reconstruction-Aware Deep Image Compressors
Punnappurath, Abhijith / Brown, Michael S. | 2020
| 2020
Table of Contents
| 2020
| 2020