Visual information retrieval in video archives



Prof. Dr. Ralph Ewerth


Kader Pustu-Iren


Deutsche Forschungsgemeinschaft – German Research Foundation (DFG)


1 January 2018 – 31 December 2020


The goal of the project is the development of new methods for the automatic analysis of visual concepts as well as person recognition in large video collections. The German Broadcasting Archive (DRA) provides recordings of GDR television broadcasts with several thousands of hours of video material. One focus is the person recognition in videos for important personalities of GDR history.


The German Broadcasting Archive (DRA) is a non-profit foundation under civil law, with offices in Frankfurt am Main and Potsdam-Babelsberg. The collection priorities of the archive at the Frankfurt location are audio recordings of contemporary history and music since the beginning of recording and historical recording media. In 1994, the DRA was extended to include the radio and television broadcasting archives of the former German Democratic Republic (GDR), initially at a location in Berlin, today in Potsdam-Babelsberg. In joint previous work, selected special GDR television broadcasts were digitized, and using innovative methods of content-based image and video analysis, have been made searchable. The material consists of approximately 3,000 hours of video footage, including the newscasts "Aktuelle Kamera", magazine broadcasts and parts of the East German television film tradition. Through the use and development of automated methods for content-based video analysis, scientists have obtained new possibilities to carry out search queries for desired scenes, camera shots and persons, or for similar images. The sustainability of the achieved very good results of scene classification (detection of visual concepts) and person recognition is intended to be further developed for future use. In the proposed project, a software system usable by archive staff will be developed to enable the DRA and other archives to easily integrate automatic video analysis methods for content-based image search. In this software system, deep learning methods will be explooited, thereby making it possible at the same time to improve person and visual concept detection and expand them to other research-intensive parts of television broadcasts. In particular, the project has the following objectives:

  1. Development of a sustainable software system for user-friendly expansion of two lexicons (concepts and persons) by archive staff,
  2. Integration of the software system in the digitalization workflows of the DRA to make automatic video analysis methods applicable to the total stock of television broadcasts in the archive,
  3. Improvement of the detection rates for concepts and persons by applying deep learning methods,
  4. Expansion of the visual concept lexicon by about 100 further concepts,
  5.  Expansion of the person lexicon to about 100 persons of GDR’s history,
  6. Improvement of the detection rates for concepts and persons through user feedback and similarity search,
  7. Development of appropriate visualizations for effective search.

In this way, is it not only possible for scientists to carry out search queries on the basis of pre-defined concepts and persons, but they can also easily expand the extensive lexicons for visual concepts and persons for their own research tasks. The developed software tools will be made available to other scientific institutions as open source software.


  • Stiftung Deutsches Rundfunkarchiv, Standort Potsdam-Babelsberg, Angelika Hörth
  • Philipps-Universität Marburg, Lehrstuhl Verteilte Systeme, Prof. Bernd Freisleben

Back to list