Man versus machine: which is better at estimating a photo’s geolocation?

TIB’s Visual Analytics Research Group develops innovative geolocalisation method

The photo shows the skyline of a city against a grey sky, with water in the foreground: which city is it? “Geolocation Estimation”, developed by the TIB – Leibniz Information Centre for Science and Technology, enables humans to compete against a machine in guessing where the photo was taken. The answer: Seattle. 

It’s usually the computer that wins, defeating its human rivals. But how does it work? It’s all down to artificial intelligence.

TIB’s Visual Analytics Research Group (led by Professor Dr. Ralph Ewerth) conducts research on the topic of visual concept detection, which is the automatic extraction of information from images. The team has developed an innovative method for estimating a photo’s geolocation: a machine learning method based on so-called convolutional neural networks (CNNs) uses not only geographical features, but also contextual information about the scene in the photo to estimate the photo’s geolocation. In this way, the neural network, which mimics neurobiological processes in the human brain, can be trained to recognise specific geographical features for various scenarios such as urban scenes, natural environments or indoor settings. “With urban images, such features may include buildings or architectural details; in the case of nature scenes, flora and fauna are considered,” stated Prof. Dr. Ewerth, describing the method.

TIB gives better results than Google’s method

The machine is demonstrably better than humans at geolocation estimation; what’s more, TIB’s innovation need not even hide behind the geolocalisation approach developed by Google researchers. “Even though we fed our system with less training images, it already delivers better and more precise data than corresponding systems by Google researchers,” remarked TIB doctoral candidates Eric Müller-Budack and Kader Pustu-Iren with a touch of pride.

TIB Director Professor Dr. Sören Auer’s goal is to develop this innovative geolocalisation technique into a TIB web service in the medium term. “With such a service, it would be much easier to locate images throughout the world in memory institutions such as libraries and archives,” stated Auer.

Incidentally, if you want to test yourself against the computer, you can use the browser-based demonstration: https://labs.tib.eu/geoestimation

Find out more about the Visual Analytics Research Group: http://tib.eu/visual-analytics

Under the title “Wo war das noch mal?“, editor Arne Grävemeyer presents the geolocation tool developed by the TIB in issue 5/2019 of c't magazine.

E. Müller-Budack, K. Pustu-Iren, R. Ewerth:
Geolocation Estimation of Photos using a Hierarchical Model and Scene Classification.
In: Proceedings of the European Conference on Computer Vision (ECCV), München,  Springer, 2018, 563-579. https://link.springer.com/chapter/10.1007/978-3-030-01258-8_35

Open Access: http://openaccess.thecvf.com/content_ECCV_2018/papers/Eric_Muller-Budack_Geolocation_Estimation_of_ECCV_2018_paper.pdf