A new Comprehensive Approach for Earth Observation Scene Classification using Image - Text Analysis
Vaduva, Corina¹; Datcu, Mihai^2
1University Politehnica of Bucharest, ROMANIA; ²DLR, GERMANY

The recent continuous increase in the number of different data sources has caused not only an important growth in the volume and complexity of the available information, but also in the diversity of the end products. Despite this fact, existing technologies enable the management of great collections of high-resolution remote sensing images that offer important details about the Earth's land cover. However, in addition to the traditional Earth Observation (EO) data, the World Wide Web hosts the largest electronic database in the world, offering easy access to supplementary information, such as pictures and text descriptions of objects distinguishable in the EO images. The fusion of both available information and knowledge would provide a broader and more realistic scene annotation.
This paper aims at presenting a new comprehensive approach for scene classification based on the combination of EO images and open access information describing objects present in the scene, such as text, maps or pictures. The main challenge comes from the necessity of leveling the data, due to the different formats and associated analysis requirements. In order to solve this problem, the authors exploit the full advantage of Kolmogorov complexity as a measure of the computational resources needed to specify one object. Despite the fact that it is a non-computable notion, the Kolmogorov complexity can be approximated by a universal similarity metric, namely the Normalized Compression Distance (NCD) which can be applied to various type of data (i.e. images, genomes, hand writing). Advantages of the NCD include its parametric free approach, lack of limitations regarding the application domain and no necessity of background knowledge about the processed objects.
The NCD measure is used to compute a distance matrix for a collection of various types of data, followed by a classification performed by extracting a hierarchy of classes from the distance matrix. This way, the results will provide a high level semantic annotation of the analyzed EO scene due to the fact that it will encapsulate information collected from various sources: EO images, description of the objects included in the images (i.e. important buildings, cities, water courses), maps, reference data (i.e. Corine Land Cover).