Sentinel 1 and 2: Searching for Physical Image Content
Schwarz, Gottfried; Datcu, Mihai
ESA’s upcoming Sentinel-1 and Sentinel-2 missions open new perspectives for the application-oriented use of SAR and optical multispectral images. We expect short and regular revisit times as well as easily available and well documented products with attractive features such as cross-polarized SAR images and optical images delivered, for instance, as spectral reflectance data.
Thus, users do not have to live any longer with simple digital units or detector counts; instead, the data provided as Sentinel products can be understood as samples of calibrated and validated physical quantities. As a consequence, users can concentrate immediately on the physics and quantitative details of the observable phenomena.
This also affects content-based image retrieval, where a user searches for images containing phenomena being similar to given examples. While retrieval systems based on visible image data can only exploit characteristic shapes or patterns, the use of Sentinel data will address the determination of real physical relationships.
This physics-based approach will allow us to employ content-based image retrieval as an attractive tool for the analysis of SAR and optical images:
Within the framework of an existing content-based image retrieval system, we address the additional capabilities of image mining due to new Sentinel products. Typical image retrieval results from existing data sets will be used to demonstrate the capabilities and limitations of current missions. These results will be extrapolated to show what gain we expect when the Sentinel products become available. The gain the we expect will be described within the limitations of the quality levels of the Sentinel products and given by the conventions being available for quality assessments via precision/recall measurements .
Another step is the use of an image mining system for semantic auto-annotation of images. While image annotation in a pre-Sentinel ground system can be understood as attaching labels typically referring to image land cover and/or land use content of image sub-areas, image annotation in a Sentinel era ground segment can be extended to new labels related to the physics-related scattering behavior of objects in SAR images or reflectance phenomena in multispectral images.
In order to define a robust set of new labels, we have to define characteristic scenarios and train our image retrieval system accordingly. This definition and training stage has to be done efficiently; the selection of a global strategy is another point that will be discussed.
The discussion will be based on the experiences gained during the design phase of an image mining system under ESA contract (http://rssportal.esa.int/deepenandlearn/tiki-index.php?page=EOLIB+Project). This project serves to set up the next-generation of image information mining systems, implementing novel techniques for image content exploration. It will exploit information about Earth observation product contents which is usually hidden in raster data, image time series and metadata, thus enabling content-based search in very large archives of high resolution image data. The developed software will be interfaced to and operated in a multi-mission payload ground segment.