Image Restoration and Enhancement for Visual Recognition
Can the application of enhancement algorithms as a pre-processing step improve image interpretability for manual analysis or automatic visual recognition to classify scene content?
To build a visual recognition system for any applicationthese days, one's first inclination is to turn to the most recent machine learning breakthrough from the area of deep learning, which no doubt has been enabled by access to millions of training images from the Internet. But there are many circumstances where such an approach cannot be used as an off-the-shelf component to assemble the system we desire, because even the largest training dataset does not take into account all of the artifacts that can be experienced in the environment. As computer vision pushes further into real-world applications, what should a software system that can interpret images from sensors placed in any unrestrictedsetting actually look like?
First, it must incorporate a set of algorithms, drawn from the areas of computational photography and machine learning, into a processing pipeline that corrects and subsequently classifies images across time and space. Image restoration and enhancement algorithms that remove corruptions like blur, noise, and mis-focus, or manipulate images to gain resolution, change perspective, and compensate for lens distortion are now commonplace in photo editing tools. Such operations are necessary to improve the quality of raw images that are otherwise unacceptable for recognition purposes. But they must be compatible with the recognition process itself, and not adversely affect feature extraction or classification.
Remarkably, little thought has been given to image restoration and enhancement algorithms for visual recognition - the goal of computational photography thus far has simply been to make images look appealing after correction. It remains unknown what impact many transformations have on visual recognition algorithms. To begin to answer that question, exploratory work is needed to find out which image pre-processing algorithms, in combination with the strongest features and supervised machine learning approaches, are promising candidates for different problem domains.
Funding: IARPA contract #2016-16070500002
Rosaura G. VidalMata, Sreya Banerjee, Walter Scheirer
Collaborators: Klemen Grm, Vitomir Struc
UG^2: a Video Benchmark for Assessing the Impact of Image Restoration and Enhancement on Automatic Visual Recognition, Rosaura Vidal Mata*, Sreya Banerjee*, Klemen Grm, Vitomir Struc, Walter J. Scheirer, Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), March 2018: [arxiv] [pdf]
*denotes equal contribution
Bridging the Gap Between Computational Photography and Visual Recognition, Rosaura G VidalMata, Sreya Banerjee, Brandon RichardWebster, Michael Albright, Pedro Davalos, Scott McCloskey, Ben Miller, Asong Tambo, Sushobhan Ghosh, Sudarshan Nagesh, Ye Yuan, Yueyu Hu, Junru Wu, Wenhan Yang, Xiaoshuai Zhang, Jiaying Liu, Zhangyang Wang, Hwann-Tzong Chen, Tzu-Wei Huang, Wen-Chi Chin, Yi-Chun Li, Mahmoud Lababidi, Charles Otto, Walter J Scheirer, January 28, 2019 [arxiv]
[dataset] [code] [CVPR workshop] [poster]
