Image Filtering 2.0: Efficient Edge-Aware Filtering Techniques and Their Applications

Jiangbo Lu, Dongbo Min, Minh N. Do

Description

The last decade has witnessed the exciting development of efficient edge-aware filtering (EAF) techniques, stemming from different theories and principles. Thanks to their strong power in adaptively dealing with various visual signals as well as significant computational and implementation advantages, this kind of modern image filtering techniques have found a great variety of applications in image/video processing, computer vision and computer graphics. These applications range broadly from addressing the classical image denoising task to recently tackling fully connected high-order conditional random ?eld (CRFs) for accurate semantic image segmentation and labeling. In these applications, the EAF methods have been employed either to allow for local data adaptivity or to efficiently integrate global supports. In this tutorial, we will first present various state-of-the-art nonlinear EAF techniques, while revealing theoretical connections, new insights and generalization. Especially, we will focus on fast filtering approaches, including using the bilateral grid, color-line model, multipoint aggregation, domain transform, and recursive data propagation. Representative applications of these filtering techniques will be illustrated and discussed, including edge-preserving smoothing, color image denoising, stereo matching, depth map enhancement, stylization, detail enhancement, and image segmentation/matting. We will also cover recent research works that efficiently deal with a fully-connected or high-order Conditional Random Field (CRF) model using the efficient EAF algorithms (e.g. based on mean field approximation or quadratic programming).

After spending a large part of the tutorial on efficient filtering and applications, we will discuss a serious computational challenge faced by most cost volume filtering-based approaches (and also MRF-based approaches), i.e., the curse of the huge discrete space in labeling problems (e.g. dense stereo and optical flow estimation). Along this direction, we will present some recent works to tackle such research challenges e.g. using a randomized hypothesis sampling and propagation approach (e.g. PatchMatch) or importance sampling on the label space, and then introduce several issues that should be addressed for developing more powerful computational tools.

Finally, we will conclude this tutorial by suggesting some intriguing problems and exciting future directions that may drive this adaptive filtering paradigm further into a cost-effective, general-purpose tool for many applications, going beyond the recent attempts of high-order CRFs inference or the randomized search approaches.


Biography

Jiangbo Lu received the B.S. and M.S. degrees in electrical engineering from Zhejiang University, Hangzhou, China, in 2000 and 2003, respectively, and the Ph.D. degree in electrical engineering, Katholieke Universiteit Leuven, Leuven, Belgium, in 2009. From April 2003 to August 2004, he was with VIA-S3 Graphics, Shanghai, China, as a Graphics Processing Unit (GPU) Architecture Design Engineer. In 2002 and 2005, he conducted visiting research at Microsoft Research Asia, Beijing, China. Since October 2004, he has been with the Multimedia Group, Interuniversity Microelectronics Center, Leuven, Belgium, as a Ph.D. Researcher. Since September 2009, he has been working with the Advanced Digital Sciences Center, Singapore, which is a joint research center between the University of Illinois at Urbana-Champaign, Urbana, and the Agency for Science, Technology and Research (A*STAR), Singapore, where he is currently a Research Scientist. His research interests include multimedia signal processing, computer vision, visual computing, video coding, interactive multimedia applications and systems, GPU-based computing, and embedded computer vision. He is an Associate Editor for IEEE Transactions on Circuits and Systems for Video Technology (TCSVT). He received the 2012 TCSVT Best Associate Editor (BAE) Award.

Dongbo Min received the B.S., M.S. and Ph.D. degrees in electrical and electronic engineering from Yonsei University, Seoul, Korea, in 2003, 2005 and 2009, respectively. From 2009 to 2010, He worked with Mitsubishi Electric Research Laboratories (MERL) as a post-doctoral researcher, where he developed a prototype of 3D video system (3DTV). Since July 2010, he has been working with the Advanced Digital Sciences Center (ADSC) in Singapore, which was jointly founded by University of Illinois at Urbana-Champaign (UIUC) and the Agency for Science, Technology and Research (A*STAR), a Singapore government agency. His research interests include 3D computer vision, GPU-based real-time system, 3D modeling, hybrid sensor system, and computational photography. He is a member of the IEEE.

Minh N. Do is an Associate Professor in the Department of Electrical and Computer Engineering at the University of Illinois at Urbana-Champaign. His work covers image and multi-dimensional signal processing, wavelets and multiscale geometric analysis, computational imaging, as well as visual information representation, and has led to about 40 journal papers. He received a Silver Medal from the 32nd International Mathematical Olympiad in 1991, a University Medal from the University of Canberra in 1997, a Doctorate Award from the EPFL in 2001, a CAREER Award from the National Science Foundation in 2003, and a Young Author Best Paper Award from IEEE in 2008. He was named a Beckman Fellow at the Center for Advanced Study, UIUC, in 2006, and received of a Xerox Award for Faculty Research from the College of Engineering, UIUC, in 2007. He was an Associate Editor of the IEEE Transactions on Image Processing, and a member of the IEEE Technical Committees on Signal Processing Theory and Methods, and on Image, Video, and Multidimensional Signal Processing.