Enhancing Photographs using Content-Specific Image Priors

Neel Joshi
Department of Computer Science and Engineering
University of California, San Diego

Ph.D. Dissertation

September 2008


(left) PSF-Estimation using Sharp Edge Prediction, (middle) Image Enhancement using Color Statistics, and (right) Image Correction using Identity-Specific Priors.



The digital imaging revolution has made the camera ubiquitous; however, image quality has not improved at the same rate as the increase in camera availability. Increasingly more cameras are small, with inexpensive lenses, no flash, and lightweight bodies that are difficult to hold steady, and this results in images with blur, noise, and poor color-balance. Consequently, there is a strong need for simple, automatic, and accurate methods for image correction. This dissertation, presents work that uses "content-specific" image models and priors for image enhancement.

Image enhancement is a challenge problem -- corrections such as deblurring, denoising, and color-correction are ill-posed, where the number of unknown values outweighs the number of observations. As a result, it is necessary to add additional information as constraints. Previous work has focused on using generic image priors that are applicable to a large number of images. In this work, we develop constraints that are tuned to the specific content of an image.

First, we discuss a fast, accurate blur estimation method that models all edges in a sharp image as step-edges. The method predicts the "sharp" version of a blurry input image and uses the two images together to solve for a PSF.

Second, we discuss a framework for image deblurring and denoising that uses local color statistics to produce sharp, low-noise results. Even when the blur function is known, deblurring an image is still quite difficult due to information loss during blurring and due to the presence of noise. In our work, we investigate using local-color statistics of an image in a joint framework for deblurring and denoising of images.

Lastly, we discuss work in methods that use "identity-specific" priors to perform corrections for images containing faces. These priors provide the guidance needed to perform high-quality corrections needed for known, familiar faces. Deblurring, super-resolution, color-balancing, and exposure correction operate independently, so that a user can correct selected image properties, while still retaining certain desired qualities of the original photo. We have also developed a prototype application for performing these corrections.


Table of Contents

Front matter (Vita, TOC, abstract, etc.)
Chapter 1 - Introduction
Chapter 2 - Previous Work
Chapter 3 - PSF Estimation using Sharp Edge Prediction
Chapter 4 - Image Enhancement using Color Statistics
Chapter 5 - Image Correction using Identity-Specific Priors
Chapter 6 - Conclusions and Future Work

Full Dissertation

Full dissertation (high quality - 72.8 MB)
Full dissertation (low quality - 7.08 MB)

Supplemental Results

Chapter 4 [pdf 8.75 MB]
Chapter 5 [webpage]
Chapter 6 [webpage] and [video -- divx 16.4 MB]

Copyright 2008 by Neel Joshi

Patents held and pending. Contact Neel Joshi for inquiries about the use of images or videos.

njoshi AT cs DOT ucsd DOT edu

Last update: September 17th, 2008.