Inverse problems in Imaging

Inverse Problems in Imaging

Fall semester, 2015

PhD program on Electrical and Computer Engineering
Department of Electrical and Computer Engineering,

Aim
Intended Learning Outcomes
Content
Recommended Texts
Prerequisites
Courseload
Assessment
Contact

Signal and image reconstruction from degraded, noise corrupted, and "blurred'' versions of the original signal/image play a key role in a number of engineering problems in remote sensing and in many imaging modalities (medical, astronomical, radar, sonar, etc.) This course addresses signal/image reconstruction under the unifying framework of inverse problems, with emphasis on the Bayesian and the regularization frameworks.

Intended Learning Outcomes

Understand the basic theory for ill-posed problems and its application to a number of engineering problems such as image deconvolution, image denoising, image decomposition, compressive sensing, and computed imaging (tomography, magnetic resonance, and radar imaging).
Grasp bibliography on inverse problems published in the area journals [e.g., Inverse problems, Journal of Inverse and Ill-posed Problems, Inverse Problems in Engineering, Inverse problems and Imaging, and IEEE Transactions (Signal Processing, Image Processing, Medical Imaging, and Geoscience and Remote Sensing)].

Content

Number of lectures	Syllabus	References
2	Introduction to inverse problems Examples of direct (forward) problems: deterministic and statistical point of view Ill-posed and ill-conditioned problems; Anatomy of linear transforms The deconvolution problem; An Illustrative example; Tikhonov regularization; Truncated Fourier decompossition (TFD); Regularization parameter selection; Generalized Tikhonov regularization; Bayesian perspective Iterative optimization	[Chp 1., RB1,RB2,L1] [App. D, RB1], [PS1]
2	Ill-Posedness and regularization of linear operators in function spaces Compact operators; Singular value decomposition (SVD); Least squares solution; Moore-Penrose pseudo inverse; Geometry of a linear inverse problem; Classification of linear operators; Minimization of quadractic functionals Regularization theory Tickhonov regularization; Truncated SVD; Generalized Tikhonov /Bayesian regularization; regularizers/priors	[Chp,2, RB2, L1], [App. E, RB1] Bacground: [App. A,B,C,E,F, RB1]
4	Review of probability theory Statistical Inference Maximum likelihood estimation: Properties; The exponential family; The Expectation Maximization algorithm Bayesian estimation; Bayes risk; loss functions; Bayes rules; Conjugate priors; Non-informative priors;	[Chp 4., RB1], [Chp. 9-12, AR2], [PM1]
2	Examples of inverse problems Linear observation mechanism Image denoising; Image deblurring; Image decomposition X-ray tomography; Emission tomography; Magnetic resonance imaging; Radar imaging Model fitting; Sparse representation; Compressed sensing Nonlinear observation mechanism Ultrasound imaging; Interferometric imaging	[Chp 3, 8, 11, RB1], [PCS1], [PT1]
4	Nonsmooth convex regularizers Total variation regularization Wavelet-based regularization	[PT1], [PW1], [PW3]
4	Optimization algorithms Optimization theory Iterative algorithms for smooth convex objective functions Iterative algorithms for nonsmooth convex objective functions Nonnegative constraints	[Ch3,. RB2], [P01,P03, P04
4	Regularization parameter selection methods Unbiased predictive risk Generalized cross-validation L-Curve Bayesian approach	[Chp. 7, RB2], [PT2]

Recomended Texts

     [RB2] C. Vogel, Computational Methods for Inverse problems, SIAM, 2002

               See C. Voguel's web page for the Solutions to the Exercices of Chapter 1 and for matlab codes

     [RB1] Bertero & Boccacci, Introduction to Inverse Problems in Imaging, IoP, 1998

Additional Reading

[AR2] L. Wasserman, All of Statistis. A Concise Course in Statistical Inference, Springer, 2004

[AR1] P. Hansen, Rank-Deficient and Discrete Ill-Posed problems, SIAM, 1998

Papers

Ultrasound

[[PU1] J. Sanches, J. Bioucas-Dias, and J. Marques, "Minimum total variation in 3D ultrasound reconstruction'',
IEEE International Conference on Image Processing-ICIP'05, 2005.

Interferometry

[PI1] J. Dias and J. Leitão, "The ZpiM algorithm for interferometric image reconstruction in SAR/SAS'',
IEEE Transactions on Image processing, vol 11, no. 4, pp. 408-422, 2002.

Total Variation

[PT2] J. Bioucas-Dias, M. Figueiredo, J. Oliveira, "Adaptive total-variation image deconvolution: A majorization-minimization
approach'', Proceedings of EUSIPCO'2006, Florence, Italy, September, 2006.

[PT1] A. Chambolle, "An algorithm for total variation minimization and applications,''
Journal of Mathematical Imaging and Vision, vol. 20, pp. 89-97, 2004.

Wavelets

[PW3] M. Figueiredo and R. Nowak, ''An EM algorithm for wavelet-based image restoration,
IEEE Transactions on Image Processing, vol. 12, no. 8, pp. 906-916, 2003.

[PW2] I. Daubechies, M. Defriese, and C. De Mol, ''An iterative thresholding algorithm for linear inverse problems with a sparsity
constraint'', Communications on Pure and Applied Mathematics, vol. LVII, pp. 1413- 1457, 2004.

[PW1] J. Bioucas-Dias, "Bayesian wavelet-based image deconvolution: a GEM algorithm exploiting a class of heavy-tailed priors'',
IEEE Transactions on Image processing, vol. 15, no. 4. pp. 937-951, 2006.

Compressed Sensing

        [PCS2] E. Candès, "Compressive sampling'', in Proceedings of the International Congress of Mathematics,
                     3, pp. 1433-1452, Madrid, Spain, 2006.

        [PCS1] E. Candès and J. Romberg, "Practical signal recovery from random projections'',
                    in SPIE International Symposium on Electronic Imaging: Computational Imaging III, San Jose, California, January, 2005.

Sampling/DFT/FFT

[PS4] P. J. S. G. Ferreira. The eigenvalues of matrices that occur in certain interpolation problems.
IEEE Trans. Signal Processing, vol. 45, no. 8, pp. 2115-2120, Aug. 1997.

[PS3] P. J. S. G. Ferreira. Interpolation in the time and frequency domains. IEEE Sig. Proc. Letters, vol. 3, no. 6, pp. 176-178, June 1996.

[PS2] P. J. S. G. Ferreira. Incomplete sampling series and the recovery of missing samples from oversampled band-limited signals.
IEEE Trans. Signal Processing, vol. 40, no. 1, pp. 225-227, Jan. 1992.

[PS1] D.R. Deller Jr, "Tom, Dick and Mary discover the FFT", IEEE Signal Processing Magazine, April 1994.

Optimization

[BO6] S. Boyed and L. Vanderberghe, Convex Optimization, Coambridge University Press, 2004

         [PO5] M. Elad, B. Matalon, and M. Zibulevsky, "Coordinate and Subspace Optimization Methods for Linear
                    Least Squares with Non-Quadratic Regularization'', to appear in the Journal on Applied and
                    Computational Harmonic Analysis, 2007.

[PO4] J. Bioucas-Dias and M. Figueiredo, "A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration",
Submitted to IEEE Transactions on Image processing, 2007.

[PO3] M. Figueiredo, J. Bioucas-Dias, and R. Nowak, "Majorization-Minimization Algorithms for Wavelet-Based Image
Deconvolution'', Submitted to IEEE Transactions on Image processing, 2006.

[PO2] J. Bioucas-Dias, M. Figueiredo, J. Oliveira, "Total variation image deconvolution: A majorization-minimization approach",
IEEE International Conference on Acoustics, Speech, and Signal Processing - ICASSP'2006, Toulouse, France, May 2006.

[PO1] D. Hunter and K. Lange. “A tutorial on MM algorithms.” The American Statistician, vol. 58, pp. 30–37, 2004.

Mixture models

[PM1] M. Figueiredo and A.K.Jain, "Unsupervised learning of finite mixture models", IEEE Transactions
on Pattern Analysis and Machine Intelligence, v.24 n.3, p.381-396, March 2002

Seminal works

       [WS3] A. Tikhonov, "Regularization of incorrectly posed problems'', Soviet Mathematics Doklady,
                    vol. 4 , pp. 1624-1627, 1963

       [WS2] L. Philips, `"A technique for the numerical solution of certain integral equations
                  of the first kind'', Journal of the Association for Computing Machinery, vol. 9, pp. 84-97, 1962

        [WS1] J. Hadamard, Lectures on Cauchy's Problem in Linear Partial Differential Equations
                  New Haven, CT, Yale University Press, 1923.

Usefull Links

   [L1] S. Tan, C. Fox, and G. Nicholls, Physics 707 (Inverse Problems Course) Lecture Notes, The University of Auckland
    [L2] Compressed Sensing Resources
    [L3] K. Peterson and M. Pederson, The Matrix Cookbook, 2008

Prerequisites

Basic course in Linear Algebra.
Familiarity with Fourier analysis including Fourier transform, discrete time Fourier transform, Fourier series, fast Fourier transform, and convolution.
Basic course in probability.
Proficiency in Matlab as it will be used to simulate data acquisition and to implement, test , and characterize many restoration algorithms presented in the course.

Courseload

The course is composed by 26 lectures, one hour and half each .

Assessment

The final grade is based on two components:

1) Coursework (60%)

2) Report: the student should produce, present, and defend a report about a specific theme (40%)

Contact

José Manuel Bioucas Dias
Instituto de Telecomunicações
Tel: 21 8418466
e-mail: bioucas@lx.it.pt