Multifocus image fusion is the process of obtaining a single image from multiple partially focused images such that the newly formed image consists of the well-defined information extracted from each source image. This paper proposes the use of saliency of the source images based on Mutual Spectral Residual. Spectral Residual brings out the unique/salient features of an image in frequency domain. The idea of proposed mutual spectral residual is to emphasize the relative unique features of a source image with respect to the other source images. The relative unique features are utilized to form saliency maps for each source image. These saliency maps can clearly indicate the focused and defocused parts of an image. Based on the saliency maps obtained, the image fusion takes place in spatial domain. Visual inspection and quantitative evaluation of the fused images obtained by the proposed method, using different evaluation metrics, demonstrate its effectiveness over several existing image fusion methods. © 2013 Elsevier Inc. All rights reserved.