In this article, a blockwise regression-based image inpainting framework is proposed. The core idea is to fill the unknown region in two stages: Extrapolate the edges to the unknown region and then fill the unknown pixels values in each sub-region demarcated by the extended edges. Canny edge detection and linear edge extension are used to respectively identify and extend edges to the unknown region followed by regression within each sub-region to predict the unknown pixel values. Two different regression models based on K-nearest neighbours and support vectors machine are used to predict the unknown pixel values. The proposed framework has the advantage of inpainting without requiring prior training on any image dataset. The extensive experiments on different images with contrasting distortions demonstrate the robustness of the proposed framework and a detailed comparative analysis shows that the proposed technique outperforms existing state-of-the-art image inpainting methods. Finally, the proposed techniques are applied to MRI images suffering from susceptibility artifacts to illustrate the practical usage of the proposed work. © 2021 Association for Computing Machinery.