In this paper we present a technique that estimates the Structure from Motion (SFM) in a recursive fashion. Traditionally successful SFM algorithms take the set of images and estimate the scene geometry and camera positions either using incremental algorithms or the global algorithms and do the refinement process [2] to reduce the reprojection error. In this work it is assumed that we don’t have complete image set at the start of the reconstruction process, unlike most of the traditional approaches present in the literature. It is assumed that the set of images come in at the regular intervals and we recursively perform the SFM on the incoming set of images and update the previously reconstructed structure with the structure estimated from the current set of images. The proposed system has been tested on two datasets which consist of 12 images and 60 images respectively and reconstructions obtained show the validity of our proposed technique. © Springer International Publishing AG 2017.