Optical Flow is a popular method of computer vision for motion estimation. In this paper, we present a refined optical flow estimation method. Central to our approach is exploiting contour information as most of the motion lies on the edges. Further, we have formulated it as sparse to dense motion estimation. Proposed method has been evaluated on challenging real life image sequences of KITTI and Fish4Knowledge database. Results demonstrate that method performs well in case of low contrast, highly cluttered background, dynamic background, occlusion and illumination change. © 2020, Springer Nature Singapore Pte Ltd.