In this paper we propose a novel algorithm for object tracking from Video images based on segmentation and Kernel based procedure. Many Kernel based object tracking algorithms have been developed during last few years. The computational complexity becomes very high in those kernel based techniques. In our proposed method the target localization problem is minimized using segmentation technique, instead of using mean shift tracking algorithm. Following segmentation technique the localization problem of target candidate gets minimized, and then comparing the target candidate with the target model by using Bhattacharya coefficient the object can easily be detected. So, the object can be tracked with less computational burden and more efficiently. The proposed algorithm is validated with an existing video sequence and another with a real time video sequence. © 2010 IEEE.