In the tracking-by-detection approach of online multiple object tracking (MOT), a major challenge is how to associate object detections on the new video frame with previously tracked objects. Two important aspects that directly influence the performance of MOT are quality of detection and accuracy in data association. The authors propose an efficient and unified MOT framework for improved object detection, followed by enhanced object tracking. The object detection and tracking are considered as two independent functions in the tracking-by-detection paradigm. In this study, object detection accuracy has been increased by employing a faster region-based convolutional neural network (Faster R-CNN) modified with the feedback region proposals from the tracker. Target association is performed by the correlation filter-based Siamese CNN model, which finds the similarity score between the input image patches. The Siamese CNN is trained using a supervised hard sample mining strategy. An optical flow-based motion model is employed to predict the next probable location of the targets from the tracker and these region proposals are fed back to the classifier module of Faster R-CNN. The authors’ extensive analysis of publicly available MOT benchmark datasets and comparison with the state-of-the-art tracking methods demonstrate competitive tracking performance of the proposed MOT framework. © The Institution of Engineering and Technology 2020