Tracking of a fish or some specific fishes in a school of fish is quite a challenging task. This could help in understanding the behavior of a fish or a small group of fish in a crowd of different varieties of fishes. In this paper we propose a technique to detect prominent objects among a large group of fishes. The problem is formulated with a stationary camera setup. The moving objects are initially detected by a spatio-contextual Gaussian mixture model based background subtraction method. Further, all the detected objects are analyzed to determine a predefined number of the most prominent objects in the scene of view. To characterize the objects we have employed a dual-feature framework, which includes color and texture features. The overall feature strength is computed by combining the two feature-strengths in an adaptive way so that, the color gets more weight if color degradation is less otherwise texture gets more weight. This weight is adaptively computed with the prior information of color degradation phenomena in underwater environment. The proposed technique is tested with a large number of underwater videos and found to perform satisfactorily. © 2020 IEEE.