Face recognition algorithms have been extensively researched for the last 3 decades or so. Even after years of research, the algorithms developed achieve practical success only under controlled environments. Their performance usually takes a dip under unconstrained scene conditions like the presence of background clutter, non-uniform illumination etc. This paper explores the contrast in performance of standard recognition algorithms under controlled and uncontrolled environments. It proposes a way to combine 3 subspace learning algorithms, namely Eigenfaces (1DPCA), 2 dimensional Principal Component Analysis (2DPCA) and Row Column 2DPCA (RC2DPCA) with a color-based segmentation approach in order to boost the recognition rates under unconstrained scene conditions. A series of steps are performed that extract all possible facial regions from an image, following which the algorithm segregates the largest candidate for a probable face, and puts a bounding box on the blob in order to isolate only the face. It was found that the proposed algorithms, formed by the combination of such segmentationmethods obtain a higher level of accuracy than the standard recognition techniques. Moreover, it serves as a general framework wherein much more robust recognition techniques could be combined to achieve boosted accuracies. © Springer International Publishing Switzerland 2016.