The performance of score-level fusion algorithms is often affected by conflicting decisions generated by the constituent matchers/classifiers. This paper describes a fusion algorithm that incorporates the likelihood ratio test statistic in a support vector machine (SVM) framework in order to classify match scores originating from multiple matchers. The proposed approach also takes into account the precision and uncertainties of individual matchers. The resulting fusion algorithm is used to mitigate the effect of covariate factors in face recognition by combining the match scores of linear appearance-based face recognition algorithms with their non-linear counterparts. Experimental results on a heterogeneous face database of 910 subjects suggest that the proposed fusion algorithm can significantly improve the verification performance of a face recognition system. Thus, the contribution of this work is two-fold: (a) the design of a novel fusion technique that incorporates the likelihood ratio test-statistic in a SVM fusion framework; and(b) the application of the technique to face recognition in order to mitigate the effect of covariate factors.