With the increase in age, there are changes in skeletal structure, muscle mass, and body fat. For recognizing faces with age variations, researchers have generally focused on the skeletal structure and muscle mass. However, the effect of change in body fat has not been studied with respect to face recognition. In this paper, we incorporate weight information to improve the performance of face recognition with age variations. The proposed algorithm utilizes neural network and random decision forest to encode age variations across different weight categories. The results are reported on the Wholslt database prepared by the authors containing 1109 images from 110 individuals with age and weight variations. The comparison with existing state-of-the-art algorithms and commercial system on Wholslt and FG-Net databases shows that the proposed algorithm outperforms existing algorithms significantly. © 2014 IEEE.