Several mathematical models have been proposed for recognizing face images with age variations. However, effect of change in body-weight is also an interesting covariate that has not been much explored. This paper presents a novel approach to incorporate the weight variations during feature learning process. In a deep learning architecture, we propose incorporating the body-weight in terms of a regularization function which helps in learning the latent variables representative of different weight categories. The formulation has been proposed for both Autoencoder and Deep Boltzmann Machine. On extended WIT database of 200 subjects, the comparison with a commercial system and an existing algorithm show that the proposed algorithm outperforms them by more than 9% at rank-10 identification accuracy. © 2015 IEEE.