Skeletal Bone age assessment is one of the routine radiological procedures performed by paediatricians and endocrinologists for investigating genetic disorders, developmental abnormalities and metabolic complications. In this process skeletal age is compared against child's chronological age to uncover discrepancies if any. Hand radiographs being the cheapest, reliable and widely used modality, are used to predict the bone age in children from 1-18 years of age. Conventional methods make use of atlases to predict the age which are time consuming, tedious and have problems of inter-observer variability. We propose an end to end approach which uses inception trained from scratch, achieves 80% accuracy in predicting age within 1 year from the ground truth. Further, attention maps are generated to explain what regions of the image, the model is focusing on while assessing the bone age and the heat maps thus generated match the features used by the radiologists while predicting manually. © 2020 IEEE.