Taking attendance in a large class is cumbersome, repetitive, and it consumes valuable class time. To avoid these problems, we propose an automatic attendance system using deep learning framework. An automatic attendance system based on the image processing consists of two steps: face detection and face recognition. Face detection and recognition are well-explored problems in computer vision domain, though they are still not solved due to large pose variations, different illumination conditions, and occlusions. In this work, we used state-of-the-art face detection model to detect the faces and a novel recognition architecture to recognize faces. The proposed face verification network is shallower than the state-of-the-art networks and it has achieved similar face recognition performance. we achieved 98.67% on LFW and 100% on classroom data. The classroom data was made by us for practical implementation of the complete network during this work. © Springer Nature Singapore Pte Ltd 2019.