Advancing state of the art in face recognition and bridging the gap between laboratory and real-world scenarios require the availability of challenging databases. One of the challenging applications of face recognition is surveillance, where unconstrained video data is captured both in day and night time (visible and near infrared spectrum). These videos have multiple subjects in each frame, which are matched with good quality gallery images. Due to the lack of an existing database for such a cross spectral cross resolution video-to-still face recognition application, this is still an open research problem. This paper presents a video database that can be utilized to benchmark face recognition algorithms addressing cross spectral cross resolution matching. The proposed Cross-Spectral Cross-Resolution Video dataset (CSCRV) contains videos pertaining to 160 subjects with an open-set protocol. We present baseline results with two commercial matchers for two experimental scenarios, where we observe very low performance of both the matchers. It is our assertion that this dataset can help researchers develop robust face recognition algorithms to handle real world surveillance scenarios. © 2016 IEEE.