In thermal face detection, researchers have generally assumed manual face detection or have designed algorithms that focus on indoor environment. However, facial properties are dependent on body temperature, surrounding environment, and any accessories or occlusion present on the face. For instance, the presence of scarfs, glasses, or any disguise accessories will alter the emitted heat pattern, thereby making it challenging to detect the face in thermal images. Similarly, daytime outdoor image acquisition has certain effects on the heat pattern compared to nighttime (or indoor controlled) image acquisition settings that affect automatic face detection performance. In this research, we provide a thorough understanding of challenges in thermal face detection along with an experimental evaluation of traditional approaches. Further, we adapt the AdaBoost face detector to yield improved performance on face detection in thermal images in both indoor and outdoor environments. We also propose a region of interest selection approach designed specifically for aiding occluded/disguised thermal face detection. Experiments are performed on the Notre Dame thermal face database as well as the IIITD databases that include variations such as disguise, age, and environmental (day/night) factors. The results suggest that while thermal face detection in semi-controlled environments is relatively easy, occlusion and disguise are challenges that require further attention. © Springer International Publishing Switzerland 2016.