Department of Computer Science & Engineering

&ldquo;IIT Jodhpur is committed to impart quality education to facilitate the creation of qualified, competent and responsible human resources to meet the emerging technological challenges of the world. It is working towards creating an equitable learning platform that is positive, accessible and effective in the face of the modern societal and scientific advancements.

Flexibility in curriculum, in all its academic programs, encompassing elements of teaching, research, creativity, innovation and entrepreneurship is the most unique feature of IIT Jodhpur. The Department of Science and Technology (DST), under the National Mission on Cyber Physical Systems, has identified IIT Jodhpur as a Technology &amp; Innovation Hub in the area of Augmented Reality and Virtual Reality. The institute has also been nominated as the coordinator for the Jodhpur City and Knowledge Cluster by the Office of Principal Scientific Adviser, Government of India.

The sprawling 852-acre IIT Jodhpur campus is a majestic tribute to the deep-rooted diverse cultural heritage of the region. The campus is an architectural masterpiece, which is slated to become an international exemplar of sustainability with net-zero energy, and advanced water and waste management strategies.&rdquo;

IIT Jodhpur

Visual object tracking is one of the major challenges in the field of computer vision. Correlation Filter (CF) trackers are one of the most widely used categories in tracking. Though numerous tracking algorithms based on CFs are available today, most of them fail to efficiently detect the object in an unconstrained environment with dynamically changing object appearance. In order to tackle such challenges, the existing strategies often rely on a particular set of algorithms. Here, we propose a robust framework that offers the provision to incorporate illumination and rotation invariance in the standard Discriminative Correlation Filter (DCF) formulation. We also supervise the detection stage of DCF trackers by eliminating false positives in the convolution response map. Further, we demonstrate the impact of displacement consistency on CF trackers. The generality and efficiency of the proposed framework is illustrated by integrating our contributions into two state-of-the-art CF trackers: SRDCF and ECO. As per the comprehensive experiments on the VOT2016 dataset, our top trackers show substantial improvement of $$14.7\%$$ and $$6.41\%$$ in robustness, $$11.4\%$$ and $$1.71\%$$ in Average Expected Overlap (AEO) over the baseline SRDCF and ECO, respectively. © 2019, Springer Nature Switzerland AG.

World University root node

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Learning Rotation Adaptive Correlation Filters in Robust Visual Object Tracking

Taking attendance in a large class is cumbersome, repetitive, and it consumes valuable class time. To avoid these problems, we propose an automatic attendance system using deep learning framework. An automatic attendance system based on the image processing consists of two steps: face detection and face recognition. Face detection and recognition are well-explored problems in computer vision domain, though they are still not solved due to large pose variations, different illumination conditions, and occlusions. In this work, we used state-of-the-art face detection model to detect the faces and a novel recognition architecture to recognize faces. The proposed face verification network is shallower than the state-of-the-art networks and it has achieved similar face recognition performance. we achieved 98.67% on LFW and 100% on classroom data. The classroom data was made by us for practical implementation of the complete network during this work. © Springer Nature Singapore Pte Ltd 2019.

Advances in Intelligent Systems and Computing

Automatic attendance system using deep learning framework

The amount of user-generated content in the cyberspace keeps increasing in the 21st century. However, it has also meant an increase in the number of cyber abuse and bullying incidents being reported. Use of profane text by individuals threatens the liberty and integrity of the digital space. Manual moderation and reporting mechanisms have been traditionally used to keep a check on such profane text. Dependency on human interpretation and delay in results have been the biggest obstacles in this system. Previous deep learning-based approaches to automate the process have involved use of traditional convolution and recurrence based sequential models. However, these models tend to be computationally expensive and have higher memory requirement. Further, they tend to produce state of the art results in binary classification but perform relatively poorly on multilabel tasks, owing to less flexibility in architecture. In today's world, classifying text in a binary way is no longer sufficient and thus a flexible solution able to generalize well on multilabel text is the need of the hour. In this paper, we propose a multihead attention-based approach for detection of profane text. We couple our model with power weighted average ensembling techniques to further improve the performance. The proposed approach does not have additional memory requirement and is less complex as compared to previous approaches. The improved results obtained by our model on publicly available real-world data further validate the same. Flexible, lightweight models which can handle multilabel text well can prove to be crucial in cracking down on social evils in the digital space. © 2019 IEEE.

IEEE International Conference on Data Mining Workshops, ICDMW

An attention ensemble based approach for multilabel profanity detection

The Visual Object Tracking challenge VOT2018 is the sixth annual tracker benchmarking activity organized by the VOT initiative. Results of over eighty trackers are presented; many are state-of-the-art trackers published at major computer vision conferences or in journals in the recent years. The evaluation included the standard VOT and other popular methodologies for short-term tracking analysis and a “real-time” experiment simulating a situation where a tracker processes images as if provided by a continuously running sensor. A long-term tracking subchallenge has been introduced to the set of standard VOT sub-challenges. The new subchallenge focuses on long-term tracking properties, namely coping with target disappearance and reappearance. A new dataset has been compiled and a performance evaluation methodology that focuses on long-term tracking capabilities has been adopted. The VOT toolkit has been updated to support both standard short-term and the new long-term tracking subchallenges. Performance of the tested trackers typically by far exceeds standard baselines. The source code for most of the trackers is publicly available from the VOT page. The dataset, the evaluation kit and the results are publicly available at the challenge website (http://votchallenge.net). © Springer Nature Switzerland AG 2019.

The sixth visual object tracking VOT2018 challenge results

Convolutional neural networks (CNNs) are one of the deep learning architectures capable of learning complex set of nonlinear features useful for effectively representing the structure of input to the network. Existing CNN architectures are invariant to small distortions, translations, scaling but are sensitive to rotations. In this paper, unlike the approaches where training samples with different orientations are included, we propose a new architecture in which addition of a rotational invariant map gives few folds of improvement towards rotational invariance to the network. We also propose an improved architecture where rotational invariance is achieved by rotationally varying the convolutional maps. We show that the proposed methods give better invariance towards rotations as compared to conventional training of CNN architecture (where the network is trained without considering the different orientation of training samples). The methods achieve rotation-independent classification by introducing few modifications in conventional CNNs, but do not add any trainable parameter to the network, thus keeping the number of free parameters/weights constant. We demonstrate the performance of proposed rotation invariant architectures for handwritten digits and texture data sets. © 2018, Springer-Verlag London Ltd., part of Springer Nature.

Pattern Analysis and Applications

Incorporating rotational invariance in convolutional neural network architecture

Restoring the desired performance of a pruned model requires a fine-tuning step, which lets the network relearn using the training data, except that the parameters are initialised to the pruned parameters. This relearning procedure is a key component in deciding the time taken in building a hardware-friendly architecture. This paper analyses the fine-tuning or retraining step after pruning the network layer-wise and derives lower bounds for the number of epochs the network will take based on the amount of pruning done. Analyses on the propagation of errors through the layers while pruning layer-wise is also performed and a new parameter named ‘Net Deviation’ is proposed which can be used to estimate how good a pruning algorithm is. This parameter could be an alternative to ‘test accuracy’ that is normally used. Net Deviation can be calculated while pruning, using the same data that was used in the pruning procedure. Similar to the test accuracy degradation for different amounts of pruning, the net deviation curves help compare the pruning methods. As an example, a comparison between Random pruning, Weight magnitude based pruning and Clustered pruning is performed on LeNet-300-100 and LeNet-5 architectures using Net Deviation. Results indicate clustered pruning to be a better option than random approach, for higher compression. © 2019, Springer Nature Switzerland AG.

Retraining Conditions: How Much to Retrain a Network After Pruning?

Deep metric learning algorithms have been utilized to learn discriminative and generalizable models which are effective for classifying unseen classes. In this paper, a novel noise tolerant deep metric learning algorithm is proposed. The proposed method, termed as Density Aware Metric Learning, enforces the model to learn embeddings that are pulled towards the most dense region of the clusters for each class. It is achieved by iteratively shifting the estimate of the center towards the dense region of the cluster thereby leading to faster convergence and higher generalizability. In addition to this, the approach is robust to noisy samples in the training data, often present as outliers. Detailed experiments and analysis on two challenging cross-modal face recognition databases and two popular object recognition databases exhibit the efficacy of the proposed approach. It has superior convergence, requires lesser training time, and yields better accuracies than several popular deep metric learning methods. © 2019 IEEE.

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

On learning density aware embeddings

Deforestation and loss of habitat have resulted in rapid decline of certain species of primates in forests. On the other hand, uncontrolled growth of a few species of primates in urban areas has led to safety issues and nuisance for the local residents. Hence, identifying individual primates has become the need of the hour - not only for conservation and effective mitigation in the wild but also in zoological parks and wildlife sanctuaries. Primates and human faces share a lot of common features like position and shape of eyes, nose and mouth. It is worth exploring whether the knowledge of human faces and recent methods learned from human face detection and recognition can be extended to primate faces. However, similar challenges relating to bias in human faces will also occur in primates. The quality and orientation of primate images along with different species of primates - ranging from monkeys to gorillas and chimpanzees will contribute to bias in effective detection and recognition. Experimental results on a primate dataset of over 80 identities show the effect of bias in this research problem. © Springer Nature Switzerland AG 2019.

Exploring bias in primate face detection and recognition

Face recognition algorithms are generally vulnerable towards presentation attacks ranging from cost-effective ways such as print and replay to sophisticated mediums such as silicone masks. Carefully designed silicone masks have real-life face texture once wore and can exhibit facial motions; thereby making them challenging to detect. In the literature, while several algorithms have been developed for detecting print and replay based attacks, limited work has been done for detecting silicone mask-based attack. In this research, we propose a computationally efficient solution by utilizing the power of CNN filters, and texture encoding for silicone mask based presentation attacks. The proposed framework operates on the principle of binarizing the image region after convolving the region with the filters learned via CNN operations. On the challenging silicon mask face presentation attack database (SMAD), the proposed feature descriptor shows 3.8% lower error rate than the state-of-the-art algorithms. © 2019 IEEE.

2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems, BTAS 2019

CHIF: Convoluted Histogram Image Features for Detecting Silicone Mask based Face Presentation Attack

Blockchain has emerged as a leading technology that ensures security in a distributed framework. Recently, it has been shown that blockchain can be used to convert traditional blocks of any deep learning models into secure systems. In this research, we model a trained biometric recognition system in an architecture which leverages the blockchain technology to provide fault tolerant access in a distributed environment. The advantage of the proposed approach is that tampering in one particular component alerts the whole system and helps in easy identification of 'any' possible alteration. Experimentally, with different biometric modalities, we have shown that the proposed approach provides security to both deep learning model and the biometric template. © 2019 IEEE.