Department of Computer Science & Engineering

&ldquo;IIT Jodhpur is committed to impart quality education to facilitate the creation of qualified, competent and responsible human resources to meet the emerging technological challenges of the world. It is working towards creating an equitable learning platform that is positive, accessible and effective in the face of the modern societal and scientific advancements.

Flexibility in curriculum, in all its academic programs, encompassing elements of teaching, research, creativity, innovation and entrepreneurship is the most unique feature of IIT Jodhpur. The Department of Science and Technology (DST), under the National Mission on Cyber Physical Systems, has identified IIT Jodhpur as a Technology &amp; Innovation Hub in the area of Augmented Reality and Virtual Reality. The institute has also been nominated as the coordinator for the Jodhpur City and Knowledge Cluster by the Office of Principal Scientific Adviser, Government of India.

The sprawling 852-acre IIT Jodhpur campus is a majestic tribute to the deep-rooted diverse cultural heritage of the region. The campus is an architectural masterpiece, which is slated to become an international exemplar of sustainability with net-zero energy, and advanced water and waste management strategies.&rdquo;

IIT Jodhpur

Tracking of a fish or some specific fishes in a school of fish is quite a challenging task. This could help in understanding the behavior of a fish or a small group of fish in a crowd of different varieties of fishes. In this paper we propose a technique to detect prominent objects among a large group of fishes. The problem is formulated with a stationary camera setup. The moving objects are initially detected by a spatio-contextual Gaussian mixture model based background subtraction method. Further, all the detected objects are analyzed to determine a predefined number of the most prominent objects in the scene of view. To characterize the objects we have employed a dual-feature framework, which includes color and texture features. The overall feature strength is computed by combining the two feature-strengths in an adaptive way so that, the color gets more weight if color degradation is less otherwise texture gets more weight. This weight is adaptively computed with the prior information of color degradation phenomena in underwater environment. The proposed technique is tested with a large number of underwater videos and found to perform satisfactorily. © 2020 IEEE.

World University root node

2020 Global Oceans 2020: Singapore - U.S. Gulf Coast

Prominent Object Detection in Underwater Environment using a Dual-feature Framework

Human activity recognition (HAR) targets the methodologies to recognize the different actions from a sequence of observations. Vision-based activity recognition is among the most popular unobtrusive technique for activity recognition. Caring for the elderly who are living alone from a remote location is one of the biggest challenges of modern human society and is an area of active research. The usage of smart homes with an increasing number of cameras in our daily environment provides the platform to use that technology for activity recognition also. The omnidirectional cameras can be utilized for fall detection activity which minimizes the requirement of multiple cameras for fall detection in an indoor living scenario. Consequently, two vision-based solutions have been proposed: one using convolutional neural networks in 3D-mode and another using a hybrid approach by combining convolutional neural networks and long short-term memory networks using 360-degree videos for human fall detection. An omnidirectional video dataset has been generated by recording a set of activities performed by different people as no such 360-degree video dataset is available in the public domain for human activity recognition. Both, the models provide fall detection accuracy of more than 90% for omnidirectional videos and can be used for developing a fall detection system for indoor health care. © 2020, Springer Nature Singapore Pte Ltd.

Advances in Intelligent Systems and Computing

Activity Recognition for Indoor Fall Detection in 360-Degree Videos Using Deep Learning Techniques

State-of-the-art deep learning models have achieved superlative performance across multiple computer vision applications such as object recognition, face recognition, and digits/character classification. Most of these models highly rely on the gradient information flows through the network for learning. By utilizing this gradient information, a simple gradient sign method based attack is developed to fool the deep learning models. However, the primary concern with this attack is the perceptibility of noise for large degradation in classification accuracy. This research address the question of whether an imperceptible gradient noise can be generated to fool the deep neural networks? For this, the role of sign function in the gradient attack is analyzed. The analysis shows that without-sign function, i.e. gradient magnitude, not only leads to a successful attack mechanism but the noise is also imperceptible to the human observer. Extensive quantitative experiments performed using two convolutional neural networks validate the above observation. For instance, AlexNet architecture yields 63.54% accuracy on the CIFAR-10 database which reduces to 0.0% and 26.39% when sign (i.e., perceptible) and without-sign (i.e., imperceptible) of the gradient is utilized, respectively.Further, the role of the direction of the gradient for image manipulation is studied. When an image is manipulated in the positive direction of the gradient, an adversarial image is generated. On the other hand, if the opposite direction of the gradient is utilized for image manipulation, it is observed that the classification error rate of the CNN model is reduced. On AlexNet, the error rate of 36.46% reduces to 4.29% when images of CIFAR-10 are manipulated in the negative direction of the gradient. To explore other enthusiastic results on multiple object databases, including CIFAR-100, fashion-MNIST, and SVHN, please refer to the full paper. © 2020 IEEE.

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

The role of 'sign' and 'direction' of gradient on the performance of CNN

In the tracking-by-detection approach of online multiple object tracking (MOT), a major challenge is how to associate object detections on the new video frame with previously tracked objects. Two important aspects that directly influence the performance of MOT are quality of detection and accuracy in data association. The authors propose an efficient and unified MOT framework for improved object detection, followed by enhanced object tracking. The object detection and tracking are considered as two independent functions in the tracking-by-detection paradigm. In this study, object detection accuracy has been increased by employing a faster region-based convolutional neural network (Faster R-CNN) modified with the feedback region proposals from the tracker. Target association is performed by the correlation filter-based Siamese CNN model, which finds the similarity score between the input image patches. The Siamese CNN is trained using a supervised hard sample mining strategy. An optical flow-based motion model is employed to predict the next probable location of the targets from the tracker and these region proposals are fed back to the classifier module of Faster R-CNN. The authors’ extensive analysis of publicly available MOT benchmark datasets and comparison with the state-of-the-art tracking methods demonstrate competitive tracking performance of the proposed MOT framework. © The Institution of Engineering and Technology 2020

Fulltext

IET Computer Vision

Harnessing feedback region proposals for multi-object tracking

Compressed sensing (CS) is one of the predominant tools used presently to explore the possibilities in accelerating cardiac and body Magnetic Resonance (MR) imaging for achieving shorter scans and accommodating a wider patient group. CS accomplishes this by manoeuvring the scanning hardware to make much fewer measurements and imposing sparsity of the MR image in a known basis on the reconstruction process. Prior works have adopted fixed transforms as well as dictionaries learned on image patches to incorporate the sparsifying basis. Despite the obvious merits of 1-D dictionary learning methods, they suffer from high computational and memory complexity when extended to higher patch sizes and are thus restricted to capturing only local features. Thus, there arises the need for an efficient framework to extend the advantages of dictionary learning to higher dimensional applications like dynamic MRI (dMRI) where there exists a strong correlation across successive frames. This work employs a tensor decomposition based dictionary learning approach to effectively extend CS to dMRI and exploit the temporal gradient (TG) sparsity, thereby retaining maximum spectral and temporal resolution at higher under-sampling factors. The proposed technique is experimentally validated to achieve significant improvement in reconstruction quality over the current state-of-art in dynamic MRI under distinct sampling trajectories and noisy conditions. Further, it facilitates faster reconstructions of the dMRI volume than existing methods, rendering it an ideal choice in critical scenarios which demand a swift diagnosis. © 2019. The copyright of this document resides with its authors.

30th British Machine Vision Conference 2019, BMVC 2019

Higher order dictionary learning for compressed sensing based dynamic MRI reconstruction

Convolutional Neural Networks (CNNs) are being used popularly for detecting and classifying objects. Rotational invariance is not guaranteed by many of the existing CNN architectures. Many attempts have been made to acquire rotational invariance in CNNs. Our approach ‘Eigenvector Orientation Corrected LeNet (EOCL)’ presents a simple method to make ordinary LeNet [1] capable of detecting rotated digits, and also to predict the relative angle of orientation of digits with unknown orientation. The proposed method does not demand any modification in the existing LeNet architecture, and requires training with digits having only single orientation. EOCL incorporates an ‘orientation estimation and correction’ step prior to the testing phase. Using Principal Component Analysis, we find the maximum spread direction (Principal Component) of each test sample and then align it vertically. We demonstrate the improvement in classification accuracy and reduction in test time achieved by our approach, on rotated-MNIST [2] and MNIST_rot_12k test datasets, compared to other existing methods. © 2020, Springer Nature Singapore Pte Ltd.

Eigenvector Orientation Corrected LeNet for Digit Recognition

Automatic kinship verification using face images involves analyzing features and computing similarities between two input images to establish kin-relationship. It has gained significant interest from the research community and several approaches including deep learning architectures are proposed. One of the law enforcement applications of kinship analysis involves predicting the kin image given an input image. In other words, the question posed here is: “given an input image, can we generate a kin-image?” This paper attempts to generate kin-images using Generative Adversarial Learning for multiple kin-relations. The proposed FamilyGAN model incorporates three information, kin-gender, kinship loss, and reconstruction loss, in a GAN model to generate kin images. FamilyGAN is the first model capable of generating kin-images for multiple relations such as parent-child and siblings from a single model. On the WVU Kinship Video database, the proposed model shows very promising results for generating kin images. Experimental results show 71.34% kinship verification accuracy using the images generated via FamilyGAN. © 2020, Springer Nature Switzerland AG.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

FamilyGAN: Generating Kin Face Images Using Generative Adversarial Networks

A well trained and generalized deep neural network (DNN) should be robust to both seen and unseen classes. However, the performance of most of the existing supervised DNN algorithms degrade for classes which are unseen in the training set. To learn a discriminative classifier which yields good performance in Zero-Shot Learning (ZSL) settings, we propose to generate an Over-Complete Distribution (OCD) using Conditional Variational Autoencoder (CVAE) of both seen and unseen classes. In order to enforce the separability between classes and reduce the class scatter, we propose the use of Online Batch Triplet Loss (OBTL) and Center Loss (CL) on the generated OCD. The effectiveness of the framework is evaluated using both Zero-Shot Learning and Generalized Zero-Shot Learning protocols on three publicly available benchmark databases, SUN, CUB and AWA2. The results show that generating over-complete distributions and enforcing the classifier to learn a transform function from overlapping to non-overlapping distributions can improve the performance on both seen and unseen classes. © 2020 IEEE

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Generalized zero-shot learning via over-complete distribution

While image generation and editing technologies such as Generative Adversarial Networks and Photoshop are being used for creative and positive applications, the misuse of these technologies to create negative applications including Deep-nude and fake news is also increasing at a rampant pace. Therefore, detecting digitally created and digitally altered images is of paramount importance. This paper proposes a hierarchical approach termed as DAD-HCNN which performs two-fold task: (i) it differentiates between digitally generated images and digitally retouched images from the original unaltered images, and (ii) to increase the explainability of the decision, it also identifies the GAN architecture used to create the image. The effectiveness of the model is demonstrated on a database generated by combining face images generated from four different GAN architectures along with the retouched images and original images from existing benchmark databases. © 2020 IEEE.

Detecting GANs and retouching based digital alterations via DAD-HCNN

Visual content has become the primary source of information, as evident in the billions of images and videos, shared and uploaded on the Internet every single day. This has led to an increase in alterations in images and videos to make them more informative and eye-catching for the viewers worldwide. Some of these alterations are simple, like copy-move, and are easily detectable, while other sophisticated alterations like reenactment based DeepFakes are hard to detect. Reenactment alterations allow the source to change the target expressions and create photo-realistic images and videos. While the technology can be potentially used for several applications, the malicious usage of automatic reenactment has a very large social implication. It is therefore important to develop detection techniques to distinguish real images and videos with the altered ones. This research proposes a learning-based algorithm for detecting reenactment based alterations. The proposed algorithm uses a multi-stream network that learns regional artifacts and provides a robust performance at various compression levels. We also propose a loss function for the balanced learning of the streams for the proposed network. The performance is evaluated on the publicly available FaceForen- sics dataset. The results show state-of-the-art classification accuracy of 99.96%, 99.10%, and 91.20% for no, easy, and hard compression factors, respectively. © 2020 IEEE.