Department of Computer Science & Engineering

&ldquo;IIT Jodhpur is committed to impart quality education to facilitate the creation of qualified, competent and responsible human resources to meet the emerging technological challenges of the world. It is working towards creating an equitable learning platform that is positive, accessible and effective in the face of the modern societal and scientific advancements.

Flexibility in curriculum, in all its academic programs, encompassing elements of teaching, research, creativity, innovation and entrepreneurship is the most unique feature of IIT Jodhpur. The Department of Science and Technology (DST), under the National Mission on Cyber Physical Systems, has identified IIT Jodhpur as a Technology &amp; Innovation Hub in the area of Augmented Reality and Virtual Reality. The institute has also been nominated as the coordinator for the Jodhpur City and Knowledge Cluster by the Office of Principal Scientific Adviser, Government of India.

The sprawling 852-acre IIT Jodhpur campus is a majestic tribute to the deep-rooted diverse cultural heritage of the region. The campus is an architectural masterpiece, which is slated to become an international exemplar of sustainability with net-zero energy, and advanced water and waste management strategies.&rdquo;

IIT Jodhpur

Human Activity Recognition (HAR) has been a compelling problem in the field of computer vision since a long time. Our focus is to address the problem of trimmed activity recognition which is to identify the class of human activity in a video which is temporally trimmed to contain only those periods where human activity is present. In the past few years there has been a transition from handcrafted features for classification to deep convolutional neural networks which work on raw video data to extract features and classify human activities. 3D convolutional neural networks learn features from both the temporal as well as spatial dimensions and prove to be very powerful in finding correlations in signals containing spatiotemporal information. 3D-CNNs have been extremely successful in activity recognition. We explore the shortcomings of a 3D-CNN architecture and propose ensembling with a 2D-CNN to overcome these for a significantly better performance in activity recognition. © 2019 IEEE.

World University root node

IEEE Region 10 Annual International Conference, Proceedings/TENCON

ECNN: Activity Recognition Using Ensembled Convolutional Neural Networks

From water sprinklers to combustion devices of launch vehicles, the applications of the liquid injectors are innumerable. Owing to the usage of non-renewable sources, there is a global movement towards improvement in the existing combustion systems. Efficiency and fuel emissions are influenced by fuel spray characteristics. One of the techniques used to capture the information on spray characteristics is highspeed shadowgraphy. Estimation of the required parameters needs analysis of image series. An automated method to replace tiring manual calculations makes it the job of a few seconds. The basic tools of image processing are employed in this study to ease the computation. © 2019 IEEE.

2019 6th International Conference on Signal Processing and Integrated Networks, SPIN 2019

Application of Digital Image Processing Method for Spray Characterization

Correlation filter based object tracking has recently gained popularity due to continuous improvements in the tracking accuracy and robustness. However, these trackers are limited by the model drift problem due to wrong target appearances learned from an incorrectly tracked frame. The model drift increases as more frames are processed and restricts the ability of long term tracking in correlation filter trackers. The proposed method introduces tracking resumption in correlation trackers using a detector mechanism that re-initializes the tracker upon a target loss identified using an adaptive threshold. Online training of both the tracker and detector modules incorporates temporal information into the proposed framework, making it robust to appearance changes of the object. The tracker and detector stages complement each other in correcting the false appearances learned from any frame, thereby mitigating the model-drift problem. A similarity matching technique estimates the final target location. Extensive experimental analysis on benchmark datasets indicate that the proposed tracker is well suited for robust long-term tracking and is superior to other state of the art methods both qualitatively and quantitatively. © 2019 Elsevier B.V.

Pattern Recognition Letters

Detection based long term tracking in correlation filter trackers

In this paper, we propose attention maps at various scales on multi-resolution feature extractor baseline network for human pose estimation. The baseline network captures information across various scales with the help of repeated bottom-up and top-down approach using successive pooling and up-sampling. We propose a network named Refinement Net for regressing the predicted heatmaps to 2D joint locations to remove ambiguities in predicted position. We experiment with three levels of attention schemes - global, heatmap and multi-resolution. Attention masks helps in generating basin of attraction that helps the network on deciding where to “look”. The proposed network performance is at par with the state-of-the-art two dimensional pose estimation methods on MPII dataset. © 2019, Springer Nature Switzerland AG.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Multi-scale Attention Aided Multi-Resolution Network for Human Pose Estimation

This paper gives the design and implementation of Xilinx FPGA based Forward Error Correction (FEC) encoder for DVB S2 system which includes BCH code followed by LDPC code and finally bit mapped to constellation for QPSK modulation. DVB-S2 FEC: (mathbf{n}=64800, mathbf{k}=32400) rate 1/2 code, with QPSK modulation scheme is considered as target for FPGA implementation. The architecture in this design efficiently uses pipeline technique along with parallel processing to optimize the hardware resources and overall latency, to accomplish FEC encoding for DVB S2 system. Coding is completed in Verilog HDL with Xilinx Virtex6 XC6VLX240T FPGA as target for hardware realization and QuestaSim simulator is used to complete the functional simulation. © 2019 IEEE.

FPGA Implementation of FEC Encoder with BCH LDPC Codes for DVB S2 System

Tasks such as action recognition requires high quality features for accurate inference. But the use of high resolution and large volume of video data poses a significant challenge for inference in terms of storage and computational complexity. In addition, compressive sensing as a potential solution to the aforementioned problems has been shown to recover signals at higher compression ratios with loss in information. Hence, a framework is required that performs good quality action recognition on compressively sensed data. In this paper, we present data-driven sensing for spatial multiplexers trained with combined mean square error (MSE) and perceptual loss using Deep convolutional neural networks. We employ subpixel convolutional layers with the 2D Convolutional Encoder-Decoder model, that learns the downscaling filters to bring the input from higher dimension to lower dimension in encoder and learns the reverse, i.e. upscaling filters in the decoder. We stack this Encoder with Inflated 3D ConvNet and train the cascaded network with cross-entropy loss for Action recognition. After encoding data and undersampling it by over 100 times (10 @ 10) from the input size, we obtain 75.05% accuracy on UCF-101 and 50.39% accuracy on HMDB-51 with our proposed architecture setting the baseline for reconstruction free action recognition with data-driven sensing using deep learning. We experimentally infer that the encoded information from such spatial multiplexers can directly be used for action recognition. © 2019, Springer Nature Switzerland AG.

Department of Chemistry

Data Driven Sensing for Action Recognition Using Deep Convolutional Neural Networks

In real-world applications, commercial off-the-shelf systems are utilized for performing automated facial analysis including face recognition, emotion recognition, and attribute prediction. However, a majority of these commercial systems act as black boxes due to the inaccessibility of the model parameters which makes it challenging to fine-tune the models for specific applications. Stimulated by the advances in adversarial perturbations, this research proposes the concept of Data Fine-tuning to improve the classification accuracy of a given model without changing the parameters of the model. This is accomplished by modeling it as data (image) perturbation problem. A small amount of “noise” is added to the input with the objective of minimizing the classification loss without affecting the (visual) appearance. Experiments performed on three publicly available datasets LFW, CelebA, and MUCT, demonstrate the effectiveness of the proposed concept. © 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019

Data fine-tuning

Disguised face recognition has wide-spread applicability in scenarios such as law enforcement, surveillance, and access control. Disguise accessories such as sunglasses, masks, scarves, or make-up modify or occlude different facial regions which makes face recognition a challenging task. In order to understand and benchmark the state-of-the-art on face recognition in the presence of disguise variations, the Disguised Faces in the Wild 2019 (DFW2019) competition has been organized. This paper summarizes the outcome of the competition in terms of the dataset used for evaluation, a brief review of the algorithms employed by the participants for this task, and the results obtained. The DFW2019 dataset has been released with four evaluation protocols and baseline results obtained from two deep learning-based state-of-the-art face recognition models. The DFW2019 dataset has also been analyzed with respect to degrees of difficulty: (i) easy, (ii) medium, and (iii) hard. The dataset has been released as part of the International Workshop on Disguised Faces in the Wild at International Conference on Computer Vision (ICCV), 2019. © 2019 IEEE.

Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019

Disguised faces in the wild 2019

IIIT-Delhi

Deep Learning Approach for Face Recognition with Disguise Variations By Mohit Chawla

Modality refers to how something happens or is experienced. It is the representation format in which information is stored. Multimodal Learning involves relating information from multiple sources. Many modalities are combined at the training stage to learn better features for improving the performance during detection/classification. In this report, I have analyzed how the techniques in multimodal deep learning have advanced, and performance improved over the years. I have also performed a case study by running a model on a novel Multimodal Fake News Dataset. The improved performance on using multimodal features representation on this dataset compared to single modality text/image representations has also been observed. I have further created a novel multimodal algorithm for Fake News Detection. We have done an in-depth analysis by running it on certain Fake News datasets to show how this multimodal algorithm is an improvement over the other Fake News detection algorithms. We also performed an ablation analysis on our computed results and tried to visually conceptualize the results with the help of plots and diagrams.