Department of Computer Science & Engineering

&ldquo;IIT Jodhpur is committed to impart quality education to facilitate the creation of qualified, competent and responsible human resources to meet the emerging technological challenges of the world. It is working towards creating an equitable learning platform that is positive, accessible and effective in the face of the modern societal and scientific advancements.

Flexibility in curriculum, in all its academic programs, encompassing elements of teaching, research, creativity, innovation and entrepreneurship is the most unique feature of IIT Jodhpur. The Department of Science and Technology (DST), under the National Mission on Cyber Physical Systems, has identified IIT Jodhpur as a Technology &amp; Innovation Hub in the area of Augmented Reality and Virtual Reality. The institute has also been nominated as the coordinator for the Jodhpur City and Knowledge Cluster by the Office of Principal Scientific Adviser, Government of India.

The sprawling 852-acre IIT Jodhpur campus is a majestic tribute to the deep-rooted diverse cultural heritage of the region. The campus is an architectural masterpiece, which is slated to become an international exemplar of sustainability with net-zero energy, and advanced water and waste management strategies.&rdquo;

IIT Jodhpur

A persistent issue in deep learning (DL) is the inability of models to function in a domain in which they were not trained. For example, a model trained to segment an organ in MRI scans often dramatically fails when tested in the domain of computed tomography (CT) scans. Since manual segmentation is extremely timeconsuming, it is often not feasible to acquire an annotated dataset in the target domain. Domain adaptation allows transfer of knowledge about a labelled source domain into a target domain. In this work, we attempt to address the differences in model performance when segmenting from intravenous contrast (IVC) enhanced or from non-contrast (NC) CT scans. Most of the publicly available, large-scale, annotated CT datasets are IVCenhanced. However, physicians frequently use NC scans in clinical practice. This necessitates methods capable of reliably functioning across both domains. We propose a novel DL framework that can segment the pancreas from non-contrast CT scans through training with the help of IVC-enhanced CT scans. Our method first utilizes a CycleGAN to create synthetic NC (s-NC) variants from IVC scans. Subsequently, we introduce a multilevel 3D UNet architecture to perform pancreas segmentation. The proposed method significantly outperforms the baseline. Experimental results show 6.2% percent improvement compared to the baseline model in terms of the Dice coefficient. To our knowledge, this method is the first of its kind in pancreas segmentation from NC CTs. © 2020 SPIE.

World University root node

Progress in Biomedical Optics and Imaging - Proceedings of SPIE

Multilevel UNet for pancreas segmentation from non-contrast CT scans through domain adaptation

Human gaze dynamics mainly concern about the sequence of the occurrence of three eye movements: fixations, saccades, and microsaccades. In this paper, we correlate them as three different states to velocities of eye movements. We build a state trajectory estimator based on ancestor sampling (ST EAS) model, which captures the features of the human temporal gaze pattern to identify the kind of visual stimuli. We used a gaze dataset of 72 viewers watching 60 video clips which are equally split into four visual categories. Uniformly sampled velocity vectors from the training set, are used to find the best suitable parameters of the proposed statistical model. Then, the optimized model is used for both gaze data classification and video retrieval on the test set. We observed 93.265% of classification accuracy and a mean reciprocal rank of 0.888 for video retrieval on the test set. Hence, this model can be used for viewer independent video indexing for providing viewers an easier way to navigate through the contents. © 2020 IEEE.

IEEE 22nd International Workshop on Multimedia Signal Processing, MMSP 2020

Eye Movement State Trajectory Estimator based on Ancestor Sampling

Due to the nonsparse representation, the use of compressed sensing (CS) for physiological signals, such as a multichannel electroencephalogram (EEG), has been a challenge. We present a generalized Bayesian CS framework that is capable of handling representations that arise in the spatiotemporal setting. The proposed model utilizes the standard linear Gaussian observation model associated with the hierarchical modeling of data using the matrix-variate Gaussian scale mixture (GSM). It deploys various random and deterministic parameters to incorporate the knowledge of spatial and temporal correlation present in data. By varying distributions over random parameters, a family of generalized hyperbolic matrix variate distributions is derived. For estimation, we rely on variational Bayes (VB) for random parameters and expectation-maximization (EM) for deterministic parameters. Furthermore, the model is compared with recent developments in matrix-variate distribution-based modeling of data, and we briefly discuss its extension to finite mixtures of skewed distributions. Finally, the framework is applied to the steady-state visual evoked potential (SSVEP)-based EEG benchmark data set, and a comparative study is conducted to show its effectiveness for the frequency detection task. One of the crucial features of the proposed model is that it simultaneously processes multichannel signals with low computational cost and time, making it suitable for real-time systems, especially in a resource-constrained environment. IEEE

IEEE Transactions on Neural Networks and Learning Systems

Block Sparse Variational Bayes Regression Using Matrix Variate Distributions With Application to SSVEP Detection

Compressive sensing (CS) using deep learning for recovery of images from measurements has been well explored in recent years. Instead of sensing/sampling full image, block or patch based compressive sensing is chosen to overcome memory and computation limitations. The drawback of this block based CS sampling and recovery is that it does not capture global context and focuses only on the local context. This results in artifacts at the boundary of two consecutive image blocks. Random Gaussian or random Bernoulli matrix are commonly used as sensing matrices to sample an image block and generate corresponding linear measurements. Although, random Gaussian or random Bernoulli matrices exhibits Restricted Isometry property (RIP), which is a guarantee for good quality reconstructed image, its two main disadvantages are: 1) large memory and computational requirements and 2) their encoded measurements doesn't generalize well to a large-scale dataset. In this paper, we propose a data adaptive CS based on deep learning framework for image recognition where 1) sampling is done considering the global context and 2) encoding to obtain measurements is learned from data, so as to achieve the generalization over large-scale dataset. © 2020 IEEE.

26th National Conference on Communications, NCC 2020

Data adaptive compressed sensing using deep neural network for image recognition

Navigation of drones can be conceivably performed by operators by analyzing the brain signals of the person. EEG signal corresponding to the motor imaginations can be used for generation of control signals for drone. Different machine learning and deep learning approaches have been developed in the state of the art literature for the classification of motor imagery EEG signal. There is still a need for developing a suitable model that can classify the motor imagery signal fast and can generate a navigation command for drone in real-time. In this paper, we have reported the performance of convolutional stacked autoencoder and Convolutional Long short term memory models for classification of Motor imagery EEG signal. The developed models have been optimized using TensorRT that speeds up inference performance and the inference engine has been deployed on Jetson TX2 embedded platform. The performance of these models have been compared with different machine learning models. © 2020, Springer Nature Switzerland AG.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Classification of Motor Imagery EEG Signal for Navigation of Brain Controlled Drones

Video skimming is one of the recently, getting popular technique for preparing preview for long watching video sequences. Most of the video skimming techniques developed in the literature uses manual intervention of users to prepare the review. Mostly the literature reported video skimming for sports and movie industries. In sports the portion of video where audience claps are used and in movie important contents are manually selected for preparing the preview. However in literature rarely any work reported for skimming of lecture video sequences. Lecture videos are generally, recorded indoor, low illuminated, noisy environment condition and contents of the scene rarely changes much. Hence designing an automatic skimming scheme is quite difficult task. In this article, we put forward an intelligent expert video skimming technique for lecture video sequences, where human intervention is not required. In the proposed scheme, initially the lecture video is segmented into a number of shots. We proposed the use of radiometric correlation technique for lecture video segmentation or finding the shot transitions. After getting the shot transitions in a video, the shots are recognized. The fuzzy K-nearest neighborhood technique is proposed to recognize the shots in a video. The shots are recognized into three categories: title slides, written texts/displayed slides and talking heads/writing hands. Three contrast based features: one existing i.e., average sharpness (AS) and two newly proposed: relative height (RH) and edge potential (EP) are used to find the contents of a frame. The frames with different contrast values are categorized to prepare the video skimming or the capsule. The media recreation is achieved by selecting a set of frames around these selected content frames. The effectiveness of the proposed scheme is demonstrated in this paper using five test sequences, including three NPTEL and two non NPTEL. It is also observed that the capsule prepared by the proposed scheme, provides a better preview of the actual sequence. The performance of the proposed scheme is tested by comparing it against three state-of-the-art techniques. The evaluation of the proposed scheme is carried out by using three evaluation measures. It is also observed that the proposed scheme is found to be better than that of the existing schemes. © 2020 Elsevier Ltd

Expert Systems with Applications

Automatic lecture video skimming using shot categorization and contrast based features

Fall detection holds immense importance in the field of healthcare, where timely detection allows for instant medical assistance. In this context, we propose a 3D ConvNet architecture which consists of 3D Inception modules for fall detection. The proposed architecture is a custom version of Inflated 3D (I3D) architecture, that takes compressed measurements of video sequence as spatio-temporal input, obtained from compressive sensing framework, rather than video sequence as input, as in the case of I3D convolutional neural network. This is adopted since privacy raises a huge concern for patients being monitored through these RGB cameras. The proposed framework for fall detection is flexible enough with respect to a wide variety of measurement matrices. Ten action classes randomly selected from Kinetics-400 with no fall examples, are employed to train our 3D ConvNet post compressive sensing with different types of sensing matrices on the original video clips. Our results show that 3D ConvNet performance remains unchanged with different sensing matrices. Also, the performance obtained with Kinetics pre-trained 3D ConvNet on compressively sensed fall videos from benchmark datasets is better than the state-of-the-art techniques. © 2020, Springer Nature Singapore Pte Ltd.

Communications in Computer and Information Science

Compressive Sensing Based Privacy for Fall Detection

One of the well-established research domains among computer vision scientists is object tracking. However, not much work has been done in underwater scenarios. This article addresses the problem of visual tracking in the underwater environment with the stationary and nonstationary camera setups. In order to deal with the underwater optical dynamics, a dominant color component-based scene representation is employed in the YCbCr color space. An adaptive approach is devised to select the Walsh-Hadamard (WH) kernels for the efficient extraction of color, edge, and texture strengths, whereas a new feature called range strength is proposed to extract the variation of intensity from underwater sequences in the local neighborhood using the WH kernel. The likelihood of these feature strengths is integrated in a particle filter framework to track the object of interest in underwater sequences. The reference feature strengths used in assigning weights to the particles are updated based on the S$\phi$rensen distance. The coefficients of feature strengths are calculated in such a way that if one feature fails, then its coefficient become insignificant, whereas the more suitable features get higher feature coefficients. The effectiveness of the proposed scheme is evaluated using the underwater video datasets: reefVid, fish4knowledge (F4K), underwaterchangedetection (UWCD), and National Oceanic and Atmospheric Administration (NOAA). The performance evaluation is performed by comparing the scheme with five recent state-of-the-art tracking schemes. The quantitative analysis of the proposed scheme is carried out using three evaluation measures: overall intersection over union, centroid location error, and average tracking error. The performance of the proposed scheme is quite encouraging in the case of sequences with hazy and degraded, partially occluded, and camouflaged challenges. © 2005-2012 IEEE.

IEEE Transactions on Industrial Informatics

Walsh-Hadamard-Kernel-Based Features in Particle Filter Framework for Underwater Object Tracking

Deep learning solutions are vulnerable to adversarial perturbations and can lead a "frog" image to be misclassified as a "deer" or random pattern into "guitar". Adversarial attack generation algorithms generally utilize the knowledge of database and CNN model to craft the noise. In this research, we present a novel scheme termed as Camera Inspired Perturbations to generate adversarial noise. The proposed approach relies on the noise embedded in the image due to environmental factors or camera noise incorporated. We extract these noise patterns using image filtering algorithms and incorporate them into images to generate adversarial images. Unlike most of the existing algorithms that require learning of noise, the proposed adversarial noise can be applied in real-time. It is model-agnostic and can be utilized to fool multiple deep learning classifiers on various databases. The effectiveness of the proposed approach is evaluated on five different databases with five different convolutional neural networks such as ResNet-50, VGG-16, and VGG-Face. The proposed attack reduces the classification accuracy of every network, for instance, the performance of VGG-16 on the Tiny ImageNet database is reduced by more than 33%. The robustness of the proposed adversarial noise is also evaluated against different adversarial defense algorithms. © 2020 IEEE.

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

Noise is inside me! generating adversarial perturbations with noise derived from natural filters

Frequency spectrum has played a significant role in learning unique and discriminating features for object recognition. Both low and high frequency information present in images have been extracted and learnt by a host of representation learning techniques, including deep learning. Inspired by this observation, we introduce a novel class of adversarial attacks, namely ‘WaveTransform’, that creates adversarial noise corresponding to low-frequency and high-frequency subbands, separately (or in combination). The frequency subbands are analyzed using wavelet decomposition; the subbands are corrupted and then used to construct an adversarial example. Experiments are performed using multiple databases and CNN models to establish the effectiveness of the proposed WaveTransform attack and analyze the importance of a particular frequency component. The robustness of the proposed attack is also evaluated through its transferability and resiliency against a recent adversarial defense algorithm. Experiments show that the proposed attack is effective against the defense algorithm and is also transferable across CNNs. © 2020, Springer Nature Switzerland AG.