Department of Computer Science & Engineering

&ldquo;IIT Jodhpur is committed to impart quality education to facilitate the creation of qualified, competent and responsible human resources to meet the emerging technological challenges of the world. It is working towards creating an equitable learning platform that is positive, accessible and effective in the face of the modern societal and scientific advancements.

Flexibility in curriculum, in all its academic programs, encompassing elements of teaching, research, creativity, innovation and entrepreneurship is the most unique feature of IIT Jodhpur. The Department of Science and Technology (DST), under the National Mission on Cyber Physical Systems, has identified IIT Jodhpur as a Technology &amp; Innovation Hub in the area of Augmented Reality and Virtual Reality. The institute has also been nominated as the coordinator for the Jodhpur City and Knowledge Cluster by the Office of Principal Scientific Adviser, Government of India.

The sprawling 852-acre IIT Jodhpur campus is a majestic tribute to the deep-rooted diverse cultural heritage of the region. The campus is an architectural masterpiece, which is slated to become an international exemplar of sustainability with net-zero energy, and advanced water and waste management strategies.&rdquo;

IIT Jodhpur

This paper reviews the NTIRE challenge on image colorization (estimating color information from the corresponding gray image) with focus on proposed solutions and results. It is the first challenge of its kind. The challenge had 2 tracks. Track 1 takes a single gray image as input. In Track 2, in addition to the gray input image, some color seeds (randomly samples from the latent color image) are also provided for guiding the colorization process. The operators were learnable through provided pairs of gray and color training images. The tracks had 188 registered participants, and 8 teams competed in the final testing phase. © 2019 IEEE.

World University root node

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

NTIRE 2019 challenge on image colorization: Report

This study is concerned with achieving the image compression using the concept of memorability. The authors have used memorability of an image, as a perceptual measure while image coding. In the proposed approach, a region-of-interest-based memorability preserving image compression algorithm which is accomplished via two sub-processes namely, memorability prediction and image compression is introduced. The memorability of images is predicted using convolutional neural network and restricted Boltzmann machine features. Based on these features, the memorability score of individual patches in an image is calculated and these scores are used to generate the memorability map. These memorability map values are used for optimised image compression. In order to validate the results, an eye tracking experiment with human participants is performed. The comparative analysis shows that the memorability-based compression outperforms the state-of-the-art compression techniques. © The Institution of Engineering and Technology.

IET Image Processing

Memorability-based image compression

Image annotation (tag) and classification play a critical role in many computer vision applications, such as image retrieval, scene understanding, scene description etc. While, databases such as ImageNet have high quality labels for images, in real world, a large number of images have missing labels or tags that completely describe the contents of an image. To solve this problem, in this paper, we work on the hypothesis that class and tag information are correlated and propose a joint optimization for image classification and annotation. We construct a unified cost function to learn the class scoring vectors as well as tag scoring vectors. The proposed approach achieves state-of-the-art results on benchmark datasets for joint tag prediction and classification. © 2018 IEEE.

Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018

Joint Image Classification and Annotation Prediction Using Iterative Learning on Local Neighbourhood

This paper introduces a context-aware reasoning framework that adapts to the needs and preferences of inhabitants continuously to provide contextually relevant recommendations to the group of users in a smart home environment. User’s activity and mobility plays a crucial role in defining various contexts in and around the home. The observation data acquired from disparate sensors, called user’s context, is interpreted semantically to implicitly disambiguate the users that are being recommended to. The recommendations are provided based on the relationship that exist among multiple users and the decision is made as per the preference or priority. The proposed approach makes extensive use of multimedia ontology in the life cycle of situation recognition to explicitly model and represent user’s context in smart home. Further, dynamic reasoning is exploited to facilitate context-aware situation tracking and intelligently recommending appropriate actions which suit the situation. We illustrate use of the proposed framework for Smart Home use-case. © 2019, Springer Nature Switzerland AG.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Context-Aware Reasoning Framework for Multi-user Recommendations in Smart Home

We present some details of Bayesian block sparse modeling using hierarchical prior having deterministic and random parameters when entries within the blocks are correlated. In particular, the effect of the threshold to prune out variance parameters of algorithms corresponding to several choices of marginals, viz. multivariate Jeffery prior, multivariate Laplace distribution and multivariate Student’s t distribution, is discussed. We also provide details of experiments with Electroencephalograph (EEG) data which shed some light on the possible applicability of the proposed Sparse Variational Bayes framework. © Springer Nature Switzerland AG 2019.

Some Comments on Variational Bayes Block Sparse Modeling with Correlated Entries

Rice is a global staple crop that has a large requirement of water at various growth and developmental stages. As a matter of fact, drought can substantially reduce rice yield. The traits responsible for drought tolerance can be selected and studied to improve rice growth and yield under stress conditions. In the present study, morphological, physiological, biochemical and molecular variations between drought tolerant (PB6 and Moroberakan) and drought sensitive (Way Rarem) varieties have been evaluated. Notable differences have been observed in root morphology, root xylem number and area, stomata number, relative water content, proline content, protein and gene expression. In addition, novel automated frameworks based on computer vision algorithms have been employed for high-throughput quantification of xylem using scanning electron microscopy images. Transcriptome analysis has revealed variation in expression of Med37c and RSOsPR10 genes among various rice varieties. The differential proteome analysis using 2-D followed by MALDI-based identification implied the role of chitinases in drought tolerance. The information generated from computational/digital image analysis as well as biochemical studies can contribute significantly towards the developing field of rice phenomics. © 2019 Elsevier B.V.

Environmental and Experimental Botany

Morphological, transcriptomic and proteomic responses of contrasting rice genotypes towards drought stress

Cataract is a common ophthalmic disorder and the leading cause of blindness worldwide. While cataract is cured via surgical procedures, its impact on iris based biometric recognition has not been effectively studied. The key objective of this research is to assess the effect of cataract surgery on the iris texture pattern as a means of personal authentication. We prepare and release the IIITD Cataract Surgery Database (CaSD) captured from 132 cataract patients using three commercial iris sensors. A non-comparative non-randomized cohort study is performed on the iris texture patterns in CaSD and authentication performance is studied using three biometric recognition systems. Performance is lower when matching pre-operative images to post-operative images (74.69 ± 9.77%) as compared to matching pre-operative images to pre-operative images (93.42 ± 1.76%). 100% recognition performance is observed on a control-group of healthy irises from 68 subjects. Authentication performance improves if cataract affected subjects are re-enrolled in the system, though re-enrollment does not ensure performance at par with pre-operative scenarios (86.67 ± 5.64%). The results indicate that cataract surgery affects the discriminative nature of the iris texture pattern. This finding raises concerns about the reliability of iris-based biometric recognition systems in the context of subjects undergoing cataract surgery.

Fulltext

Scientific reports

Phacoemulsification Cataract Surgery Affects the Discriminative Capacity of Iris Pattern Recognition

Deep neural network (DNN) architecture based models have high expressive power and learning capacity. However, they are essentially a black box method since it is not easy to mathematically formulate the functions that are learned within its many layers of representation. Realizing this, many researchers have started to design methods to exploit the drawbacks of deep learning based algorithms questioning their robustness and exposing their singularities. In this paper, we attempt to unravel three aspects related to the robustness of DNNs for face recognition: (i) assessing the impact of deep architectures for face recognition in terms of vulnerabilities to attacks, (ii) detecting the singularities by characterizing abnormal filter response behavior in the hidden layers of deep networks; and (iii) making corrections to the processing pipeline to alleviate the problem. Our experimental evaluation using multiple open-source DNN-based face recognition networks, and three publicly available face databases demonstrates that the performance of deep learning based face recognition algorithms can suffer greatly in the presence of such distortions. We also evaluate the proposed approaches on four existing quasi-imperceptible distortions: DeepFool, Universal adversarial perturbations, l2, and Elastic-Net (EAD). The proposed method is able to detect both types of attacks with very high accuracy by suitably designing a classifier using the response of the hidden layers in the network. Finally, we present effective countermeasures to mitigate the impact of adversarial attacks and improve the overall robustness of DNN-based face recognition. © 2019, Springer Science+Business Media, LLC, part of Springer Nature.

International Journal of Computer Vision

Detecting and Mitigating Adversarial Perturbations for Robust Face Recognition

Very low resolution (VLR) image recognition corresponds to classifying images with resolution 16x16 or less. Though it has widespread applicability when objects are captured at a very large stand-off distance (e.g. surveillance scenario) or from wide angle mobile cameras, it has received limited attention. This research presents a novel Dual Directed Capsule Network model, termed as DirectCapsNet, for addressing VLR digit and face recognition. The proposed architecture utilizes a combination of capsule and convolutional layers for learning an effective VLR recognition model. The architecture also incorporates two novel loss functions: (i) the proposed HR-anchor loss and (ii) the proposed targeted reconstruction loss, in order to overcome the challenges of limited information content in VLR images. The proposed losses use high resolution images as auxiliary data during training to 'direct' discriminative feature learning. Multiple experiments for VLR digit classification and VLR face recognition are performed along with comparisons with state-of-the-art algorithms. The proposed DirectCapsNet consistently showcases state-of-the-art results; for example, on the UCCS face database, it shows over 95% face recognition accuracy when 16x16 images are matched with 80x80 images. © 2019 IEEE.

Proceedings of the IEEE International Conference on Computer Vision

Dual directed capsule network for very low resolution image recognition

Researchers have explored the importance of Siamese networks in deep learning. With recent developments in deep learning and the effectiveness of deep dictionary learning, this research proposes the architecture of Siamese Deep Dictionary Learning. We first propose the architecture followed by solving the optimization problem. The experimental effectiveness is demonstrated on five different image databases pertaining to two classification problems: face verification and kinship verification. The experiments show that the proposed Siamese Deep Dictionary Learning yields comparable results compared to state-of-the-art algorithms on all five databases. © 2019 IEEE.