Department of Computer Science & Engineering

&ldquo;IIT Jodhpur is committed to impart quality education to facilitate the creation of qualified, competent and responsible human resources to meet the emerging technological challenges of the world. It is working towards creating an equitable learning platform that is positive, accessible and effective in the face of the modern societal and scientific advancements.

Flexibility in curriculum, in all its academic programs, encompassing elements of teaching, research, creativity, innovation and entrepreneurship is the most unique feature of IIT Jodhpur. The Department of Science and Technology (DST), under the National Mission on Cyber Physical Systems, has identified IIT Jodhpur as a Technology &amp; Innovation Hub in the area of Augmented Reality and Virtual Reality. The institute has also been nominated as the coordinator for the Jodhpur City and Knowledge Cluster by the Office of Principal Scientific Adviser, Government of India.

The sprawling 852-acre IIT Jodhpur campus is a majestic tribute to the deep-rooted diverse cultural heritage of the region. The campus is an architectural masterpiece, which is slated to become an international exemplar of sustainability with net-zero energy, and advanced water and waste management strategies.&rdquo;

IIT Jodhpur

Smart application has been focused more to solve real-world problems like smart home, smart street lighting system etc., but the concept of data transmission between devices require precise attention to preserve security with better performance output in device communication. The blockchain is a decentralized computation and information sharing platform to enable different players to verify on the fly. This paper presents a hyper-ledger fabric-based blockchain architecture using a one-way hash function to employ distributed data storage, to enhance the security of the smart applications, and to improve performance in the computation. The proposed protocol resists to different attacks, i.e., modification, impersonation, replay, man-in-the-middle, stolen verifies, device injection, false reputation, and appending. Moreover, performance results are comparatively better rather than relevant vehicular communication protocols. © 2019 IEEE.

World University root node

Proceedings of 2019 25th Asia-Pacific Conference on Communications, APCC 2019

BlockCom: Blockchain-based efficient communication and storage protocol

Power efficiency is one of the grand challenge problems facing computer architecture in recent years. Driven by the growth towards green computing, it is imperative to design architectures that can provide maximum power savings while incurring minimal overhead on real estate and performance. Towards this we propose a state-preserving mechanism for dynamically configuring DRAM banks based on utilization. We propose a novel mechanism via which it is possible to dynamically power down and power on banks while taking the bank utilization and overall performance into account. Over extensive experimentation using memory intensive applications, we can observe a power savings of up to 12.31% while incurring an average performance loss of 0.82% over baseline executions. © 2019 IEEE.

Proceedings - International Symposium on Quality Electronic Design, ISQED

State Preserving Dynamic DRAM Bank Re-Configurations for Enhanced Power Efficiency

In this study, the authors propose a framework SUGAMAN (Supervised and Unified framework using Grammar and Annotation Model for Access and Navigation). SUGAMAN is a Hindi word meaning ‘easy passage from one place to another’. SUGAMAN synthesises textual description from a given floor plan image, usable by visually impaired to navigate by understanding the arrangement of rooms and furniture. It is the first framework for describing a floor plan and giving direction for obstacle-free movement within a building. The model learns five classes of room categories from 1355 room image samples under a supervised learning paradigm. These learned annotations are fed into a description synthesis framework to yield a holistic description of a floor plan image. Authors demonstrate the performance of various supervised classifiers on room learning and provided a comparative analysis of system generated and human-written descriptions. The contribution of this study includes a novel framework for description generation from document images with graphics while proposing a new feature representing the floor plans, text annotations for a publicly available data set, and an algorithm for door to door obstacle avoidance navigation. This work can be applied to areas like understanding floor plans and design of historical monuments, and retrieval. © The Institution of Engineering and Technology 2019

Department of Mathematics

Fulltext

IET Image Processing

Sugaman: Describing floor plans for visually impaired by annotation learning and proximity-based grammar

Over the last decade there has been growing interest in understanding the brain activity, in the absence of any task or stimulus, captured by the resting-state functional magnetic resonance imaging (rsfMRI). The resting state patterns have been observed to be exhibiting complex spatio-temporal dynamics and substantial effort has been made to characterize the dynamic functional connectivity (dFC) configurations. However, the dynamics governing the state transitions that the brain undergoes and their relationship to stationary functional connectivity still remains an open problem. One class of approaches attempts to characterize the dynamics in terms of finite number of latent brain states, however, such attempts are yet to amalgamate the underlying anatomical structural connectivity (SC) with the dynamics. Another class of methods links individual dynamic FCs with the underlying SC but does not characterize the temporal evolution of FC. Further, the latent states discovered by previous approaches could not be directly linked to the SC, thereby motivating us to discover the underlying lower-dimensional manifold that represents the temporal structure. In the proposed approach, the discovered manifold is further parameterized as a set of local density distributions, or latent transient states. We propose an innovative method that learns parameters specific to the latent states using a graph-theoretic model (temporal Multiple Kernel Learning, tMKL) that inherently links dynamics to the structure and finally predicts the grand average FC of the test subjects by leveraging a state transition Markov model. The proposed solution does not make strong assumptions about the underlying data and is generally applicable to resting or task data for learning subject-specific state transitions and for successfully characterizing SC-dFC-FC relationship through a unifying framework. Training and testing were done using the rs-fMRI data of 46 healthy participants. tMKL model performs significantly better than the existing models for predicting resting state functional connectivity based on whole-brain dynamic mean-field model (DMF), single diffusion kernel (SDK) model and multiple kernel learning (MKL) model. Further, the learned model was tested on an independent cohort of 100 young, healthy participants from the Human Connectome Project (HCP) and the results establish the generalizability of the proposed solution. More importantly, the model retains sensitivity toward subject-specific anatomy, a unique contribution towards a holistic approach for SC-FC characterization. © 2018 Elsevier Inc.

School of Artificial Intelligence and Data Science (AIDE)

NeuroImage

Resting state dynamics meets anatomical structure: Temporal multiple kernel learning (tMKL) model

Recovering textured 3D models of non-rigid human body shapes is challenging due to self-occlusions caused by complex body poses and shapes, clothing obstructions, lack of surface texture, background clutter, sparse set of cameras with non-overlapping fields of view, etc. Further, a calibration-free environment adds additional complexity to both - reconstruction and texture recovery. In this paper, we propose a deep learning based solution for textured 3D reconstruction of human body shapes from a single view RGB image. This is achieved by first recovering the volumetric grid of the non-rigid human body given a single view RGB image followed by orthographic texture view synthesis using the respective depth projection of the reconstructed (volumetric) shape and input RGB image. We propose to co-learn the depth information readily available with affordable RGBD sensors (e.g., Kinect) while showing multiple views of the same object during the training phase. We show superior reconstruction performance in terms of quantitative and qualitative results, on both, publicly available datasets (by simulating the depth channel with virtual Kinect) as well as real RGBD data collected with our calibrated multi Kinect setup. © 2018. The copyright of this document resides with its authors. It may be distributed unchanged freely in print or electronic forms.

British Machine Vision Conference 2018, BMVC 2018

Deep textured 3D reconstruction of human bodies

In today's time, we use Leave Application System managed by some central authority. This can lead to corruption, misuse by a single authority or unnecessary time for approval. So, in this paper, we have developed smart contract architecture for Leave Management System using Solidity and Ethereum, which is a decentralized app architecture leveraged from Blockchain Technology. We also include smartphones as IoT devices by which any client can use this app. Due to the resource-constrained nature of these devices, the smart contract app is coupled with an alternative centralized architecture which is a classic client/server architecture with an underlying Blockchain backend. © 2019 IEEE.

2019 11th International Conference on Communication Systems and Networks, COMSNETS 2019

Develop Leave Application using Blockchain Smart Contract

We propose a shape-based solution for speckle removal from ultrasound images. The method is operable in both low contrast and high contrast imaging scenarios. The approach introduces shape information alongside structural information in the speckle removing filter. By iteratively minimizing a shape fidelity penalty to reduce speckle, the proposed filter facilitates superior segmentation of blood vessels. The effectiveness of the proposed method is established through experimentation on the ultrasound images of human blood vessels. The results show at more than 7% improvement in PSNR values compared to other state-of-the-art approaches. © 2019 IEEE.

Proceedings - International Conference on Image Processing, ICIP

Shape Based Speckle Removal for Ultrasound Image Segmentation

Automatic image colorization is a very interesting computer graphics problem wherein an input grayscale image is transformed into its RGB domain. However, it is an ill-posed problem as there can be multiple RGB outcomes for a particular grayscale pixel. The problem further complicates if noise is present in the grayscale image. In this paper, we propose a Robust Image Colorization using Self-attention based Progressive Generative Adversarial Network (RIC-SPGAN) which consists of residual encoder-decoder (RED) network and a Self-attention based progressive Generative network (SP-GAN) in a cascaded form to perform the denoising and colorization of the image. We have used self-attention based progressive network to model the long-range dependencies and gradually enhanced the resolution of the colorized image for faster, stable and variation rich features for generation of the image. We also presented the stabilization technique of the presented generative model. Our model has shown exceptional perceptual results on noisy and normal grayscale images. We have trained our model on ILSVRC2012. The visual results on images of DIV2K with and without noise has been presented in the paper along with the failure cases of the model. © 2019 IEEE.

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

Robust image colorization using self attention based progressive generative adversarial network

Text present in images are not merely strings, they provide useful cues about the image. Despite their utility in better image understanding, scene texts are not used in traditional visual question answering (VQA) models. In this work, we present a VQA model which can read scene texts and perform reasoning on a knowledge graph to arrive at an accurate answer. Our proposed model has three mutually interacting modules: I. proposal module to get word and visual content proposals from the image, ii. fusion module to fuse these proposals, question and knowledge base to mine relevant facts, and represent these facts as multi-relational graph, iii. reasoning module to perform a novel gated graph neural network based reasoning on this graph. The performance of our knowledge-enabled VQA model is evaluated on our newly introduced dataset, viz. text-KVQA. To the best of our knowledge, this is the first dataset which identifies the need for bridging text recognition with knowledge graph based reasoning. Through extensive experiments, we show that our proposed method outperforms traditional VQA as well as question-answering over knowledge base-based methods on text-KVQA. © 2019 IEEE.

Proceedings of the IEEE International Conference on Computer Vision

From strings to things: Knowledge-enabled VQA model that can read and reason

Visual Question Answering (VQA) has emerged as an important problem spanning Computer Vision, Natural Language Processing and Artificial Intelligence (AI). In conventional VQA, one may ask questions about an image which can be answered purely based on its content. For example, given an image with people in it, a typical VQA question may inquire about the number of people in the image. More recently, there is growing interest in answering questions which require commonsense knowledge involving common nouns (e.g., cats, dogs, microphones) present in the image. In spite of this progress, the important problem of answering questions requiring world knowledge about named entities (e.g., Barack Obama, White House, United Nations) in the image has not been addressed in prior research. We address this gap in this paper, and introduce KVQA - the first dataset for the task of (world) knowledge-aware VQA. KVQA consists of 183K question-answer pairs involving more than 18K named entities and 24K images. Questions in this dataset require multi-entity, multi-relation, and multi-hop reasoning over large Knowledge Graphs (KG) to arrive at an answer. To the best of our knowledge, KVQA is the largest dataset for exploring VQA over KG. Further, we also provide baseline performances using state-of-the-art methods on KVQA. © 2019, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019