&ldquo;IIT Jodhpur is committed to impart quality education to facilitate the creation of qualified, competent and responsible human resources to meet the emerging technological challenges of the world. It is working towards creating an equitable learning platform that is positive, accessible and effective in the face of the modern societal and scientific advancements.

Flexibility in curriculum, in all its academic programs, encompassing elements of teaching, research, creativity, innovation and entrepreneurship is the most unique feature of IIT Jodhpur. The Department of Science and Technology (DST), under the National Mission on Cyber Physical Systems, has identified IIT Jodhpur as a Technology &amp; Innovation Hub in the area of Augmented Reality and Virtual Reality. The institute has also been nominated as the coordinator for the Jodhpur City and Knowledge Cluster by the Office of Principal Scientific Adviser, Government of India.

The sprawling 852-acre IIT Jodhpur campus is a majestic tribute to the deep-rooted diverse cultural heritage of the region. The campus is an architectural masterpiece, which is slated to become an international exemplar of sustainability with net-zero energy, and advanced water and waste management strategies.&rdquo;

IIT Jodhpur

This study presents supervised framework for automatic recognition and retrieval of interactions (SAFARRIs), a supervised learning framework to recognise interactions such as pushing, punching, and hugging, between a pair of human performers in a video shot. The primary contribution of the study is to extend the vectors of locally aggregated descriptors (VLADs) as a compact and discriminative video encoding representation, to solve the complex class partitioning problem of recognising human interaction. An initial codebook is generated from the training set of video shots, by extracting feature descriptors around the spatiotemporal interest points computed across frames. A bag of action words is generated by encoding the first-order statistics of the visual words using VLAD. Support vector machine classifiers (1 against all) are trained using these codebooks. The authors have verified SAFARRI's accuracy for classification and retrieval (query by example). SAFARRI is free from tracking or recognition of body parts and capable of identifying the region of interaction in video shots. It gives superior retrieval and classification performances over recently proposed methods, on two publicly available human interaction datasets. © The Institution of Engineering and Technology 2016.

Department of Computer Science & Engineering

World University root node

IET Computer Vision

Supervised framework for automatic recognition and retrieval of interaction: A framework for classification and retrieving videos with similar human interactions

An automatic lookup tool, which matches and retrieves similar floorplans from a large repository of digitized architectural floorplans can prove to be of immense help for the architects while designing new projects. In this paper, we have proposed a framework for the matching and retrieval of similar architectural floorplans under the query by example paradigm. We propose a room layout segmentation and adjacent room detection algorithm to represent layouts as an undirected graph. We have also proposed a novel graph spectral embedding feature to uniquely represent the layout of the architectural floorplan. This helps in effective and efficient matching of the room layouts. Room semantics in terms of both the room structures and room decor is used to retrieve similar floorplans from the repository. To match the semantic similarity between a pair of floorplans, we have proposed a two stage matching technique. We have validated the effectiveness of our proposed framework by performing experiments on publicly available floorplan dataset and achieved high retrieval accuracy. © 2016 IEEE.

Proceedings - International Conference on Pattern Recognition

A unified framework for semantic matching of architectural floorplans

This paper presents generalized spatiotemporal analysis and lookup tool (GESTALT), an unsupervised framework for content-based video retrieval. GESTALT takes a query video and retrieves “similar” videos from the database. Motion and dynamics of appearance (shape) patterns of a prominent moving foreground object are considered as the key components of the video content and captured using corresponding feature descriptors. GESTALT automatically segments the moving foreground object from the given query video shot and estimates the motion trajectory. A graph-based framework is used to explicitly capture the structural and kinematics property of the motion trajectory, while an improved version of an existing spatiotemporal feature descriptor is proposed to model the change in object shape and movement over time. A combined match cost is computed as a convex combination of the two match scores, using these two feature descriptors, which is used to rank-order the retrieved video shots. Effectiveness of GESTALT is shown using extensive experimentation, and comparative study with recent techniques exhibits its superiority. © 2015, Springer-Verlag London.

Signal, Image and Video Processing

Use of trajectory and spatiotemporal features for retrieval of videos with a prominent moving foreground object

This paper presents VIDeo Content Analysis and Retrieval (VIDCAR), an unsupervised framework for Content-Based Video Retrieval (CBVR) using representation of the dynamics in the spatio-temporal model extracted from video shots. We propose Dynamic Multi Spectro Temporal-Curvature Scale Space (DMST-CSS), an improved feature descriptor for enhancing the performance of CBVR task. Our primary contribution is in representation of the dynamics of the evolution of the MST-CSS surface. Unlike the earlier MST-CSS descriptor [22], which extracts geometric features after the evolving MST-CSS surface converges to a final formation, this DMST-CSS captures the dynamics of the evolution (formation) of the surface and is thus more robust. We have represented the dynamics of MST-CSS surface as a multivariate time series to obtain a DMST-CSS descriptor. A global kernel alignment technique has been adapted to compute a match cost between query and model DMST-CSS descriptor. In our experiments, VIDCAR was shown to have greater precision recall than the competitors on five datasets. © Springer-Verlag London 2014.

International Journal of Multimedia Information Retrieval

VIDCAR: An unsupervised CBVR framework for identifying similar videos with prominent object motion

Extraction of the moving foreground object from a given video shot is an important task for spatiotemporal analysis and content representation in many computer vision and digital video processing applications. We propose an iterative framework based on energy minimization, for segmenting the prominent moving foreground object efficiently from moving camera video (MCV) shots. The solution obtained using graph-cut for figure-ground classification is enhanced using features extracted over a set of neighboring frames. This is used to iteratively update the foreground and background probability (tri-) maps and hence the graph weights. The segmentation results from neighboring frames are integrated as constraints to iteratively guide the energy minimization process, for an efficient solution. The proposed framework is automatic and does not require any human interaction (neither initialization nor refinement). Our method outperforms recent state-of-the-art moving object segmentation techniques on benchmark datasets with MCV shots. © 2014, Springer-Verlag London.

Prominent moving object segmentation from moving camera video shots using iterative energy minimization

In this paper we propose SEVA (Salient Events for Video Analytics), a framework for accurate detection and localization of salient events from a given video shot. Our proposed method is based on statistical learning theory and models salient event detection as a one-class classification problem. Video frames are split into blocks for extracting both the spatial and temporal features. Given a video shots we first track the moving foreground blob. Features are extracted using only the foreground pixels to avoid influence of the background. Using Support Vector Data Description (SVDD) in kernel feature space for each block in a given video frame, the decision boundary for the normal activity class is modeled. For a test video sequence, feature vectors are computed from the video frames and the learnt model is utilized to classify each block as normal or salient. Finally, we have adapted a spatio-temporal smoothing approach to remove the false positives. We have reported both qualitative and quantitative results of our experiments on two real-world benchmarked video datasets. Performance of SEVA is compared with five recent works on video event detection to validate its effectiveness. © 2014 IEEE.

Proceedings - 4th International Conference on Emerging Applications of Information Technology, EAIT 2014

SEVA: A salient event detection framework from video shots using support vector data description

In this paper, we address the problem of Content Based Video Retrieval using a multivariate time series modeling of features. We particularly focus on representing the dynamics of geometric features on the Spatio-Temporal Volume (STV) created from a real world video shot. The STV intrinsically holds the video content by capturing the dynamics of the appearance of the foreground object over time, and hence can be considered as a dynamical system. We have captured the geometric property of the parameterized STV using the Gaussian curvature computed at each point on its surface. The change of Gaussian curvature over time is then modeled as a Linear Dynamical System (LDS). Due to its capability to efficiently model the dynamics of a multivariate signal, Auto Regressive Moving Average (ARMA) model is used to represent the time series data. Parameters of the ARMA model are then used for video content representation. To discriminate between a pair of video shots (time series), we have used the subspace angle between a pair of feature vectors formed using ARMA model parameters. Experiments are done on four publicly available benchmark datasets, shot using a static camera. We present both qualitative and quantitative analysis of our proposed framework. Comparative results with three recent works on video retrieval also show the efficiency of our proposed framework. © 2013, Springer-Verlag London.

Multivariate time series modeling of geometric features of spatio-temporal volumes for content based video retrieval

This paper presents a genre-specific modeling strategy capable of improving the task of content based video classification and the speed of data retrieval operations. With the ever increasing growth of video data it is important to classify video shots into groups based on its content. For that reason, it is of primary concern to design systems that could automatically classify videos into different genres based on its content. We consider the genre recognition task as a classification problem. We use support vector machines to perform the classification task and propose an improved video classification method. The experimental results show that genre-specific modeling of features can significantly improve the performance. Results have been compared with two contemporary works on video classification, to demonstrate the superiority of our proposed framework. © 2013, Springer-Verlag London.

Fulltext

Genre-specific modeling of visual features for efficient content based video shot classification and retrieval

This paper presents the design of STAR (Spatio-Temporal Analysis and Retrieval), an unsupervised Content Based Video Retrieval (CBVR) System. STAR's key insight and primary contribution is that it models video content using a joint spatio-temporal feature representation and retrieves videos from the database which have similar moving object and trajectories of motion. Foreground moving blobs from a moving camera video shot are extracted, along with a trajectory for camera motion compensation, to form the space-time volume (STV). The STV is processed to obtain the EMST-CSS representation, which can discriminate across different categories of videos. Performance of STAR has been evaluated qualitatively and quantitatively using precision-recall metric on benchmark video datasets having unconstrained video shots, to exhibit efficiency of STAR. © 2013 IEEE.

2013 4th National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics, NCVPRIPG 2013

STAR: A Content Based Video Retrieval system for moving camera video shots

A framework has been proposed for representing the trajectory of a moving object, using a novel hyperstring based approach for efficient retrieval of video shots. The hyperstring based model unifies both the structural and kinematic features for an improved representation of the trajectory. A Constraint-driven Adjacency Graph matching (CAGM) algorithm has been proposed to measure the similarity between a pair of query and model hyperstrings. Experiments have been performed on benchmark datasets of trajectories (one synthetic and three real-world video shots), to assess the performance (using Precision-Recall metric) of the proposed model. Results have been compared with two similar published works on video retrieval using trajectories, to demonstrate the superiority of our proposed framework. © 2012 IEEE.

Proceedings - 2012 3rd International Conference on Emerging Applications of Information Technology, EAIT 2012

A novel hyperstring based descriptor for an improved representation of motion trajectory and retrieval of similar video shots with static camera

Multi-Spectro-Temporal Curvature Scale Space (MST-CSS) had been proposed as a video content descriptor in an earlier work, where the peak and saddle points were used for feature points. But these are inadequate to capture the salient features of the MST-CSS surface, producing poor retrieval results. To overcome these, we propose EMST-CSS (Enhanced MST-CSS) as a better feature representation with an improved matching method for CBVR (Content Based Video Retrieval). Comparative study with the existing MST-CSS representation and two state-of-the-art methods for CBVR shows enhanced performance on one synthetic and two real-world datasets. © 2012 IEEE.

Proceedings - 2012 IEEE International Symposium on Multimedia, ISM 2012

Enhancing the MST-CSS representation using robust geometric features, for efficient Content Based Video Retrieval (CBVR)

In this work, we propose a framework for a robust Content Based Video Retrieval (CBVR) system with free hand query sketches, using the Multi-Spectro Temporal-Curvature Scale Space (MST-CSS) representation. Our designed interface allows sketches to be drawn to depict the shape of the object in motion and its trajectory. We obtain the MST-CSS feature representation using these cues and match with a set of MST-CSS features generated offline from the video clips in the database (gallery). Results are displayed in rank ordered similarity. Experimentation with benchmark datasets shows promising results. © 2012 IEEE.