Header menu link for other important links
X

Multimodal deep learning

Raghav Sood, Mayank Vatsa,
Published in
2019
Abstract

Modality refers to how something happens or is experienced. It is the representation format in which information is stored. Multimodal Learning involves relating information from multiple sources. Many modalities are combined at the training stage to learn better features for improving the performance during detection/classification. In this report, I have analyzed how the techniques in multimodal deep learning have advanced, and performance improved over the years. I have also performed a case study by running a model on a novel Multimodal Fake News Dataset. The improved performance on using multimodal features representation on this dataset compared to single modality text/image representations has also been observed. I have further created a novel multimodal algorithm for Fake News Detection. We have done an in-depth analysis by running it on certain Fake News datasets to show how this multimodal algorithm is an improvement over the other Fake News detection algorithms. We also performed an ablation analysis on our computed results and tried to visually conceptualize the results with the help of plots and diagrams.

About the journal
Open AccessNo