Recent advances in the Brain-Computer Interface (BCI) systems state that the accurate Motor Imagery (MI) classification using Electroencephalogram (EEG) plays a vital role. In this paper, we propose a novel real-time feature extraction and classification architecture for four class MI using a combination of Kullback-Leibler Regularized Riemannian Mean (KLRRM) and Linear SVM. By using the KL regularization, the robustness of the features extracted to the noise and outliers is improved. The performance of the proposed architecture is analyzed on the four class MI dataset 2a from the BCI Competition IV. The performance analysis shows that the proposed architecture achieves an average classification accuracy of 74.43% and 51.53% for both the good and noisy subjects respectively. Also, the emphasis is laid on understanding the performance of regularization, and the improvement of robustness to the noise and outliers is demonstrated using the noisy subjects. © 2018 IEEE.