Two popular representation learning paradigms are dictionary learning and deep learning. While dictionary learning focuses on learning 'basis' and 'features' by matrix factorization, deep learning focuses on extracting features via learning 'weights' or 'filter' in a greedy layer by layer fashion. This paper focuses on combining the concepts of these two paradigms by proposing deep dictionary learning and show how deeper architectures can be built using the layers of dictionary learning. The proposed technique is compared with other deep learning approaches, such as stacked autoencoder, deep belief network, and convolutional neural network. Experiments on benchmark data sets show that the proposed technique achieves higher classification and clustering accuracies. On a real-world problem of electrical appliance classification, we show that deep dictionary learning excels where others do not yield at-par performance. We postulate that the proposed formulation can pave the path for a new class of deep learning tools. © 2016 IEEE.