In this paper, we address the problem of unsupervised learning of usual patterns of activities in an area under surveillance and detecting deviant patterns. We use video epitomes for segmenting foreground objects from background and obtain an approximate shape, trajectory and temporal information in the form of space-time patches. We apply pLSA for finding correlations among these patches to learn usual activities in the scene. We also extend pLSA to classify a novel video as usual or unusual. © 2008 IEEE.