Objective video quality assessment (VQA) is the use of computational models to predict the video quality in line with the perception of the human visual system (HVS). It is challenging due to the underlying complexity, and the relatively limited understanding of the HVS and its intricate mechanisms. There are two important issues regarding VQA: (a) the temporal factors apart from the spatial ones also need to be considered, (b) the contribution of each factor and their interaction to the overall video quality needs to be determined. In this paper, we attempt to tackle the first issue by utilizing the variation of spatial quality along the temporal axis. The second issue is addressed by the use of machine learning; we believe this to be more convincing since the relationship between the factors and the overall quality is derived via training with substantial ground truth (i.e. subjective scores). Experiments conducted using two publicly available video databases show the effectiveness of the proposed full-reference metric in comparison to the relevant existing VQA metrics. © 2011 IEEE.