Bitumen in the oil sands industry is separated from sand using a water-based gravity separation process in a primary separation vessel (PSV). The interface between the froth and the middlings layer is an important parameter to control for optimal operation of the PSV unit. In this paper, a method using computer vision based on convolutional neural networks (CNNs) and Kalman filter (KF) is designed for the detection of the interface in PSV, with the CNN estimating both the interface level and the image quality. The proposed method consists of two parts: An offline and an online stage, wherein the parameters of CNN and KF are trained in the offline stage using the available data. The algorithm is made robust to any new type of occlusions, not present in the training dataset, in the online stage. Experimental results demonstrate that the proposed method is accurate and robust to different abnormalities in the process. © 1963-2012 IEEE.