Objectively assessing the perceptual quality of an ocular fundus image is essential for the reliable diagnosis of various ocular diseases. A fair amount of work has been done in this field to date. However, the generalizability of the current work is limited, as the existing quality models were developed and evaluated with data-sets built with limited subjective inputs. This paper aims at addressing this limitation with the following two contributions. First, a new fundus image quality assessment (FIQuA) data-set is presented, containing 1500 fundus images with three classes of quality: Good, Fair, and Poor. Also, for each image, subjective scores (in the range [0-10]) were collected for six quality parameters, including structural and generic properties of the fundus images. Second, a new multivariate regression based convolutional neural network (CNN) model is proposed to predict the fundus image quality. The proposed model consists of two individually trained blocks. The first block consists of four pre-trained models, trained against the subjective scores for the six quality parameters, and aims at deriving the optimized features for classification. Next, the optimized features from each of the four models are ensembled together and transferred to the second block for final classification. The proposed model achieves a strong correlation with the subjective scores, with the values 0.941, 0.954, 0.853, and 0.401 obtained for SROCC, LCC, KCC, and RMSE respectively. Its classification accuracy is 95.66% over the FIQuA data-set, and 98.96% and 88.43% respectively over the two publicly available data-sets DRIMDB and EyeQ. © 2013 IEEE.