Due to the difficulties of radiologists to detect micro-calcification clusters, computer-aided detection (CAD) system is much needed. Many researchers have undertaken the challenge of building an efficient CAD system and several feature extraction methods are being proposed. Most of them extract low- or mid-level features which restrict the accuracy of the overall classification. We observed that high-level features lead to a better diagnosis and convolutional neural network (CNN) is the best-known model to extract high-level features. In this paper, we propose to use a CNN architecture to do both of the feature extraction and classification task. Our proposed network was applied to both MIAS and DDSM databases, and we have achieved accuracy of 99.074 % and 99.267 %, respectively, which we believe that is the best reported so far. © Springer Nature Singapore Pte Ltd. 2018.