The identification of minerals in petrographic thin sections is essentially required in petrological research, and is a prerequisite for further understanding of rock classification, petrogenesis, material flow and evolution history. Traditional methods rely on manual identification with optical microscope, which is costly, time-consuming, and subject to expert judgment and personal experience. Following the development of deep learning technology, it is possible for computer to automatically extract more accurate semantic information from images of petrographic thin sections. This paper proposes a deep learning-based method on petrographic thin section images for automatic mineral identification, which not only utilizes the deep convolutional neural network to extract different mineral features in the images for semantic segmentation and recognition, but also takes into account the plane polarized light images and cross polarized light images for comprehensive automatic identification. Our paper used the photomicrograph dataset of rocks for petrology teaching at Nanjing University for mineral identification and achieved the overall accuracy of 86.7% and Kappa coefficient of 0.818 demonstrating the advantage of the proposed approach compared with those of the traditional image classification methods.