Abstract: In view of the requirement to separate gangue from coal in coal preparation factory and avoid complex artificial feature design process based on computer vision in the past, an end-to-end gangue detection method by deep learning based on YOLOv3 is proposed.In order to reduce the size and speed up the operation of YOLOv3, depth-wise separable convolution and transpose convolution are used to improve the backbone network of YOLOv3.In order to improve feature integration from different levels, the spatial pyramid pooling layer is added to the model.For the purpose of accelerating convergence and accurately locating objects in the model, balanced L1 loss and distance-IoU loss are introduced.The results show that the proposed algorithm can accurately detect the gangue from the mixture of coal and gangue in real time, which provides an effective guarantee for improving the quality of coal and sorting efficiency.