Machine vision detection of foreign objects in coal using deep learning

Wang Weidong; Zhang Kanghui; Lü Ziqi; Gu Zhaochuang; Qian Hanwen; Zhang Qingyi

doi:10.19606/j.cnki.jmst.2021.01.013

Volume 6 Issue 1

Mar. 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Mining Science and Technology > 2021 > 6(1): 115-123

Wang Weidong, Zhang Kanghui, Lü Ziqi, Gu Zhaochuang, Qian Hanwen, Zhang Qingyi. Machine vision detection of foreign objects in coal using deep learning[J]. Journal of Mining Science and Technology, 2021, 6(1): 115-123. doi: 10.19606/j.cnki.jmst.2021.01.013

Citation:

PDF( 10882 KB)

Machine vision detection of foreign objects in coal using deep learning

doi: 10.19606/j.cnki.jmst.2021.01.013

School of Chemical & Environmental Engineering, China University of Mining & Technology-Beijing, Beijing 100083, China

Received Date: 2020-05-06
Rev Recd Date: 2020-09-21
Publish Date: 2021-02-01

Abstract

Abstract

In the process of coal processing, due to the limitation of mining conditions, a large number of foreign objects are mixed into raw coal, which leads to clogging of heavy medium system pipelines, seriously affecting the production efficiency and restricting the improvement of coal quality. In order to effectively solve the impact of foreign objects in coal, machine vision technology is used to complete the detection and removal of foreign objects. Owing to irregular shapes and complex features for foreign objects in coal, a multi-scale feature fusion semantic segmentation network structure is proposed for end-to-end detection of images of foreign objects collected at the actual production site. In the network, the encoder uses residual convolutional blocks to extract features and captures context information, and the multi-scale feature map obtained by the pooling operation is fused at the decoder. The encoder and decoder are connected by layer jump to better combine the background semantic information of images for end-to-end training. The cause of model misdetection was analyzed through the visualization method of class activation map, a loss function was proposed to alleviate the model misdetection caused by the surface and background interference of coal and gangue, and the results were refined using conditional random fields to get a binary image. The experimental results show that the model can effectively segment the foreign bodies in the coal gangue system, and the mean intersection over union of the model on the test set is 77.83 %.
- foreign objects detection,
- machine vision,
- deep learning,
- semantic segmentation,
- coal

FullText(HTML)

References(25)

References

[1]	柴利文. 同煤集团降低出口煤杂物含量的措施[J]. 煤质技术, 2008(1): 4-5. doi: 10.3969/j.issn.1007-7677.2008.01.002 Chai Liwen. Tongmei Group's measures to reduce the content of exported coal debris[J]. Coal Quality Technology, 2008(1): 4-5. doi: 10.3969/j.issn.1007-7677.2008.01.002
[2]	邓洪刚, 王晓薇. 多煤种混配条件下除杂物系统的应用与改进[J]. 中国新技术新产品, 2011(7): 32-32. doi: 10.3969/j.issn.1673-9957.2011.07.026 Deng Honggang, Wang Xiaowei. Application and improvement of debris removal system under multi-coal blending conditions[J]. China New Technology & New Products, 2011(7): 32-32. doi: 10.3969/j.issn.1673-9957.2011.07.026
[3]	李文利, 朱子祺. 石圪台选煤厂杂物治理措施[J]. 煤炭加工与综合利用, 2011(4): 9-10. doi: 10.3969/j.issn.1005-8397.2011.04.003 Li Wenli, Zhu Ziqi. Managing measures of sundry articles in getai coal preparation plant[J]. Coal Processing and Comprehensive Utilization, 2011(4): 9-10. doi: 10.3969/j.issn.1005-8397.2011.04.003
[4]	张万青, 李晓燕. 完善除杂设施严把产品质量关[J]. 煤质技术, 2009(2): 31-32. doi: 10.3969/j.issn.1007-7677.2009.02.012 Zhang Wanqing, Li Xiaoyan. Discussion on advancing the eliminating impurities equipment and improving the quality of products[J]. Goal Quality and Technology, 2009(2): 31-32. doi: 10.3969/j.issn.1007-7677.2009.02.012
[5]	王卫东, 张康辉, 吕子奇, 等. 基于机器视觉的煤中杂物智能分选系统研究[J]. 选煤技术, 2020(2): 87-91. https://www.cnki.com.cn/Article/CJFDTOTAL-XMJS202002021.htm Wang Weidong, Zhang Kanghui, Lü Ziqi, et al. Research on intelligent sorting system of coal debris based on machine vision[J]. Coal Preparation Technology, 2020(2): 87-91. https://www.cnki.com.cn/Article/CJFDTOTAL-XMJS202002021.htm
[6]	栾永春. 基于视频分析的煤矿井下皮带异物检测系统[J]. 科技经济导刊, 2017(17): 70. https://www.cnki.com.cn/Article/CJFDTOTAL-JJKJ201717053.htm Luan Yongchun. Foreign body detection system of coal mine belt based on video analysis[J]. Science & Technology Economic Guide, 2017(17): 70. https://www.cnki.com.cn/Article/CJFDTOTAL-JJKJ201717053.htm
[7]	郜振国. 煤矿井下运输异物检测关键技术研究[D]. 徐州: 中国矿业大学, 2018.
[8]	Rong D, Xie L, Ying Y, et al. Computer vision detection of foreign objects in walnuts using deep learning[J]. Computers and Electronics in Agriculture, 2019, 162: 1001-1010. doi: 10.1016/j.compag.2019.05.019
[9]	Zhao X H, Guo X Y, Luo J, et al. Efficient detection method for foreign fibers in cotton[J]. Information Processing in Agriculture, 2018, 5(3): 320-328. doi: 10.1016/j.inpa.2018.04.002
[10]	Ronneberger O, Fischer P, Brox T, et al. U-Net: convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer Assisted Intervention, 2015: 234-241.
[11]	Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
[12]	He K, Zhang X, Ren S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916. doi: 10.1109/TPAMI.2015.2389824
[13]	Krizhevsky A, Sutskever I, Hinton G E.Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems, 2012: 1097-1105.
[14]	Long J, Shelhamer E, Darrell T.Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3431-3440.
[15]	Badrinarayanan V, Kendall A, Cipolla R.Segnet: A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. doi: 10.1109/TPAMI.2016.2644615
[16]	Chen L C, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4): 834-848. doi: 10.1109/TPAMI.2017.2699184
[17]	张帆, 李亚杰, 孙晓辉. 无线感知与视觉融合的井下目标跟踪定位方法[J]. 矿业科学学报, 2018, 3(5): 484-491. http://www.cnki.com.cn/Article/CJFDTotal-KYKX201805009.htm Zhang Fan, Li Yajie, Sun Xiaohui. A novel method of mine target tracking and location based on wireless sensor and visual recognition[J]. Journal of Mining Science and Technology, 2018, 3(5): 484-491. http://www.cnki.com.cn/Article/CJFDTotal-KYKX201805009.htm
[18]	郑晶, 曹子原, 姜天琪, 等. 基于深度信念神经网络的微震波到时拾取方法[J]. 矿业科学学报, 2018, 3(6): 521-526. http://www.cnki.com.cn/Article/CJFDTotal-KYKX201806002.htm Zheng Jing, Cao Ziyuan, Jiang Tianqi, et al. Deep belief neural network-based arrival picking for micro- seismic data[J]. Journal of Mining Science and Technology, 2018, 3(6): 521-526. http://www.cnki.com.cn/Article/CJFDTotal-KYKX201806002.htm
[19]	杨小彬, 王逍遥, 周世禄, 等. 基于改进广义回归神经网络的工作面低氧预测模型研究[J]. 矿业科学学报, 2019, 4(5): 434-440. doi: 10.19606/j.cnki.jmst.2019.05.008 Yang Xiaobin, Wang Xiaoyao, Zhou Shilu, et al. Prediction model of working face hypoxia based on improved generalized regression neural network[J]. Journal of Mining Science and Technology, 2019, 4(5): 434-440. doi: 10.19606/j.cnki.jmst.2019.05.008
[20]	Guan S, Khan A A, Sikdar S, et al. Fully Dense UNet for 2-D sparse photoacoustic tomography artifact removal[J]. IEEE Journal of Biomedical and Health Informatics, 2019, 24(2): 568-576. http://www.ncbi.nlm.nih.gov/pubmed/31021809
[21]	吴守鹏. 基于机器视觉的运煤皮带异物识别方法研究[D]. 徐州: 中国矿业大学, 2019.
[22]	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[23]	左宗成, 张文, 张东映. 融合可变形卷积与条件随机场的遥感影像语义分割方法[J]. 测绘学报, 2019, 48(6): 718-726. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201906007.htm Zuo Zongcheng, Zhang Wen, Zhang Dongying. A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 718-726. https://www.cnki.com.cn/Article/CJFDTOTAL-CHXB201906007.htm
[24]	张新峰, 郭宇桐, 蔡轶珩, 等. 基于DCNN和全连接CRF的舌图像分割算法[J]. 北京航空航天大学学报, 2019, 45(12): 2364-2374. https://www.cnki.com.cn/Article/CJFDTOTAL-BJHK201912003.htm Zhang Xinfeng, Guo Yutong, Cai Yiheng, et al. A tongue image segmentation algorithm based on DCNN and fully connected CRF[J]. Journal of Beijing University of Aeronautics and Astronautics, 2019, 45(12): 2364-2374. https://www.cnki.com.cn/Article/CJFDTOTAL-BJHK201912003.htm
[25]	李承珊. 基于深度卷积神经网络的图像语义分割研究[D]. 北京: 中国科学院光电技术研究所, 2019.

Relative Articles

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(11) / Tables(3)

Get Citation

PDF

XML

Article Metrics

Article views (1147) PDF downloads(88)

Machine vision detection of foreign objects in coal using deep learning

doi: 10.19606/j.cnki.jmst.2021.01.013

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Proportional views

Export File

Citation

Format

Content