AN END-TO-END STEEL SURFACE DEFECT DETECTION APPROACH VIA FUSING MULTIPLE HIERARCHICAL FEATURES

Abstract

A complete defect detection task aims to achieve the specific class and precise location of each defect in an image, which makes it still challenging for applying this task in practice. The defect detection is a composite task of classification and location, leading to related methods are often hard to take into account the accuracy of both. And the implementation of defect detection depends on a special detection dataset which contains expensive manual annotations. In this paper, we proposed a novel defect detection system based on deep learning and focused on a practical industrial application: steel plate defect inspection. In order to achieve strong classification-ability, this system employs a baseline convolution neural network (CNN) to generate feature maps at each stage. And then the proposed multilevel-feature fusion network (MFN) combines multiple hierarchical features into one feature, which can include more location details of defects. Based on these multilevel features, a region proposal network (RPN) is adopted to generate regions of interest (ROIs). For each ROI, a detector, consisting of a classifier and a bounding box regressor, produces the final detection results. Finally, we set up a defect detection dataset NEU-DET for training and evaluating our method. On the NEU-DET, our method achieves 74.8/82.3 mAP with baseline networks ResNet34/50 by using 300 proposals. In addition, by using only 50 proposals, our method can detect at 20 fps on a single GPU and reach 92% of the above performance, hence the potential for real-time detection.  

Existing System

HOG and LBP feature based SVM.

Proposed System

A single-scale image of an arbitrary size is processed by a CNN, and the convolutional feature maps at each stage of the ConvNet are produced (ConvNet represents the convolutional part of a CNN). We extract multiple feature maps and then aggregate them in the same dimension by using a lightweight multilevel-feature fusion network, named MFN. In this way, MFN features have the characteristics from several hierarchical levels of ConvNet. Next, Region Proposal Network (RPN) is employed to generate region proposals (ROIs) over the MFN feature. Finally, the MFN feature corresponding to each ROI is transformed into a fixed-length feature through the ROI pooling and the global average pooling (GAP) layers. The feature is fed into two fully-connected layers. One is a one-of-(C+1) defect classification layer (“cls”) and the other is a bounding-box regression layer (“loc”).  

Architecture

BLOCK DIAGRAM