Aluminum alloy die castings have high specific strength, specific stiffness, and corrosion resistance, and are widely used in automobiles, aerospace, electronic products, and medical equipment. In actual die-casting process, due to combined factors of abnormal raw material composition, unreasonable mold design, and improper process parameters, defects such as pores, shrinkage cavities, shrinkage, and slag inclusions will be generated inside aluminum alloy die castings. These defects inside die casting will further expand under action of alternating stress, and if they are not discovered in time, they will cause serious safety hazards. In order to detect internal defects of aluminum alloy die castings during production process, X-ray imaging technology is required to inspect their internal defects.
At present, nearly 80% of aluminum alloy die-casting manufacturers use manual defect determination. Evaluation criteria for manual defect determination vary from person to person, which is difficult to unify and easy to cause false detection and missed detection. In determination process, quality inspector needs to pay attention to location of die-casting defect in addition to size of defect area. Key positions of aluminum alloy die-castings include step holes, through holes, threaded holes, etc. In order to meet inspection needs of aluminum alloy die-castings, internal defect segmentation algorithm of aluminum alloy die-castings needs to segment key positions in parts in addition to internal defects. In order to meet above needs, it is necessary to carry out segmentation algorithm research based on multi-task learning around X-ray images of aluminum alloy die-castings.
In recent years, deep learning has been widely studied and applied in the fields of computer vision, natural language processing, and recommendation algorithms. Convolutional neural network (CNN) based on deep learning can automatically obtain required features for task through self-learning in complex scenes with a large amount of data. MERY D designed CNN model Xnet-Ⅱ to identify internal defects of aluminum alloy castings, proposed an ellipse generation model and adversarial generation network to simulate casting defects and expand defect samples. Researchers proposed a spatial attention bilinear convolutional neural network to classify internal defect types of aluminum alloy die castings. A railway casting X-ray defect detection algorithm based on Mask R-CNN was proposed. Proposed model can better identify bubbles and shrinkage defects in images. A casting internal defect segmentation network with adaptive depth and receptive field selection was proposed. Introduction of an adaptive depth selection module in CNN helps to identify similar defect types. Proposed model can segment and classify defects in X-ray images.
Semantic segmentation algorithm based on multi-task learning integrates multiple related segmentation tasks, extracts complementary features from a single data for related tasks by designing a unified segmentation model, and improves segmentation accuracy of each task. Researchers proposed a FusionNet multi-task segmentation model, designed a boundary perception branch to supervise boundary features of target, and improved segmentation accuracy of model in boundary area. A two-way encoding model was designed, and an attention mechanism module was introduced when extracting boundaries to improve segmentation accuracy of model at the boundary. A multi-task segmentation model is proposed to extract main features of target. Main features and edge features are supervised by designing corresponding loss functions to improve segmentation performance of model in two tasks.
There is no obvious correlation between defect segmentation task and key position segmentation task of aluminum alloy die castings. In existing multi-task segmentation algorithm research, there are few reports on multi-task segmentation algorithms with low correlation. This study conducts research on multi-task segmentation algorithms for defect segmentation and key position segmentation. First, a key position boundary-aware decoder is proposed to improve segmentation accuracy of key positions. In order to improve feature extraction ability of low-correlation tasks, a local shared encoding model is further designed to improve model. Finally, an interactive attention module is constructed to adaptively fuse key position features and their boundary features to improve segmentation accuracy of two tasks, aiming to provide a reference for defect detection and analysis of aluminum alloy die castings.
Key positions and defects of aluminum alloy die castings are shown in Figure 1. Casting Parts X-Ray Image Multi-Task Defect Segmentation Network (CXMTDS-Net) consists of three modules: Partly Parameters Sharing Two-Stream Encoder Module (PPSTSEM), Defect Segmentation Decoder (DSD) and Key Location Segmentation Decoder (KLSD), as shown in Figure 2. PPSTSEM module shares some parameters of encoding in shallow stage, and designs encoding branches for different tasks in deep stage to extract semantic features of two low-correlation tasks. In DSD module, decoding model A and decoding model B are used to hierarchically decode features in encoder to obtain defect area. In KLSD, a decoder for key location segmentation is proposed, and a key location boundary segmentation decoder is designed to improve segmentation accuracy of key locations. An interactive perception module is further designed to adaptively fuse key location features and boundary features, thereby improving performance of boundary location segmentation and boundary segmentation.
In view of complex scene and defect area boundary of aluminum alloy die castings, a deep learning-based aluminum alloy die casting X-ray image defect segmentation network (Casting Parts X-Ray Image Defect Segmentation Network, CXDS-Net) is proposed, as shown in Figure 3. Defect segmentation model uses UNet architecture as backbone network, and encoder uses ResNet101 model to improve ability to extract effective features in complex scenes.

 

Figure 1 Key positions and defect examples of aluminum alloy die castings

 

Figure 2 Multi-task segmentation model structure of aluminum alloy die castings

 

Figure 3 Defect segmentation model of aluminum alloy die castings X-ray images
Due to low correlation between die casting defect segmentation task and casting key position segmentation task, directly introducing key position segmentation decoder in defect segmentation decoder cannot meet segmentation requirements. Therefore, a decoder for key position segmentation task of die castings (Key Parts Branch, KPB) is designed, as shown in Figure 2. Key position is more obvious in X-ray image. In decoder, features extracted after original image encoding can be directly used to obtain sufficient texture features and deep semantic features at key position. In this study, feature maps of each level of original image encoding are represented by x0~x4, specific structures of decoding modules A and B in key position segmentation decoder are shown in Figure 4. Although key positions in aluminum alloy die castings have obvious features in X-ray images, shapes and positions of key positions are diverse, and there are discontinuities in some boundaries of key positions. When key position decoder is directly used for segmentation, segmentation accuracy of some segmentation results at boundaries is low. A boundary-aware decoder is proposed to segment boundaries of key positions, as shown in Figure 5. According to different encoding levels, three boundary-aware modules (Edge-Aware Branch, EAB) are designed respectively. In EAB_1, shallow feature in encoding module, x4 feature image, is used to segment key position and its boundaries at the same time. In EAB_2, three decoding modules are designed to use x2, x3, and x4 feature maps in encoding module for semantic segmentation and boundary segmentation. In EAB_3, a decoder structure consistent with KPB is designed to make full use of features of encoders at different stages. In subsequent experiments, EAB model designed based on different fusion stages will be quantitatively evaluated.

 

Figure 4 Structure of key position segmentation model of aluminum alloy die castings

 

Figure 5 Boundary-aware key position segmentation decoder strategy
In high-correlation multi-task segmentation tasks, researchers widely use shared encoders to encode features related to each task at the same time. Encoder with shared parameters under multi-task supervision can extract complementary features and improve segmentation accuracy of each task respectively. For low-correlation multi-task segmentation encoding models, in shallow encoding stage, structural features, boundary information, and texture information of die-casting defects of aluminum alloy die castings can be extracted from image. In deep encoding stage, deep semantic features are extracted for different regions. In same encoding module, effective complementary semantic information cannot be improved. This direct parameter sharing encoding method will reduce segmentation accuracy of two tasks.
In shallow feature extraction stage, texture features of die-casting defects, texture information of key positions, and boundary features are encoded simultaneously by parameter sharing. In proposed PPSTSEM, convolution layer parameters of newly designed deep coding branch are consistent with model parameters of convolution layer corresponding to original coding. According to different deep coding levels, deep coding branch in PPSTSEM can be divided into three types: only using 5_1 convolution layer, using 4_1 and 5_1 convolution layers, using 3_1, 4_1, and 5_1 convolution layers, as shown in Figure 6. In experimental phase, deep coding stages using different levels will be verified.
In order to further improve segmentation accuracy of key position segmentation task, an interactive attention module (IAM) is proposed, as shown in Figure 7. By adopting features of other tasks that are highly relevant to this task, attention mechanism module is introduced to obtain attention map of features of this task, extracted attention map is inverted to obtain complementary attention map, complementary attention map is multiplied with features from other tasks at pixel level to obtain features complementary to this task, and further added with original features at pixel level to obtain output features of module.

 

Figure 6 Local parameter sharing dual-path encoding method

 

Figure 7 Interactive perception module
A dataset of X-ray images of automotive aluminum alloy die castings was constructed. In actual automotive parts inspection process, inspectors placed parts to be inspected into X-ray image inspection system. X-ray inspection system consists of 4 parts: X-ray emission source, mechanical transmission system, digital flat panel detector and computer image processing system.
Constructed dataset contains 3,200 X-ray images that provide manual annotation results of defect areas. Annotation contents mainly include: defect area pixel level, key positions in aluminum alloy die castings and their boundary pixel level, as shown in Figure 8. Key positions in automotive aluminum alloy die castings mainly include holes to be processed, end faces, and surfaces that cooperate with other parts. In aluminum alloy die castings, if defects appear in key positions, they will cause great safety hazards to processing of next process and subsequent use of parts. In quality inspection process, it is necessary to focus on distribution of defects in such key positions.

 

Figure 8 Annotation example of X-ray image dataset of automotive aluminum alloy die castings

 

Table 1 Results of baseline model using different key position boundary-aware decoders

 

Table 2 Results of baseline model using different local parameter sharing dual-path encoders
This experiment uses CXDS-Net+ EAB_1 model as benchmark network. Experimental results show that three proposed PPSTSEMs can effectively improve segmentation accuracy of model in key position segmentation task, among which PPSTSEM_2 achieves the best in both mIoUK and mIoUE indicators.
In order to further improve segmentation accuracy at key position while ensuring defect segmentation accuracy, an interactive attention module is proposed. In key position decoding stage, the key position decoding features and key position boundary decoding features are complementarily fused through attention mechanism module to improve segmentation accuracy of each task. This experiment uses CXDSNet+EAB_1 model as benchmark network to verify effectiveness of IAM model. Results are shown in Table 3. It can be seen that after introduction of IAM in decoding stage, model has improved in both mIoUK and mIoUE indicators by complementary fusion of key position features and their boundary features, thereby verifying effectiveness of proposed IAM for key position segmentation task. Effectiveness of model after integrating each module into benchmark network is further verified. Experimental results are shown in Table 4. Experimental results show that proposed multi-task defect segmentation model can simultaneously improve segmentation accuracy of defect segmentation and key position segmentation, two tasks with low correlation, can well meet needs of segmenting defect areas and key position areas in actual quality inspection process.

 

Table 3 Results of baseline model using interactive attention module

 

Table 4 Comparison of ablation tests of multi-task segmentation models
Figure 9 shows an example of segmentation results of CXDS-Net+KPB and proposed multi-task segmentation model CXMTDS-Net for key positions and their boundaries. From segmentation results, it can be seen that CXMTDS-Net model has significantly improved segmentation performance of key positions compared with CXDS-Net+KPB. CXDS-Net+KPB only has a good segmentation effect at key positions with significant boundaries, and segmentation accuracy in areas with blurred boundaries needs to be further improved. CXMTDS-Net has better segmentation accuracy in areas where boundaries of key positions are not obvious, as shown in Figure 10. It can be seen that proposed model can achieve accurate segmentation of defect areas.

 

Figure 9 Example of key position segmentation results

 

Figure 10 Example of CXMTDS-Net segmentation results
A multi-task segmentation model CXMTDS-Net is proposed for defect segmentation and key position segmentation tasks, which can effectively improve accuracy of key position segmentation. In order to improve segmentation accuracy of salient targets, FAN D P et al. proposed a reverse attention module (RAM), see Figure 11. This module can be used to explore intrinsic connection between segmented area and its boundary, thereby improving segmentation accuracy of model for target area. Qin X B et al. proposed a residual refinement module (RRM), designed an encoding and decoding structure to correct model prediction results, thereby improving segmentation accuracy of model in boundary area. Taking CXDS-Net +EAB_2+PPSTSEM_2 as benchmark network, RAM and RRM methods were fused with benchmark network and quantitatively compared with proposed CXMTDS-Net model to verify effectiveness of proposed method in improving accuracy of key position segmentation, as shown in Figure 12.

 

Figure 11 Schematic diagram of RAM module structure

 

Figure 12 Schematic diagram of decoding module C and RMM structure

 

Table 5 Comparison of results of proposed CXMTDS-Net with other methods
Conclusion
(1) For detection of aluminum alloy casting defects, a key position boundary perception decoding module is proposed to supervise key position features and boundary area features respectively, which significantly improves key position segmentation accuracy. Compared with benchmark network, mIoUK increased by 0.7.
(2) A local parameter sharing dual-path encoder was designed to share parameters in shallow feature extraction, and to construct encoding branches for different tasks in deep encoding stage, so as to achieve efficient feature extraction of low-correlation tasks, improve mIoUK and mIoUE indicators by 1.1 and 0.7 respectively.
(3) Interactive fusion mechanism in key position and key position boundary features was explored, and an interactive attention module was developed. Attention mechanism was introduced to adaptively fuse features of key position and boundary area, effectively improving segmentation accuracy of model for key position, improving mIoUK and mIoUE indicators by 0.4 and 1.8 respectively.