A Computer Vision-Based Quality Assessment Technique for R2R Printed Silver Conductors on Flexible Plastic Substrates

Abstract

The demand for flexible large-area optoelectronic devices has been growing significantly during recent years. Roll-to-roll (R2R) printing facilitates the cost-efficient industrial production of different optoelectronic devices. Nonetheless, the performance of these devices is highly dependent on the printing quality and number of defects of R2R printed conductors. The image processing technique is an efficient nondestructive testing (NDT) methodology used to detect such defects. In this study, a computer vision-based assessment tool was utilized to visualize R2R printed silver conductors’ defects on flexible plastic substrates. A multistage defect detection technique was proposed to detect and classify both printing-induced defects and imperfections as well as the misalignment of the printed conductors with respect to the reference design. The method proved to be a very reliable approach that can be used independently or in conjunction with electrical testing methods for quality assurance purposes during the production of R2R prints.

1. Introduction

The organic electronics industry has significantly benefited from the high-volume manufacturing of R2R-based flexible electronics as a viable upscaling solution [1,2,3,4]. Moreover, R2R printing makes the cost-efficient production of opto-electronic components, including organic light-emitting diodes (OLEDs) and organic and polymer solar cells (OPVs), possible [5,6,7,8]. As the opto-electronic devices are built based on a multilayered structure design, defects and imperfections at different layers have a severe effect on the functionality of the entire system. Thus, functionality tests are conducted after the production of each layer in order to minimize the presence of defects and increase the quality of the final product [8]. Defect detection at early stages of the production of opto-electronics would increase the production yield and reduce the wastage [9]. Early-stage defect detection ensures that the defective components will not be utilized in the next stage of the production process. Moreover, it will make the recycling process of the defective components easier and more efficient.

There have been several attempts towards detecting defects in printed electronic devices and microelectromechanical systems (MEMS) based on NDT methodologies. Attempts include [9,10] implementing a machine-learning (ML) automatic defect recognition (ADR) approach with a statistically significant accuracy. Nonetheless, ML-based approaches, especially deep learning techniques, require a large-size dataset and significant annotated and labelled data. This research is a part of a European Commission-funded project titled “OLEDSOLAR Project” [11,12] aimed to develop innovative fabrication processes and in-line monitoring methodologies to detect defects in flexible conductive materials. Due to the limited dataset and sample data as well as the imbalanced defect type occurrence frequency, this study aims to explore the possibility of using NDT methodologies based on computer vision and image processing approaches for the in-line quality assessment of different steps in processing for the procurement of thin-film photovoltaics.

2. Materials and Methods

The structures of flexible resistance lines were prepared at a roll-to-roll pilot line using rotary screen printing techniques. The layout included various shapes (straight, horseshoe, round-edged meander, and straight-edged meander) with different nominal line widths of 80 µm, 100 µm, 300 µm, 500 µm, and 1000 µm, respectively (Figure 1).

For data capturing, a large area printing quality measurement system was designed and implemented. The system was based on a high-luminance line light and a large format lens and with a line scan camera with 8192 pixels. This allowed for scanning of the whole-print repeat length with a 20 μm resolution during R2R printing.

The camera (Figure 2) was tested in a laboratory environment using the camera’s manufacturer’s software by utilizing the rewinding machine for samples’ movements.

During the printing run, three different silver pastes (Asahi LS-411AW, DuPont 5025, and DuPont 5028) were used for patterning the shapes on a 125 µm thick pretreated PET Melinex ST506 substrate roll. The test layout was used to assess the printing quality for different silver pastes both visually and electrically. Each printed line was considered as an individual device and did not provide any other functionality but electrical conductivity. In a single sheet, there were 42 different resistances in both vertical and horizontal orientations with different layouts (Figure 3).

Several sample images (59 sample images including 2478 sub-images) with defects from the batch were chosen to review the different types of possible defects in the prints. As can be seen in Figure 4, there were 3 types of disconnections in the sample image that would result in failure during the testing. The defects can be sub-categorized into three types including dotted gap, wide gap, and narrow gap disconnections.

After an initial review of the defects, a gold-standard reference image was created from one of the sample images that had the least defects. The reference image was retouched to remove defects and converted into a defect-free version and into a perfect condition. This reference image was used to compare against each sample image in the batch. This task proved to be painstaking as the researchers had to retouch the image down to a pixel-level accuracy in order to eliminate the remaining defects and imperfections. Once the reference image was created, it was cropped into 42 sub-images, each containing a single resistance, and they were saved for future comparisons. Figure 5 depicts all the steps involved in the algorithm’s whole process.

As the sample images in the dataset were very large with a resolution of 8192 × 12,000 pixels, any unnecessary parts of the image needed to be removed in order to make the defect detection process faster and have overall less pixels to work on. Thus, the next step involved cropping the image particularly in the margin area of the images.

Once this step was completed, each sample image was cut into 42 sub-images, each containing a single resistance line.

The sub-images cropped from the sample image dataset were then superimposed on their corresponding counterpart from the reference image. The sub-images were then converted to greyscale images and aligned with their corresponding reference sub-images using a homography algorithm [13] to match both sub-images over each other and eliminate any misalignments or offsets between the reference and sample sub-images based on Equations (1) and (2). As all the images taken by the camera were placed over a fixed position before capturing, the alignment algorithm had to implement a minor adjustment.

$$H=\left[\begin{array}{ccc}{h}_{00}& h_{01}& h_{02}\\ h_{10}& h_{11}& h_{12}\\ h_{20}& h_{21}& h_{22}\end{array}\right]$$

(1)

If $H$ is a 3 × 3 matrix of a homography and the first set of corresponding points ($x_1$, $y_1$) are considered in the first image and ($x_2$, $y_2$) in the second image, then the homography $H$ will be as follows:

$$H=\left[\begin{array}{c}{x}_1\\ y_1\\ 1\end{array}\right]=H\left[\begin{array}{c}{x}_2\\ y_2\\ 1\end{array}\right]=\left[\begin{array}{ccc}{h}_{00}& h_{01}& h_{02}\\ h_{10}& h_{11}& h_{12}\\ h_{20}& h_{21}& h_{22}\end{array}\right]\left[\begin{array}{c}{x}_2\\ y_2\\ 1\end{array}\right]$$

(2)

Once the sub-images were aligned, the next step was to convert the image into a binary image (Figure 6) to make it ready as a suitable input for the defect detection algorithm.

The next phase of the testing was to check the quality of the sample’s sub-image in comparison to the reference image. To do this, the sub-images from the reference image were subtracted from the sub-images produced from the results of the alignment stage of the process. The remainder of this arithmetic calculation showed the difference between the reference image and the sample images, hence defining the quality (closeness to the reference) of the sample images. The algorithm gives a score in percentage on the quality of the sample image print by calculating the number of pixel differences from the reference image. The higher the presence of differential pixels, the lower the score the algorithm would give, where 100% is considered to be a perfect sample with zero differential pixels.

For the defect detection algorithm, the authors of this paper implemented a connected-component labelling (CCL) algorithm using the classical algorithm to detect connected regions in binary digital images [14].

To eliminate the remaining noise, a morphological operation (erosion) was applied before passing the image to the next step.

Once each connected component was defined and labelled, the algorithm passed through the image for a blob detection test controlled by parameters such as size, color, and shape, where the shape’s parameters consisted of three parameters including inertia, convexity, and circularity. The algorithm iterates through every connected component to see how many blobs the component contains in an iterative process. The idea is to check whether each component has either one or two blobs in it (represented by red dots in Figure 5). If it has two blobs, that means that each end is connected to the other, hence the presence of a single component. Similarly, if it contains a single blob, that means that somewhere alongside the line, we have a disconnection (see Figure 7).

3. Results

The authors of this paper have conducted pixel analysis over the images, and based on the images’ information, each pixel in the sample images was 20 μm or 0.02 mm. The algorithm’s accuracy in detecting a gap is about 360 square μm or 0.36 square mm. Anything below that will not be detected by the algorithm as a defect, i.e., there should be a minimum defective gap of 3 × 3 pixels. These findings were then cross-verified using precision measurement tools and confirmed that the algorithm’s detected pixel size deviation is very close to the gold-standard measurements of the pixel sizes, which is 19.5 μm. Moreover, caused by the limited number of defects present in the sample images, the authors of this paper also introduced artificial defects (disconnections) using an image editing tool to evaluate the algorithm with more defect types using data augmentation to ensure the accuracy of the algorithm remains high.

Table 1. Results of the output of the ADR system.

image name: Sample 075 timestamp: 2022-05-02 20:27:26.365079	20 sub-image name: CH_WO_08L264 quality %: 90.44 health condition: Defective
0 sub-image name: AH_WO_08L232_9 quality %: 90.45 health condition: Defective	21 sub-image name: CH_WO_1L264 quality %: 96.28 health condition: Healthy
1 sub-image name: AH_WO_1L232_9 quality %: 99.0 health condition: Defective	22 sub-image name: CH_WO_3L264 quality %: 98.26 health condition: Healthy
2 sub-image name: AH_WO_3L232_9 quality %: 99.19 health condition: Defective	23 sub-image name: CH_WO_5L264 quality %: 99.61 health condition: Healthy
3 sub-image name: AH_WO_5L232_9 quality % 98.19 health condition: Healthy	24 sub-image name: CH_W1L264 quality %: 96.47 health condition: Healthy
4 sub-image name: AH_W1L232_9 quality %: 98.62 health condition: Healthy	25 sub-image name: CV_WO_08L264 quality %: 98.13 health condition: Defective
5 sub-image name: AV_WO_08L232_9 quality %: 99.85 health condition: Defective	26 sub-image name: CV_WO_1L264 quality %: 98.89 health condition: Healthy

demonstrates a sample output of the ADR system consisting of each resistance line’s quality and health condition saved as a JavaScript Object Notation (JASON) file.

It was observed that from the 59 sample images and their 2478 sub-images, there were 2000 defects in which the ADR algorithm managed to detect 98% of the true positive defects.

4. Discussion and Conclusions

The developed model used computer vision techniques and image processing methodologies to not only detect defects in the printed resistance lines, but also to measure the quality of the final products compared to a gold standard.

Nonetheless, as the complexity of the defects increases, the need for more general and robust techniques such as ML is felt [15]. As a result, due to the limited image sample availability (59 sample images including 2478 sub-images) as well as the imbalanced defect type occurrence frequency, a machine learning approach could not be implemented. Thus, a solution based on image processing techniques was applied.

There were several obstacles in detecting such disconnections including the inability to use machine learning models due to the very limited dataset as well as the different types of failures in terms of their shapes. A universal disconnection detection method had to be applied to go through every line, both vertical and horizontal, and check whether the lines had a defect. Another issue was the sheer amount of noise, including dust, particles, and hairs, that made the detection very difficult as some of the resistances’ line thicknesses were similar or even smaller than the dust or hair sizes.

This study aimed to design and implement an in-line monitoring methodology and an innovative manufacturing process to detect defects in flexible conductive materials. Due to the limited dataset and sample data, this study aimed to explore the possibility of using NDT methodologies based on computer vision and image processing approaches for the in-line quality assessment of different steps in processing for the procurement of thin-film photovoltaics. A multistage ADR methodology was proposed to detect and classify both printing-induced defects and imperfections as well as the misalignment of the printed conductors with respect to the reference design. The method proved to be very reliable (98% accuracy) and can be used independently or in conjunction with electrical testing methods for quality assurance purposes during the production of R2R prints. For the image processing framework, the algorithms could be modified for new printed flexible circuits. This is achievable with a limited dataset as it relies on image processing algorithms but requires more tailoring of the algorithms, which corresponds to more development time.