Removing the background using the light pattern for segmentation

In this section, we are going to develop a basic algorithm that will enable us to remove the background using a light pattern. This preprocessing gives us better segmentation. The input image without noise is as follows:

If we apply a basic threshold, we will obtain an image result like this:

We can see that the top image artifact has a lot of white noise. If we apply a light pattern and background removal technique, we can obtain an awesome result in which we can see that there are no artifacts in the top of image, like the previous threshold operation, and we will obtain better results when we have to segment. We can see the result of background removal and thresholding in the following image:

Now, how can we remove the light from our image? This is very simple: we only need a picture of our scenario without any objects, taken from exactly the same position and under the same lighting conditions that the other images were taken under; this is a very common technique in AOI because the external conditions are supervised and well-known. The image result for our case is similar to the following image:

Now, using a simple mathematical operation, we can remove this light pattern. There are two options for removing it:

• Difference
• Division

The difference option is the simplest approach. If we have the light pattern L and the image picture I, the resulting removal R is the difference between them:

R= L-I 

This division is a bit more complex, but simple at the same time. If we have the light pattern matrix L and the image picture matrix I, the result removal R is as follows:

R= 255*(1-(I/L)) 

In this case, we divide the image by the light pattern, and we have the assumption that if our light pattern is white and the objects are darker than the background carrier tape, then the image pixel values are always the same or lower than the light pixel values. The result we obtain from I/L is between 0 and 1. Finally, we invert the result of this division to get the same color direction range and multiply it by 255 to get values within the range of 0-255.

In our code, we are going to create a new function called removeLight with the following parameters:

• An input image to remove the light/background
• A light pattern, Mat
• A method, with a 0 value for difference and 1 for division

The result is a new image matrix without light/background. The following code implements the removal of the background through the use of the light pattern:

Mat removeLight(Mat img, Mat pattern, int method) 
{ 
  Mat aux; 
  // if method is normalization 
  if(method==1) 
  { 
    // Require change our image to 32 float for division 
    Mat img32, pattern32; 
    img.convertTo(img32, CV_32F); 
    pattern.convertTo(pattern32, CV_32F); 
    // Divide the image by the pattern 
    aux= 1-(img32/pattern32); 
    // Convert 8 bits format and scale
    aux.convertTo(aux, CV_8U, 255); 
  }else{ 
    aux= pattern-img; 
  } 
  return aux; 
} 

Let's explore this. After creating the aux variable to save the result, we select the method chosen by the user and pass the parameter to the function. If the method that was selected is 1, we apply the division method.

The division method requires a 32-bit float of images to allow us to divide the images and not truncate the numbers into integers. The first step is to convert the image and light pattern mat to floats of 32 bits. To convert images of this format, we can use the convertTo function of the Mat class. This function accepts four parameters; the output converted image and the format you wish to convert to the required parameters, but you can define alpha and beta parameters, which allow you to scale and shift the values following the next function, where O is the output image and I the input image:

O(x,y)=cast<Type>(α * I(x,y)+β)

The following code changes the image to 32-bit float:

// Required to change our image to 32 float for division 
Mat img32, pattern32; 
img.convertTo(img32, CV_32F); 
pattern.convertTo(pattern32, CV_32F); 

Now, we can carry out the mathematical operations on our matrix as we described, by dividing the image by the pattern and inverting the result:

// Divide the image by the pattern 
aux= 1-(img32/pattern32); 

Now, we have the result but it is required to return it to an 8-bit depth image, and then use the convert function as we did previously to convert the image's mat and scale from 0 to 255 using the alpha parameter:

// Convert 8 bits format 
aux.convertTo(aux, CV_8U, 255); 

Now, we can return the aux variable with the result. For the difference method, the development is very easy because we don't have to convert our images; we only need to apply the difference between the pattern and image and return it. If we don't assume that the pattern is equal to or greater than an image, then we will require a few checks and truncate values that can be less than 0 or greater than 255:

aux= pattern-img; 

The following images are the results of applying the image light pattern to our input image:

In the results that we obtain, we can check how the light gradient and the possible artifacts are removed. But what happens when we don't have a light/background pattern? There are a few different techniques to obtain this; we are going to present the most basic one here. Using a filter, we can create one that can be used, but there are better algorithms to learn about the background of images where the pieces appear in different areas. This technique sometimes requires a background estimation image initialization, where our basic approach can play very well. These advanced techniques will be explored in Chapter 8, Video Surveillance, Background Modeling, and Morphological Operations. To estimate the background image, we are going to use a blur with a large kernel size applied to our input image. This is a common technique used in optical character recognition (OCR), where the letters are thin and small relative to the whole document, allowing us to do an approximation of the light patterns in the image. We can see the light/background pattern reconstruction in the left-hand image and the ground truth in the right-hand:

We can see that there are minor differences in the light patterns, but this result is enough to remove the background. We can also see the result in the following image when using different images. In the following image, the result of applying the image difference between the original input image and the estimated background image computed with the previous approach is depicted:

The calculateLightPattern function creates this light pattern or background approximation:

Mat calculateLightPattern(Mat img) 
{ 
  Mat pattern; 
  // Basic and effective way to calculate the light pattern from one image 
  blur(img, pattern, Size(img.cols/3,img.cols/3)); 
  return pattern; 
} 

This basic function applies a blur to an input image by using a big kernel size relative to the image size. From the code, it is one-third of the original width and height.