In the last exercise, we alredy know how select the color of the lane on highway, this is very important for the camera of self-driving car. In this case, I’ll assume that the front facing camera that took the image is mounted in a fixed position on the car, such that the lane lines will always appear in the same general region of the image. Next, I’ll take advantage of this by adding a criterion to only consider pixels for color selection in the region where we expect to find the lane lines.
Check out the code below. The variables left_bottom
, right_bottom
, and apex
represent the vertices of a triangular region that I would like to retain for my color selection, while masking everything else out. Here I’m using a triangular mask to illustrate the simplest case, but later you’ll use a quadrilateral, and in principle, you could use any polygon.
import matplotlib.pyplot as plt import matplotlib.image as mpimg import numpy as np # Read in the image and print some stats image = mpimg.imread('test.jpg') print('This image is: ', type(image), 'with dimensions:', image.shape) # Pull out the x and y sizes and make a copy of the image ysize = image.shape[0] xsize = image.shape[1] region_select = np.copy(image) # Define a triangle region of interest # Keep in mind the origin (x=0, y=0) is in the upper left in image processing # Note: if you run this code, you'll find these are not sensible values!! # But you'll get a chance to play with them soon in a quiz left_bottom = [0, 539] right_bottom = [900, 300] apex = [400, 0] # Fit lines (y=Ax+B) to identify the 3 sided region of interest # np.polyfit() returns the coefficients [A, B] of the fit fit_left = np.polyfit((left_bottom[0], apex[0]), (left_bottom[1], apex[1]), 1) fit_right = np.polyfit((right_bottom[0], apex[0]), (right_bottom[1], apex[1]), 1) fit_bottom = np.polyfit((left_bottom[0], right_bottom[0]), (left_bottom[1], right_bottom[1]), 1) # Find the region inside the lines XX, YY = np.meshgrid(np.arange(0, xsize), np.arange(0, ysize)) region_thresholds = (YY > (XX*fit_left[0] + fit_left[1])) & \ (YY > (XX*fit_right[0] + fit_right[1])) & \ (YY < (XX*fit_bottom[0] + fit_bottom[1])) # Color pixels red which are inside the region of interest region_select[region_thresholds] = [255, 0, 0] # Display the image plt.imshow(region_select) # uncomment if plot does not display # plt.show()
Combining Color and Region Selections
Now you’ve seen how to mask out a region of interest in an image. Next, let’s combine the mask and color selection to pull only the lane lines out of the image.
Check out the code below. Here we’re doing both the color and region selection steps, requiring that a pixel meet both the mask and color selection requirements to be retained.
import matplotlib.pyplot as plt import matplotlib.image as mpimg import numpy as np # Read in the image image = mpimg.imread('test.jpg') # Grab the x and y sizes and make two copies of the image # With one copy we'll extract only the pixels that meet our selection, # then we'll paint those pixels red in the original image to see our selection # overlaid on the original. ysize = image.shape[0] xsize = image.shape[1] color_select= np.copy(image) line_image = np.copy(image) # Define our color criteria red_threshold = 0 green_threshold = 0 blue_threshold = 0 rgb_threshold = [red_threshold, green_threshold, blue_threshold] # Define a triangle region of interest (Note: if you run this code, # Keep in mind the origin (x=0, y=0) is in the upper left in image processing # you'll find these are not sensible values!! # But you'll get a chance to play with them soon in a quiz 😉 left_bottom = [0, 539] right_bottom = [900, 300] apex = [400, 0] fit_left = np.polyfit((left_bottom[0], apex[0]), (left_bottom[1], apex[1]), 1) fit_right = np.polyfit((right_bottom[0], apex[0]), (right_bottom[1], apex[1]), 1) fit_bottom = np.polyfit((left_bottom[0], right_bottom[0]), (left_bottom[1], right_bottom[1]), 1) # Mask pixels below the threshold color_thresholds = (image[:,:,0] < rgb_threshold[0]) | \ (image[:,:,1] < rgb_threshold[1]) | \ (image[:,:,2] < rgb_threshold[2]) # Find the region inside the lines XX, YY = np.meshgrid(np.arange(0, xsize), np.arange(0, ysize)) region_thresholds = (YY > (XX*fit_left[0] + fit_left[1])) & \ (YY > (XX*fit_right[0] + fit_right[1])) & \ (YY < (XX*fit_bottom[0] + fit_bottom[1])) # Mask color selection color_select[color_thresholds] = [0,0,0] # Find where image is both colored right and in the region line_image[~color_thresholds & region_thresholds] = [255,0,0] # Display our two output images plt.imshow(color_select) plt.imshow(line_image) # uncomment if plot does not display # plt.show()
In the next quiz, you can vary your color selection and the shape of your region mask (vertices of a triangle left_bottom
, right_bottom
, and apex
), such that you pick out the lane lines and nothing else.
After combine region-making and color-classification:
import matplotlib.pyplot as plt import matplotlib.image as mpimg import numpy as np # Read in the image image = mpimg.imread('test.jpg') # Grab the x and y size and make a copy of the image ysize = image.shape[0] xsize = image.shape[1] color_select = np.copy(image) line_image = np.copy(image) # Define color selection criteria # MODIFY THESE VARIABLES TO MAKE YOUR COLOR SELECTION red_threshold = 200 green_threshold = 200 blue_threshold = 200 rgb_threshold = [red_threshold, green_threshold, blue_threshold] # Define the vertices of a triangular mask. # Keep in mind the origin (x=0, y=0) is in the upper left # MODIFY THESE VALUES TO ISOLATE THE REGION # WHERE THE LANE LINES ARE IN THE IMAGE left_bottom = [115, 540] right_bottom = [800, 540] apex = [455, 300] # Perform a linear fit (y=Ax+B) to each of the three sides of the triangle # np.polyfit returns the coefficients [A, B] of the fit fit_left = np.polyfit((left_bottom[0], apex[0]), (left_bottom[1], apex[1]), 1) fit_right = np.polyfit((right_bottom[0], apex[0]), (right_bottom[1], apex[1]), 1) fit_bottom = np.polyfit((left_bottom[0], right_bottom[0]), (left_bottom[1], right_bottom[1]), 1) # Mask pixels below the threshold color_thresholds = (image[:,:,0] < rgb_threshold[0]) | \ (image[:,:,1] < rgb_threshold[1]) | \ (image[:,:,2] < rgb_threshold[2]) # Find the region inside the lines XX, YY = np.meshgrid(np.arange(0, xsize), np.arange(0, ysize)) region_thresholds = (YY > (XX*fit_left[0] + fit_left[1])) & \ (YY > (XX*fit_right[0] + fit_right[1])) & \ (YY < (XX*fit_bottom[0] + fit_bottom[1])) # Mask color and region selection color_select[color_thresholds | ~region_thresholds] = [0, 0, 0] # Color pixels red where both color and region selections met line_image[~color_thresholds & region_thresholds] = [255, 0, 0] # Display the image and show region and color selections plt.imshow(image) x = [left_bottom[0], right_bottom[0], apex[0], left_bottom[0]] y = [left_bottom[1], right_bottom[1], apex[1], left_bottom[1]] plt.plot(x, y, 'b--', lw=4) plt.imshow(color_select) plt.imshow(line_image)
Final result we have:
