MaixCAM MaixPy Line Tracking

Before reading this article, make sure you already know how to develop MaixCAM. For details, please read Quick Start.

Introduction

In vision applications, the function of tracking line is often required in applications such as line-following robot. In this article, we will describe:

• How to use MaixPy to tracking line.

• How to tracking line using MaixCam's default application

How to use MaixPy to tracking line

The maix.image.Image module in MaixPy provides the get_regression method, which can conveniently tracking line.

Code example

A simple example of finding and drawing a line.

from maix import camera, display, image

cam = camera.Camera(320, 240)
disp = display.Display()

# thresholds = [[0, 80, 40, 80, 10, 80]] # red
thresholds = [[0, 80, -120, -10, 0, 30]] # green
# thresholds = [[0, 80, 30, 100, -120, -60]] # blue

while 1:
    img = cam.read()

    lines = img.get_regression(thresholds, area_threshold = 100)
    for a in lines:
        img.draw_line(a.x1(), a.y1(), a.x2(), a.y2(), image.COLOR_GREEN, 2)
        theta = a.theta()
        rho = a.rho()
        if theta > 90:
            theta = 270 - theta
        else:
            theta = 90 - theta
        img.draw_string(0, 0, "theta: " + str(theta) + ", rho: " + str(rho), image.COLOR_BLUE)

    disp.show(img)

Steps:

1. import image, camera, display modules

   from maix import image, camera, display
   

2. Initialize camera and display

   cam = camera.Camera(320, 240) # Initialise camera, output resolution 320x240 in RGB format.
   disp = display.Display()
   

3. Get the image from the camera and display it

   while 1:
       img = cam.read()
       disp.show(img)
   

4. Call the get_regression method to find the straight line in the camera image and draw it to the screen

   lines = img.get_regression(thresholds, area_threshold = 100)
   for a in lines:
      img.draw_line(a.x1(), a.y1(), a.x2(), a.y2(), image.COLOR_GREEN, 2)
      theta = a.theta()
      rho = a.rho()
      if theta > 90:
         theta = 270 - theta
      else:
         theta = 90 - theta
      img.draw_string(0, 0, "theta: " + str(theta) + ", rho: " + str(rho), image.COLOR_BLUE)
   

   • img is the camera image read via cam.read(), when initialised as cam = camera.Camera(320, 240), the img object is an RGB image with a resolution of 320x240.
   • img.get_regression is used to find straight lines, thresholds is a list of colour thresholds, each element is a colour threshold, multiple thresholds are passed in if multiple thresholds are found at the same time, and each colour threshold has the format [L_MIN, L_MAX, A_MIN, A_MAX, B_MIN, B_MAX], where L, A, B are the three channels of LAB colour space, L channel is the luminance, A channel is the red-green channel, B channel is the blue-yellow channel. pixels_threshold is a pixel area threshold used to filter some unwanted straight lines.
   • for a in lines is used to iterate through the returned Line objects, where a is the current Line object. Normally the get_regression function will only return one Line object, but if you need to find more than one line, try the find_line method.
   • Use img.draw_line to draw the found line, a.x1(), a.y1(), a.x2(), a.y2() represent the coordinates of the ends of the line.
   • Use img.draw_string to show the angle between the line and the x-axis in the upper left corner, and a.theta() is the angle between the line and the y-axis, which is converted to theta for easier understanding, a.rho() is the length of the vertical line from the origin to the line.

5. Run the code through the maixvision, you can find the line, look at the effect!

   

Common Parameter Explanations

Here are explanations of commonly used parameters. If you cannot find parameters that can implement your application, you may need to consider using other algorithms or extending the required functionality based on the current algorithm's results.

This article introduces commonly used methods. For more APIs, please see the image section of the API documentation.

Increasing the speed of line tracking

Here are a few ways to increase the speed of line tracking

1. Choose a suitable resolution

   The larger the resolution, the slower the calculation speed, you can choose a more suitable resolution according to the recognition distance and accuracy requirements.

2. Use gray scale image

   When using gray scale recognition, the algorithm will only process one channel, there is a faster recognition speed, in the environment of a single color will be very useful. Note that only l_min and l_max are valid when passing thresholds to get_regression when using gray scale image recognition.

   Methods for get gray scale image:

   # Example 1
   cam = camera.Camera(320, 240， image.Format.FMT_GRAYSCALE)    	# Support after MaixPy v4.2.1
   gray_img = cam.read()											# get gray scale image
   
   # Example 2
   cam = camera.Camera(320, 240)
   img = cam.read()
   gray_img = img.to_format(image.Format.FMT_GRAYSCALE)			# get gray scale image
   

How to tracking line using MaixCam's default application

To quickly verify the line tracking functionality, you can use the line_tracking application provided by MaixCam to experience the line finding effect.

How to use it

1. Select and open the Line tracking application.
2. Click on the line in the screen that needs to be identified and the colour of the line will be displayed on the left hand side
3. Click on the colour to be detected on the left (the colour below L A B in the screen)
4. The line will be identified and the coordinates and angle of the line will be output from the serial port.

Demo

Advanced operations

Manual adjustment of LAB threshold to tracking line

The application provides manual setting of LAB threshold to tracking line accurately.

Steps:

1. Click the options icon in the bottom-left corner to enter configuration mode.
2. Point the camera at the object you need to find, click on the target object on the screen, and the left side will display a rectangular frame of the object's color and show the LAB values of that color.
3. Click on the bottom options L Min, L Max, A Min, A Max, B Min, B Max. After clicking, a slider will appear on the right side to set the value for that option. These values correspond to the minimum and maximum values of the L, A, and B channels in the LAB color format, respectively.
4. Referring to the LAB values of the object color calculated in step 2, adjust L Min, L Max, A Min, A Max, B Min, B Max to appropriate values to identify the corresponding color blobs. For example, if LAB = (20, 50, 80), since L=20, to accommodate a certain range, set L Min=10 and L Max=30. Similarly, since A=50, set A Min=40 and A Max=60. Since B=80, set B Min=70 and B Max=90.

Getting Detection Data via Serial Protocol

The line tracking application supports reporting detected straight line information via the serial port (default baud rate is 115200).

Since only one report message is sent, we can illustrate the content of the report message with an example.

For instance, if the report message is:

AA CA AC BB 0E 00 00 00 00 E1 09 FC 01 01 00 E9 01 6F 01 57 00 C1 C6

• AA CA AC BB: Protocol header, fixed content

• 0E 00 00 00: Data length, the total length excluding the protocol header and data length, here means the length is 14.

• E1: Flag bit, used to identify the serial message flag

• 09: Command type, for the line tracking application, this value is fixed at 0x09.

• FC 01 01 00 E9 01 6F 01 57 00: The coordinates and angle information for both ends of line, with each value represented as a 2-byte value in little-end format. FC 01 and 01 00 indicate that the coordinates of the first endpoint are (508, 1), E9 01 and 6F 01 indicate that the coordinates of the second endpoint are (489, 367), and 57 00 indicates that the angle of the line to the x-axis is 87 degrees

• C1 C6: CRC checksum value, used to verify if the frame data has errors during transmission.