OpenCV based line following

    Nowadays, autopilot courses for cars are very popular. This nano-grade from Udacity is probably the most well-known option.

    Many people learn from it and spread their decisions. I also could not pass by and carried away.

    The difference is that the course involves the development of an algorithm based on the data provided, and I did everything for my robot .

    The first task that students face in studying computer vision is to follow a line on the road. Many articles are written on this topic, here are some of the most detailed:

    Everything is pretty simple and the scheme of work comes down to several points:

    I stuck white tape on the floor and got down to business.

    In your works mentioned, the task was also to find the yellow line, so they worked with HLS and HSV colors. Since my line was only white, I decided not to bother with this and limit myself to a black and white filter.


    Immediately began problems with geometry. The students in the pictures are striking the arrow into the horizon. Still, it detects many lines that the authors had to combine. However, their lines were well directed, and there was no garbage in the pictures.

    I have a completely different picture. The geometry of the tape band was far from straight. Glare on the floor generated noises.

    After applying Canny, it turned out like this:

    And the lines of Hafa were like this:


    Strengthening the criteria, we managed to eliminate garbage, but almost all the lines found on the strip disappeared. To rely on such tiny stretches would be stupid.

    In general, the results were extremely unstable, and it occurred to me to try a different approach.

    Instead of lines, I began to look for contours. Having made the assumption that the largest circuit - this is electrical tape, managed to get rid of the garbage. (Then it turned out that the large white plinth occupied more space in the frame than electrical tape. I had to cover it with a sofa cushion).
    If we take the minimum rectangle bounding the contour, then the average longitudinal line is very well suited to the role of the motion vector.


    The second problem was with the lighting. I very successfully laid one side of the track in the shadow of the sofa and it was absolutely impossible to process the photo of the whole track with the same settings. As a result, I had to implement a dynamic cutoff on a black and white filter. The algorithm is as follows - if after applying the filter in the picture too much white (more than 10%), then the threshold should be raised. If too little (less than 3%) - omit. Practice has shown that, on average, in 3-4 iterations it is possible to find the optimal cutoff.

    Magic numbers are in a separate config (see below), you can play with them in search of optimum.

    defbalance_pic(image):global T
        ret = None
        direction = 0for i in range(0, tconf.th_iterations):
            rc, gray = cv.threshold(image, T, 255, 0)
            crop = Roi.crop_roi(gray)
            nwh = cv.countNonZero(crop)
            perc = int(100 * nwh / Roi.get_area())
            logging.debug(("balance attempt", i, T, perc))
            if perc > tconf.white_max:
                if T > tconf.threshold_max:
                    breakif direction == -1:
                    ret = crop
                T += 10
                direction = 1elif perc < tconf.white_min:
                if T < tconf.threshold_min:
                    breakif  direction == 1:
                    ret = crop
                T -= 10
                direction = -1else:
                ret = crop
                breakreturn ret      

    Having adjusted machine vision, it was possible to move on to the movement itself. The algorithm was as follows:

    • 0.5 seconds go straight
    • make a photo
    • find a vector
    • if the beginning of the vector is displaced relative to the center of the picture, we slightly taxi in the right direction
    • if the tilt angle of the vector deviates from the vertical more than necessary - we rush in the right direction
    • if it happened suddenly that the strip disappeared from the frame, we made an assumption that we passed the turn and began to turn in the direction of the last steering or tilting of the vector at the previous step

    Abbreviated version of the code (Full - on Gitkhab ):

    defcheck_shift_turn(angle, shift):
        turn_state = 0if angle < tconf.turn_angle or angle > 180 - tconf.turn_angle:
            turn_state = np.sign(90 - angle)
        shift_state = 0if abs(shift) > tconf.shift_max:
            shift_state = np.sign(shift)
        return turn_state, shift_state
    defget_turn(turn_state, shift_state):
        turn_dir = 0
        turn_val = 0if shift_state != 0:
            turn_dir = shift_state
            turn_val = tconf.shift_step if shift_state != turn_state else tconf.turn_step
        elif turn_state != 0:
            turn_dir = turn_state
            turn_val = tconf.turn_step
        return turn_dir, turn_val                
            last_turn = 0
            last_angle = 0for i in range(0, iterations):
                a, shift = get_vector()
                if a isNone:
                    if last_turn != 0:
                        a, shift = find_line(last_turn)
                        if a isNone:
                            breakelif last_angle != 0:
                        logging.debug(("Looking for line by angle", last_angle))
                        turn(np.sign(90 - last_angle), tconf.turn_step)
                turn_state, shift_state = check_shift_turn(a, shift)
                turn_dir, turn_val = get_turn(turn_state, shift_state)
                if turn_dir != 0:
                    turn(turn_dir, turn_val)
                    last_turn = turn_dir
                    last_turn = 0
                last_angle = a


    Roughly, but confidently, the tank crawls along the trajectory:

    But he collected a gif from the debug graphics:

    Algorithm Settings

    ## Picture settings# initial grayscale threshold
    threshold = 120# max grayscale threshold
    threshold_max = 180#min grayscale threshold
    threshold_min = 40# iterations to find balanced threshold
    th_iterations = 10# min % of white in roi
    white_min=3# max % of white in roi
    white_max=12## Driving settings# line angle to make a turn
    turn_angle = 45# line shift to make an adjustment
    shift_max = 20# turning time of shift adjustment
    shift_step = 0.125# turning time of turn
    turn_step = 0.25# time of straight run
    straight_run = 0.5# attempts to find the line if lost
    find_turn_attempts = 5# turn step to find the line if lost
    find_turn_step = 0.2# max # of iterations of the whole tracking
    max_steps = 100

    The code is on Github .

    Also popular now: