Perception and computer vision for autonomous robots

Inverse Lucas Kanade Tracker – Tracking a car with robustness against illumination variance

In this project I have implemented the Inverse Lucas-Kanade template tracker. My code is tested on videos from the Visual Tracker benchmark database.
We first make a bounding box around the object to be tracked in the first frame and define that as our region of interest. Essentially, we will be matching this template to every frame, and using the affine transform we try to detect it as it moves.
The LK algorithm is a minimization type algorithm where we try to minimize the sum of the squared error between two images – the template T and the warped image I.
The adaptive brightness technique is used to make the tracking more robust when there is a fluctuation in the brightness of the object being tracked.

Inverse Lucas Kanade Tracker – Tracking Usain bolt

He was way fast than expected, the tracking results show failure at certain frames, where we reinitialized the templates to efficiently track in the entire video.
We first make a bounding box around the object to be tracked in the first frame and define that as our region of interest. Essentially, we will be matching this template to every frame, and using the affine transform we try to detect it as it moves.
The LK algorithm is a minimization type algorithm where we try to minimize the sum of the squared error between two images – the template T and the warped image I.

Inverse Lucas Kanade Tracker – Tracking a baby fighting a dragon

Tracking the baby was not easy, the video was really fast and the baby kept moving itself in orientations. This means the template which we fed initially for tracking won’t be enough. A series of templates with different orientations of the baby was used to successful tracking.
We first make a bounding box around the object to be tracked in the first frame and define that as our region of interest. Essentially, we will be matching this template to every frame, and using the affine transform we try to detect it as it moves.
The LK algorithm is a minimization type algorithm where we try to minimize the sum of the squared error between two images – the template T and the warped image I.

Lane Detection and Turn Prediction for Self Driving cars

This is an implementation of lane detection used in the self-driving cars for the application of the Lane Departure Warning System.

Homography: Four points are chosen from one every frame of the video and we perform Homography so as to obtain a top view or Bird’s view of the image captured by the camera.
Undistort the input image using the camera intrinsic parameters and apply Gaussian Blurr.
Detecting and refining the lines: Hough lines approach is used to detect the lines. In this approach, we use the polar coordinates of the image.
Turn Prediction: We have used the approach where the gradient is calculated and by taking the mean of the gradients, we check if the mean is in a particular range. Based on the range the gradients fall in we classify the direction of the car as left, right and straight.