In this post I show how, from a live camera feed, one can find a marker of a specific color using the basic OpenCV2 manipulations in python.
Suppose that you have a flying robot (like an AR.Drone, for example) that gives you a camera feed. You want the robot to wander around randomly in the search of a specific color in the environment. The OpenCV code should be able, for every image, to find the biggest blob of a specific color and return its position in the image for further computation. Moreover, this color detection have to be done very fast, because the robot must react very quickly.
Sebastien Audet and myself recently needed to do this very basic task using OpenCV in Python. Since we believe that this is a very generic task, I decided to put the code in order to stop reinventing the wheel all the time.
One thing is important to know though: There is two OpenCV versions in Python, which are cv2 and the old cv. Using the old one is not recommended, and using a mix of the two is even worst. The main notable difference between them is the data type that are used: While the old OpenCV uses various image formats and is mainly procedural, the new once have a more object-oriented usage and it manipulates only numeric matrices. Therefore, OpenCV2 is much easier to read, learn, manipulate and comprehend.
Thus, after wandering around a bit, we learned how to use OpenCV2 only. I assure you, now that I discovered this new version, I will never look back again.
Now, about this OpenCV2 tutorial… What color are we searching for? Let’s say flashing green (the room where we were working had a big green painted concrete pillar). A simple HSV image analyzer showed us that green was between (35,50,50) and (60,255,255). So, we defined our color thresholds. OpenCV2 only uses numpy matrices, so:
1 2 3 4 5 6 7
Pay attention to the last line here. cv_image is an image from our camera feed that is in the old OpenCV format. Why? Simply because I used ROS' cv_bridge to convert the camera feed from the ROS format to an OpenCV format, but unfortunately this bridge only return old OpenCV image types. Note also the ‘[:,:]’ that would normally be useless because it only says that we need the whole table. We saw in various forums that if we don’t do like that while converting to new OpenCV2 format (with the help of the np.asarray function), crashes may occur sometimes.
At that point, we defined our color thresholds and have a BGR frame (that is, a Blue-Green-Red matrix) that we need to scan. In order to find the pixels within the specific color thresholds, we will convert the frame to an HSV format.
There is multiple format convertion constants with cv2, and we noticed that the constants naming convention is much better with OpenCV2 than with its predecessor. One can easily find the different conversion formats, since they all look alike (cv2.COLOR_BGR2HSV, cv2.COLOR_HSV2BGR, cv2.COLOR_RGB2HSV, cv2.COLOR_HSV2GRAY, and so on).
Now that we have the good image format, let’s apply a threshold to set 1 to every pixels within the color range and 0 to the other pixels:
This returns us a gray scaleds image with each value set only to 1 or 0.
Since the image certainly have noisy pixels, we will erode and dilate it to make noise disapear and to fill holes within structural patterns in the image. There is multiple way to do that. One is to erode or to dilate once. Another one is called opening or closing the image, which is a combination of an erosion and of a dilation (opening is an erosion followed by a dilation, and a closing is the other way around).
Here’s how it’s done in Python. We need a structuring element to make these operations. Again, cv2 constants are self-descriptives. Also, notice that the parameter ‘iterations’ is optional and that there is multiple defaulted arguments. We would be stuck in other languages, but in python because we can simply name the argument as we call for a function, and the language processor will do the rest. (See there)
1 2 3 4
Here’s the link to these functions' documentation. You will find all the optional parameters and the different shapes of the structuring element, which are infinite since OpenCV2 allows custom elements.
What if you want to show the modified image? A simple imshow will do. Although, we recommend using a wait, since live feeds tends to freeze if you don’t… Don’t wait for a better answer, I don’t have one. ;-)
Now, we will find the contours of the white spots in the image, find the biggest area and draw a bounding rectangle around it. Here’s the code, and I refer you all to a nice blog of Abid Rahman, who have done a great job documenting these functions. Enjoy!
http://opencvpython.blogspot.ca/2012/06/hi-this-article-is-tutorial-which-try.html http://opencvpython.blogspot.ca/2012/05/skeletonization-using-opencv-python.html http://opencvpython.blogspot.ca/2012/06/contours-2-brotherhood.html http://docs.opencv.org/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html
1 2 3 4 5 6 7 8 9 10 11 12 13
Oh and one last thing: if you what to search on google for opencv documentation with python, the following keywords are useful:
- opencv python my-searched-function-keywords
And add “documentation” to your keywords to find the official official opencv documentation website, which is very good if you are searching for a method’s arguments.