Thursday, May 8, 2014

How to Calibrate a Camera with OpenCV and Python

Often you a faced with the problem of determining the intrinsic calibration of a camera. For example to straighten wide-angle photos, structure from motion and 3D reconstructions, augmented reality applications etc. With OpenCV the process is fairly simple but I found that a lot of people are unsure how to do it. Here's an example of calibrating a GoPro Hero 3 camera.

1) Install OpenCV

First, you need to have OpenCV installed. Check the OpenCV website or this older blog post if you need help.

2) Print the checkered calibration pattern

You can use any chessboard like pattern but using the default one that comes with OpenCV means you don't have to change any parameters.

3) Attach the pattern to a flat surface

For example a wall, door or floor. Take lots of photos of your pattern from many varying angles and distances. Like this:



4) Run calibration script

Put the photos in a folder on your machine and run the OpenCV python sample file calibrate.py like this:
$  python calibrate.py "calibration_samples/GOPR00*.JPG"
Here you use the path pattern that matches your files as a string (note the quotes!). You should see something like this:
processing calibration_samples/GOPR0045.JPG... ok
processing calibration_samples/GOPR0046.JPG... ok
processing calibration_samples/GOPR0047.JPG... ok
processing calibration_samples/GOPR0048.JPG... ok
processing calibration_samples/GOPR0049.JPG... ok
...
After all the calibration images have been processed you will get the calibration parameters:
RMS: 0.171988082483
camera matrix:
[[ 611.18384754    0.          515.31108992]
 [   0.          611.06728767  402.07541332]
 [   0.            0.            1.        ]]
distortion coefficients:  [-0.36824145  0.2848545   0.00079123  0.00064924 -0.16345661]
Done!

If the calibration pattern was not detected in an image you will get that instead of 'ok'. This happens. As long as you have enough images where the detection works, you should be fine. Take more photos, add to the folder and run again if needed.

Note that the camera matrix is dependent on the resolution of your images. If you use another resolution from the same camera, you need to scale the focal length (the values on the diagonal). The radial distortion parameters are independent of resolution.

5) Unwarp an image just to check

For example like this:
import numpy as np
import cv2

# copy parameters to arrays
K = np.array([[611.18384754, 0, 515.31108992], [0, 611.06728767, 402.07541332], [0, 0, 1]])
d = np.array([-0.36824145, 0.2848545, 0, 0, 0]) # just use first two terms (no translation)

# read one of your images
img = cv2.imread("calibration_samples/GOPR0059.JPG")
h, w = img.shape[:2]

# undistort
newcamera, roi = cv2.getOptimalNewCameraMatrix(K, d, (w,h), 0) 
newimg = cv2.undistort(img, K, d, None, newcamera)

cv2.imwrite("original.jpg", img)
cv2.imwrite("undistorted.jpg", newimg)



Here are the sample files I used if you want to reproduce (calibration_samples.zip, undist.py).

Wednesday, May 7, 2014

Entrepreneurial Computer Vision Challenge


At the upcoming LDV Vision Summit this June 4th, I'll be coordinating an Entrepreneurial Computer Vision Challenge together with Serge Belongie.

Teams can choose from three problem domains involving shoe attributes, egocentric (first person)video and YouTube video text [1].

This is a quick-turnaround event judged by a panel of experts from academia and industry including Gary Bradski, Tamara Berg and Navneet Dalal.

For more information on the Vision Summit, see http://visionsummit.net and for details on the Entrepreneurial Computer Vision Challenge and an application form, see here.

You can register for the event using the discount code "JanErik" to get 40% off (first 20 to use the code get it). Hope to see you in New York!

[1] Shoe attribute and egocentric video datasets kindly provided by Kristen Grauman's lab at UT Austin.

Friday, November 15, 2013

Sonja Kovalevsky Days 2013

I was invited to give a talk at the Sonja Kovalevsky Days to show some bright high school mathematics stars what working with mathematics can be like. The presentation titled "Image recognition and mathematics" is available here.

I covered two main tools, support vector machines and principal component analysis and showed how these can be used in face detection and recognition.

The code for 2D PCA visualization is available here. The Eigenfaces example is from scikit-learn here. The SVM plots are from this, this, and this example.

Monday, August 19, 2013

Anaconda for Scientific Computing and Computer Vision

I recently came across the Anaconda Python distribution. It comes with Python, NumPy, SciPy, Pandas, Matplotlib, Numba and many other packages and you can set up virtual environments for any combination of Python and package versions. Since it uses vistual environments it installs in a folder and doesn't mess with your other Python installations. A list of included packages is here.

Using Anaconda


Here's the quick version of how to get started. Open a terminal and create an environment. For example, to create a new named environment with Python 2.7:
$ conda create -n py27 python=2.7 anaconda
To activate this environment, use:
$ source activate py27
To deactivate this environment, use:
$ source deactivate
Activation and deactivation works the same as Python's virtualenv (as far as I can tell).

Try an example with images, machine learning, and plotting


Download the scikit image quantization example. Activate your environment:
$ source activate py27
Notice that the prompt shows you are in the environment. Start an iPython session:
(py27)$ ipython --pylab
Run the file you downloaded:
In [1]: execfile("plot_color_quantization.py")
You should get three plots like the ones described here. Here's one of them

Monday, April 22, 2013

Watermarking images with OpenCV and NumPy

Here's a simple Python example of adding a watermark to an image using NumPy and OpenCV. I created a black/white version of the OpenCV logo for this example but feel free to use any image you like. In this case I wanted to have a white watermark image, white text, and the rest of the image unchanged. If you have watermark images in color or grayscale the same process should work. Here's the code using the OpenCV license plate sample image:
import numpy as np
import cv2

# read images
original = cv2.imread('data/licenseplate_motion.jpg')
mark = cv2.imread('logo.png') 

m,n = original.shape[:2]

# create overlay image with mark at the upper left corner, use uint16 to hold sum
overlay = np.zeros_like(original, "uint16")
overlay[:mark.shape[0],:mark.shape[1]] = mark

# add the images and clip (to avoid uint8 wrapping)
watermarked = np.array(np.clip(original+overlay, 0, 255), "uint8")

# add some text 5 pixels in from the bottom left
cv2.putText(watermarked, "Watermarking with OpenCV", (5,m-5), cv2.FONT_HERSHEY_PLAIN, fontScale=1.0, color=(255,255,255), thickness=1)

cv2.imshow("original", original)
cv2.imshow("watermarked", watermarked)
cv2.waitKey()
cv2.destroyAllWindows()
Note that I created an array of type "uint16" to hold the sum of the two images before clipping with NumPy's clip function. The text is added with OpenCV's putText function which will need position, font (out of the available OpenCV fonts) and text scale and a color tuple (here white). The result looks like this:

Tuesday, September 18, 2012

Installing PIL on OS X Mountain Lion

Installing PIL, the Python Imaging Library, on OS X sometimes gives problems with missing JPEG support. I have run into this problem before but could not find my solution so I'm adding it here as a note to self with "PIL" and "install" tags so I can locate the trick again in the future.
Here's the procedure.

1. Get libjpeg from http://www.ijg.org/files/jpegsrc.v8c.tar.gz. Then in the unpacked folder:
./configure
make
sudo make install

2. Download PIL from http://effbot.org/downloads/Imaging-1.1.7.tar.gz. (Remove PIL first if installed already.)

3. In the unpacked folder:
python setup.py build --force
sudo python setup.py install
The magic part for me was "--force". Without it it doesn't work. If you still don't get "JPEG support available" in the console when you run the first build command, then you can try adding the path to libjpeg in setup.py (basically just replace the "None" with your path). For example like this:
JPEG_ROOT = "usr/local/lib"

Sunday, August 12, 2012

Reading Gauges - Detecting Lines and Circles

I received a question from a reader on how I would approach reading a simple gauge with one needle on a good frontal image of a circular gauge meter. This makes a good example to introduce Hough transforms. Detecting circles or lines using OpenCV and Python is conceptually simple (each particular use-case requires some parameter tuning though). Below is a simple example using the OpenCV Python interface for detecting lines, line segments and circles. The documentation for the three relevant functions are here. You can also find more on using the Python interface and the plotting commands in Chapter 10 of my book.
import numpy as np
import cv2

"""
Script using OpenCV's Hough transforms for reading images of 
simple dials.
"""

# load grayscale image
im = cv2.imread("gauge1.jpg")
gray_im = cv2.cvtColor(im, cv2.COLOR_RGB2GRAY)

# create version to draw on and blurred version
draw_im = cv2.cvtColor(gray_im, cv2.COLOR_GRAY2BGR)
blur = cv2.GaussianBlur(gray_im, (0,0), 5)

m,n = gray_im.shape

# Hough transform for circles
circles = cv2.HoughCircles(gray_im, cv2.cv.CV_HOUGH_GRADIENT, 2, 10, np.array([]), 20, 60, m/10)[0]

# Hough transform for lines (regular and probabilistic)
edges = cv2.Canny(blur, 20, 60)
lines = cv2.HoughLines(edges, 2, np.pi/90, 40)[0]
plines = cv2.HoughLinesP(edges, 1, np.pi/180, 20, np.array([]), 10)[0]

# draw 
for c in circles[:3]:
 # green for circles (only draw the 3 strongest)
 cv2.circle(draw_im, (c[0],c[1]), c[2], (0,255,0), 2) 

for (rho, theta) in lines[:5]:
 # blue for infinite lines (only draw the 5 strongest)
 x0 = np.cos(theta)*rho 
 y0 = np.sin(theta)*rho
 pt1 = ( int(x0 + (m+n)*(-np.sin(theta))), int(y0 + (m+n)*np.cos(theta)) )
 pt2 = ( int(x0 - (m+n)*(-np.sin(theta))), int(y0 - (m+n)*np.cos(theta)) )
 cv2.line(draw_im, pt1, pt2, (255,0,0), 2) 

for l in plines:
 # red for line segments
 cv2.line(draw_im, (l[0],l[1]), (l[2],l[3]), (0,0,255), 2)
  
cv2.imshow("circles",draw_im)
cv2.waitKey()

# save the resulting image
cv2.imwrite("res.jpg",draw_im)
This will in turn; read an image, create a graylevel version for the detectors, detect circles using HoughCircles(), run edge detection using Canny(), detect lines with HoughLines(), detect line segments with HoughLinesP(), draw the result (green circles, blue lines, red line segments), show the result and save an image. The result can look like this: From these features, you should be able to get an estimate on the gauge reading. If you have large images, you should probably scale them down first. If the images are noisy, you should adjust the blurring for the edge detection. There are also threshold parameters to play with, check the documentation for what they mean. Good luck.