Thursday, February 12, 2015

LDV Vision Summit 2015

This year is the second year for the LDV Vision Summit, a great computer vision event with a different twist. The people attending last year were a mix of researchers, entrepreneurs, investors, and people interested in the space of imaging. The result of mixing these people in one single event was amazing and resulted in investments, job offers, and lots of new connections.

This year it will be a two-day event with a full day dedicated to technology. On that day the top finalists of the Entrepreneurial Computer Vision Challenges will present on stage. If you’re a researcher/tinkerer/startup, take a look and submit a solution. You have two months to work on something great!

I’m advising, probably judging for the the competitions, and speaking about something in the intersection of large scale mapping with computer vision. Very excited to be helping Evan, Serge and the other organizers.

If you’re in doubt, read Serge Belongie’s great blog post about his impression from last year and why you should attend this year.

Join us in New York on May 19 & 20 at the SVA Theatre.

Here's a video from my talk last year:

Thursday, May 8, 2014

How to Calibrate a Camera with OpenCV and Python

Often you a faced with the problem of determining the intrinsic calibration of a camera. For example to straighten wide-angle photos, structure from motion and 3D reconstructions, augmented reality applications etc. With OpenCV the process is fairly simple but I found that a lot of people are unsure how to do it. Here's an example of calibrating a GoPro Hero 3 camera.

1) Install OpenCV

First, you need to have OpenCV installed. Check the OpenCV website or this older blog post if you need help.

2) Print the checkered calibration pattern

You can use any chessboard like pattern but using the default one that comes with OpenCV means you don't have to change any parameters.

3) Attach the pattern to a flat surface

For example a wall, door or floor. Take lots of photos of your pattern from many varying angles and distances. Like this:

4) Run calibration script

Put the photos in a folder on your machine and run the OpenCV python sample file like this:
$  python "calibration_samples/GOPR00*.JPG"
Here you use the path pattern that matches your files as a string (note the quotes!). You should see something like this:
processing calibration_samples/GOPR0045.JPG... ok
processing calibration_samples/GOPR0046.JPG... ok
processing calibration_samples/GOPR0047.JPG... ok
processing calibration_samples/GOPR0048.JPG... ok
processing calibration_samples/GOPR0049.JPG... ok
After all the calibration images have been processed you will get the calibration parameters:
RMS: 0.171988082483
camera matrix:
[[ 611.18384754    0.          515.31108992]
 [   0.          611.06728767  402.07541332]
 [   0.            0.            1.        ]]
distortion coefficients:  [-0.36824145  0.2848545   0.00079123  0.00064924 -0.16345661]

If the calibration pattern was not detected in an image you will get that instead of 'ok'. This happens. As long as you have enough images where the detection works, you should be fine. Take more photos, add to the folder and run again if needed.

Note that the camera matrix is dependent on the resolution of your images. If you use another resolution from the same camera, you need to scale the focal length (the values on the diagonal). The radial distortion parameters are independent of resolution.

5) Unwarp an image just to check

For example like this:
import numpy as np
import cv2

# copy parameters to arrays
K = np.array([[611.18384754, 0, 515.31108992], [0, 611.06728767, 402.07541332], [0, 0, 1]])
d = np.array([-0.36824145, 0.2848545, 0, 0, 0]) # just use first two terms (no translation)

# read one of your images
img = cv2.imread("calibration_samples/GOPR0059.JPG")
h, w = img.shape[:2]

# undistort
newcamera, roi = cv2.getOptimalNewCameraMatrix(K, d, (w,h), 0) 
newimg = cv2.undistort(img, K, d, None, newcamera)

cv2.imwrite("original.jpg", img)
cv2.imwrite("undistorted.jpg", newimg)

Here are the sample files I used if you want to reproduce (,

Wednesday, May 7, 2014

Entrepreneurial Computer Vision Challenge

At the upcoming LDV Vision Summit this June 4th, I'll be coordinating an Entrepreneurial Computer Vision Challenge together with Serge Belongie.

Teams can choose from three problem domains involving shoe attributes, egocentric (first person)video and YouTube video text [1].

This is a quick-turnaround event judged by a panel of experts from academia and industry including Gary Bradski, Tamara Berg and Navneet Dalal.

For more information on the Vision Summit, see and for details on the Entrepreneurial Computer Vision Challenge and an application form, see here.

You can register for the event using the discount code "JanErik" to get 40% off (first 20 to use the code get it). Hope to see you in New York!

[1] Shoe attribute and egocentric video datasets kindly provided by Kristen Grauman's lab at UT Austin.

Friday, November 15, 2013

Sonja Kovalevsky Days 2013

I was invited to give a talk at the Sonja Kovalevsky Days to show some bright high school mathematics stars what working with mathematics can be like. The presentation titled "Image recognition and mathematics" is available here.

I covered two main tools, support vector machines and principal component analysis and showed how these can be used in face detection and recognition.

The code for 2D PCA visualization is available here. The Eigenfaces example is from scikit-learn here. The SVM plots are from this, this, and this example.

Monday, August 19, 2013

Anaconda for Scientific Computing and Computer Vision

I recently came across the Anaconda Python distribution. It comes with Python, NumPy, SciPy, Pandas, Matplotlib, Numba and many other packages and you can set up virtual environments for any combination of Python and package versions. Since it uses vistual environments it installs in a folder and doesn't mess with your other Python installations. A list of included packages is here.

Using Anaconda

Here's the quick version of how to get started. Open a terminal and create an environment. For example, to create a new named environment with Python 2.7:
$ conda create -n py27 python=2.7 anaconda
To activate this environment, use:
$ source activate py27
To deactivate this environment, use:
$ source deactivate
Activation and deactivation works the same as Python's virtualenv (as far as I can tell).

Try an example with images, machine learning, and plotting

Download the scikit image quantization example. Activate your environment:
$ source activate py27
Notice that the prompt shows you are in the environment. Start an iPython session:
(py27)$ ipython --pylab
Run the file you downloaded:
In [1]: execfile("")
You should get three plots like the ones described here. Here's one of them

Monday, April 22, 2013

Watermarking images with OpenCV and NumPy

Here's a simple Python example of adding a watermark to an image using NumPy and OpenCV. I created a black/white version of the OpenCV logo for this example but feel free to use any image you like. In this case I wanted to have a white watermark image, white text, and the rest of the image unchanged. If you have watermark images in color or grayscale the same process should work. Here's the code using the OpenCV license plate sample image:
import numpy as np
import cv2

# read images
original = cv2.imread('data/licenseplate_motion.jpg')
mark = cv2.imread('logo.png') 

m,n = original.shape[:2]

# create overlay image with mark at the upper left corner, use uint16 to hold sum
overlay = np.zeros_like(original, "uint16")
overlay[:mark.shape[0],:mark.shape[1]] = mark

# add the images and clip (to avoid uint8 wrapping)
watermarked = np.array(np.clip(original+overlay, 0, 255), "uint8")

# add some text 5 pixels in from the bottom left
cv2.putText(watermarked, "Watermarking with OpenCV", (5,m-5), cv2.FONT_HERSHEY_PLAIN, fontScale=1.0, color=(255,255,255), thickness=1)

cv2.imshow("original", original)
cv2.imshow("watermarked", watermarked)
Note that I created an array of type "uint16" to hold the sum of the two images before clipping with NumPy's clip function. The text is added with OpenCV's putText function which will need position, font (out of the available OpenCV fonts) and text scale and a color tuple (here white). The result looks like this:

Tuesday, September 18, 2012

Installing PIL on OS X Mountain Lion

Installing PIL, the Python Imaging Library, on OS X sometimes gives problems with missing JPEG support. I have run into this problem before but could not find my solution so I'm adding it here as a note to self with "PIL" and "install" tags so I can locate the trick again in the future.
Here's the procedure.

1. Get libjpeg from Then in the unpacked folder:
sudo make install

2. Download PIL from (Remove PIL first if installed already.)

3. In the unpacked folder:
python build --force
sudo python install
The magic part for me was "--force". Without it it doesn't work. If you still don't get "JPEG support available" in the console when you run the first build command, then you can try adding the path to libjpeg in (basically just replace the "None" with your path). For example like this:
JPEG_ROOT = "usr/local/lib"