Object pose estimation

The Problem

For the chess robots, it would be nice if it could prepare a game off chess from scattered pieces, to be able to do this the robot must know what pieces are what and their orientation in 3D space to grab em.

Also dubbed object picking in 3D.


The Method

The plan is rather simple, Construct a 3D model consisting of few points for each chess piece, segment the silhouttes from the picking scene, fit the constructed model to the silhouttes. The Best fit should be the chess piece.


I assume that there from one image will be lots of problems obtaining a usefull silhoutte, therefore using multiple images and corelating the probability off the recovered models, could be a method for increasing detection rate.


Construction the model.

First silhoutte images was generated of key orientations. Images from the front and the side was acquired off the chess pieces. The silhoutte was segmented using a threshold and a search for the biggest blob. example of segmented knigh given below.

To mark the points of interes, the defining object points, a program to mark the pieces was/are being developed. The program uses OpenGL for the visuals, the planes can be dragged to the best position. Example image given below, also for the knight.

The above image shows the knight approximately aligned and with example points marked. It is not obvious from this image, but when the are rotated it is visible that the scale of the images is not correct right now. Images are currently always stretced to fit a square. Scaling them individually might be a sollution, the question is if I should use the original image sizes or make it possible to scale the images in the program.. Anyhow, on the next picture you can see the problem that the foot is not the same sice of both silhouttes, like they should be.

Marking program current features.

• Optimized for chess pieces ( variable names in the code )
• Transform the images in their respective planes.
• Scale the images in their respective planes.
  
• can rotate 3D object
• nice vis using OpenGL custom shader
• Can save/load parameters to/from file
• Zooming ability of head and foot of pieces

TODO...

Nice to have features:
make it possible to rotate the planes, right now it looks like it is not required, but for feature completness off the programm it would be nice.

Known Bugs:
When an area is zoomed, the intersection of mouse click rays and the planes are wrong.