Since getting the tracking engine going, I've turned my attention to tackling the calibration interface. This interface and it's underlying engine are a required bit of functionality for the camera pose estimation system.
The calibration engine determines the lens distortion. Lens distortion is what makes straight lines look curved (think a fish-eye lens bending the sides of a building). Correcting for lens distortion means we provide a much more accurate tracking marker positions, which leads to a better camera pose estimation solve.
It's only early stages, but here's what I can do so far. I can manually add calibration points onto a clip image. Once added, I can then access a right-click context menu that provides processing options, including running the points through a checkered corner match and calculating a grid pattern from the points. Here's the three steps visualised:
The last image shows the checkerboard gird (purple points and lines). From this grid we can pull lines of points. We can use these lines of points to calculate the distortion values. And thus, we can use these values to undistort the image.
And this is the key. We can undistort our tracking marker positions, which in turn allows us to calculate a camera pose as though the tracked positions have come from a perfect pinhole camera. Generally speaking, the more precise your undistorted tracks are, the more precise your camera pose estimation will be.
Above is a side-by-side comparison of the manually placed markers to the calibrated checker corner matches. Moving the slider left to right, you should be able to see a small difference in point positions. Once we've honed our points into the checkered corners, we make a grid with horizontal and vertical lines of points, and use them to calculate the distortion values.
I'm currently not able to show an undistorted image visually, but the numbers are there. The system works.
One issue I want to improve is the time it takes to build the initial grid of points. Currently, points are manually added. There is functionality under-the-hood to search and find corners automatically, but it's slow on large resolution images. So manual placement is faster. But I think I can speed this up even more by adding some functionality that adds rows and columns of points for you.
Another aspect I need to consider is being able to save the lens calibration profile. It will probably be useful to reuse distortion profiles in other shots where the image has been captured with the same camera and lens. My options here will be to save out to a file, or save as a preset inside the program preferences. Both have their merits, and I'll probably implement both ways at some point.