Tantalizingly close

So close and yet so far. That is how it feels at the moment. I found the bug I mentioned in the last post, and had hoped would be the last piece of the puzzle before I had camera ego motion calculations working, but it wasn't to be.

I now have a problem with the build up of error over consecutive frames. My sanity check to see if ego motion calculation is working is to run the system completely stationary ( the only option at the moment anyway ). Therefore the ego motion calculation between frames should render a 0, 0, 0, 0 result. i.e No movement in any of the 3 spatial dimensions and no rotation. Unfortunately I am getting results that look more like this after one iteration:
-----------------------------------------
New position after ego motion estimation:
X = -0.044358336825996
Y = 0.025654321647698
Z = -0.003337081060976
Theta = 0.431281320335258

Obviously subsequent iterations compound this error and it quickly spirals out of control.

Now I realise in the real system there will be the odometery feeding into this and the odometery and ego motion will be combined in a kalman filter to get the actual estimated motion. In this way the accumulative error won't matter so much as it will be dampened by the odometery each iteration.

My concern is that this error is far too big to start off with. I can accept (and was expecting) some error between frames, but in the above example the system believes it has moved 4cm to the left. 4cm is too much. What I have to do now is identify the source of this error. I suspect it may be something to do with excessive accuracy in the SIFT feature 3d position calculations. Any error between calculated feature locations could be being exacerbated by the least squares minimisation. That at least is my uneducated guess at this point. More investigation will hopefully shed some light on this issue.

Nearly there ?

At the risk of jinxing myself completely I think I am nearly at the point where my vision system is correctly calculating camera ego motion! Now if that didn't make you excited it should because it is very very cool.

I managed to spend a bit of time hacking on the weekend and made a huge amount of progress. Now that I have identified what I suspect (but haven't yet confirmed) are libsifts short comings and moved to using Lowe's example binary to extract SIFT features things are starting to work. The matching algorithm I was using to match features between stereo images and between frames was also a bit broken, so I ported Lowe's example matching algorithm from C to C# and that greatly improved matching performance. So now I have one bug to squash and ego motion calculations will be working. I'm pretty sure I know where the problem lies so hopefully I'll get that nailed in the next day or two.

At that point I will have one remaining task before I can declare phase one complete. I still have to figure out how to use the camera extrinsic parameters to give more accuracy to the stereo triangulation calculations. At that point the system will be able to accurately build a map of SIFT features observed in it's environment, and simultaneously localise itself within that environment.

Break through! ...Sort of

Well, about Wednesday night I was looking at the output of one of the sift annotated images from my vision system and I finally snapped. It just looked wrong. It had been bothering me for a while that it didn't seem to be selecting very interesting features and taking an unreasonable amount of interest in the bedroom wall, so I broke out the example binary provided by Professor Lowe himself (the inventor of the SIFT algorithm) for comparison.

As I suspected running the two SIFT implementations against identical images yielded completely different results. Lowe's implementation finding many more and far more interesting features than the version I had been using for months.

This discovery was simultaneously bad and good news. Bad in that it meant I had a bunch of work to do, but good in that I had finally discovered the source of all my frustration.

My new plan of attack has been to write a bunch of C# code to invoke Lowe's binary out of process and parse the resulting key file back into my program. It's all a bit messy really but this whole project is still just a proof of concept so I'm not too stressed about performance at the moment. There is no doubt in my mind at all that C# is magnitudes slower than C for this sort of work, but development time is significantly less and for a prototype that's what matters.

I still have a bunch of work to do before I fully understand stereo vision and the associated camera calibration requirements but I am getting there.

Trying to find the motivation...

It's been ages since I did any work on my vision project and I find myself struggling to find the motivation. I have been stuck on a tricky problem for a while now and I think I need a breakthrough to get some interest back. The problem is that the calculations I am using to calculate the real world 3d coordinates of features are giving completely nonsensical answers. I am having trouble establishing whether the formulas I am using are wrong, or if the feature matching isn't working properly and I am trying to work out pose information for a bad stereo feature match...

If only I could find a site on the web somewhere that had a definitive discussion of calculating real world 3d coordinates from stereo images that would at least help discount the first option.

Back from holiday and Go NASA!

Well, I have just returned from the most fantastic holiday in the Scottish highlands. If anyone is thinking of heading up there and doing a tour I can definitely recommend MacBackpackers and if you can get on a tour lead by Colin even better! I will try an get photos up on
http://www.jamesfitzsimons.com as soon as possible.

I have been following the Discovery return to flight mission with great interest. I have always been a bit of a space nut, but this mission really caught my attention partly because of the test flight nature of it I guess. I think NASA are absolutely fantastic and don't get anywhere near the recognition or funding the deserve in spite of the rubbish the media would have you believe. To prove a point, here are a few interesting figures I found with a quick bit of googling:

2005 NASA budget $16.2 billion
2005 military budget $400.1 billion
Cost (to US) of war in Iraq $314 billion
US GDP 2005 $11,750 billion

So, NASA gets 0.14 % of the US GDP or put another way, 4% of what the US spends on it's military.

Pretty sad.