Internship Report 2018 (3/3)

3D Reconstruction from Stereo Images

The last internship project I’ll tell you about didn’t result in a satisfactory result in the limited time available, but at the same time it was the one that I definitely found most interesting for reasons I will explain below. While 3D reconstruction is a really interesting topic in general, it is essential for companies like Microtec because in order to cut logs into the required planks in the most efficient way possible. This is usually done via laser-scanning or with stereoscopic cameras, or in the case of Microtec it even can be done with CT-scanners. Since I already had experience with neural networks for 3D reconstruction we decided to test a few approaches to see whether they were feasible. Since at that moment we had no better ground truth images available I had to resort to using cropped and very sparse feature maps created with OpenCV, and also decided to predict depth instead of disparity since the images were not yet undistorted. All considered the data was less than optimal.

What you can see here are the left and right images of the logs as well as the location and depth of the features detected by OpenCV(bottom left) and the mask of valid points and bounding box (bottom right) that are automatically created to reduce outliers and features detected on the background. This had to be done to give the network a reason to ignore and flatten the background. The images were fed into a siamese network similar to the one used in my thesis project. While the network architecture, masking strategy and training process for this model wasn’t trivial, the main task here was bringing the available data into a form that made it usable at all which I found really exciting. In the end the results turned out to be quite fuzzy, since in this kind of data outliers are very overrepresented while smooth surfaces contain almost no features and also the model had never learned to create sharp edges and structures. However, in my opinion even getting to the point I did was quite a feat. The model succeeded to correctly represent the size and shape of the logs as well as giving them some fuzzy but noticeable structure, even showing some interesting details and deformations. I would love to show you GIFs of the three-dimensional reconstructions but unfortunately I don’t have that data available so all I can show you are a few depth-maps. I hope you like them!

This concludes the posts on my internship experience at Microtec. I hope you enjoyed them as much as I enjoyed my time working there and maybe I could inspire you with some of the stuff I learned and did there. Of course I would also like to thank Microtec for helping me make this amazing experience and for allowing me to publish these posts. If you ever have the opportunity to work with them or for them you should strongly consider it.