Lloyd Hughes (M.Sc Applied Mathematics).

Scene Aware Augmented Reality: Implementing 3D Reconstruction on a Mobile Device.

Since the advent of the computer age humans have been focused on how we can incorporate these machines into our lives in a manner which supports Ray Kurzweil’s singularity model, where man and machine will essentially become one. Augmented Reality is just another step towards this ideological state, whereby digital information is seamlessly combined into our view of the world in order to allow the user to access and utilize information in a natural manner. The problem with AR in the past has been that many of these systems are power hungry and have required fairly substancial computers in order to operate, and thus have not been suitable for everyday use. However, mobile devices, such as smart phones and tablets, now incorporate many high density computational elements such as multi-core CPUs and mid-range GPUs thus effectively allowing the previously computationally expensive computer vision techniques to be run on these platforms in order to effectively augment our world. Mobiles devices are also relatively cheap and thus allow for AR systems to be used by many more people than in the past.

My research will focus on the design and implementation of a software system which will allow for dense 3D models of a rigid scene to be generated from a live video stream on a mobile device. This system will allow accurate 3D models to be created which can then be used for accurately augmenting digital information into the scene. This research will provide insights into monocular computer vision techniques for 3D reconstruction as well as into efficient computational structures and strategies for use on a mobile device.

Having a system which allows for the creation of 3D models from the video stream on a mobile device will open the doors to new AR applications in many different fields. An application of this could be scene aware gaming, whereby a game can be played in real-time in a real environment and the characters will understand and react to the environment in a realistic manner. Other examples can be seen in 3D modeling for use in animation, visual effects, mechanical simulations, medical applications, interior design and many more.

My contact details: lhhughes@ml.sun.ac.za