With the recent growth of virtual reality (VR) applications, there is a demand to create highly immersive environments in which the avatar that the user embodies reflects any kind of actions in the virtual world as precise as possible. The major action humans use for interacting with the world is grasping of objects with their hands. In real world, human hand and fingers are constrained by the object’s shape and the intended use of the object. However in a virtual environment where the realistic physical contact with the object can not be sensed by the human user, the visual representation of the virtual hand grasping various objects requires tedious manual animation.
Gleechi provides a software solution called VirtualGrasp which makes it possible to animate natural looking grasping interactions in real-time based on the constraints of the virtual world (such as shape of objects, kinematics of the hand, etc). This solution is not a hand tracking algorithm, but a tool that animates a given hand model.
To best automate the animation of hand and fingers when interacting with virtual 3D objects, a good understanding and representation of the 3D geometric properties of the object becomes very important pre step. Most objects in the real world are composed with different geometrical and also functional parts, and grasping action are often focused on one of these identified parts of the objects. Therefore to be able to segment objects into optimalbest sets of parts and then apply grasp on the identified parts has been researched in the robotic grasping community .
Recently machine learning techniques that exploit the deep structure of neural networks has achieved significant progress towards many practical industrial problems. In the context of 3D geometric data, deep neural network (DNN) is an active research area with a lot of potential applications ranging from 3D shape reconstruction, recognition, retrieval, segmentation, etc. The goal of this thesis is to exploit DNN for object shape representation for the purpose of human hand grasp animation. The scope of the project include apply DNN for part-based object representation, which will involve segmentation and shape representation of the segmented parts.
Summarize state-of-the-art of deep learning study aimed at modeling and representing 3D object shape and segmentation of the shape.
Collect training database for object shape representations.
Implement modeling and training of DNNs, using Caffe deep learning framework , in C++.
Test, optimize and evaluate the implemented process using the database.
Summarize and discuss the findings in a report / thesis.
 Selection of robot pre-grasps using box-based shape approximation. By Huebner, Kai, and Danica Kragic. 2008.
 Multi-view convolutional neural networks for 3d shape recognition. By SU, Hang, MAJI, Subhransu, KALOGERAKIS, Evangelos, et al. 2015
About the company
Gleechi is a Stockholm-based startup that has developed the first software to enable hand interaction for animated hands and robot hands. The technology is based on 8 years of robotics research, and is used to enable free and natural hand interaction in VR. In addition, the company has a collaboration with a world-leading automation company to use the same technology in robotic applications. The Gleechi team combines expertise in various areas such as robotics, computer vision and machine learning, and has received several awards, including Super Startup of 2015 by Veckans Affärer and ALMI Invest and Winner of the european competition EIT Digital Idea Challenge.