Project H: How two hackers are using Virtual Reality to transform healthcare

At  HackerEarth, we regularly host hackathons and frequently discover awesome hacks that can be game-changers across industries.

Presenting Project H—one of the best hacks from the Digital India Hackathon conducted by ACM, India in association with HackerEarth.

Read on to learn more about this real-life VR application.

What is Project H?

Aimed at solving problems related to healthcare, Project H provides a virtual-reality experience of holding real products in a virtual environment. It is designed to provide medical students with realistic surgery simulation.The application, when built with the right set of hardware tools, gives the user an immersive user experience.

Medical students will greatly benefit from the simulated experience of handling organs during surgery. It will also make teaching medicine to large groups of students cost-effective.

Using the concept of experiencing the real-world in cyberspace, this hack provides a new dimension to finding solutions for different problems through virtual reality.

Technologies/platforms/hardware/languages

• Haptic technology
• Unity
• Arduino Pro Micro
• Blender
• Vibrators
• Servo motor
• Dual-axis joystick
• C#
• C

Components and functionalities

The two main components of Project H are the haptic glove and simulation software.

Haptic glove

The haptic glove is the most innovative part of this project. Its features are as follows:

• Built with actuators and vibrators (controlled by a small-sized Arduino Pro Micro) that give users kinesthetic and tactile feedback, respectively
• Has a 180-unit, finger-positioning resolution that takes over the finger positioning and gives kinesthetic feedback, when the glove is worn; also has a designed five-motor actuator mechanism that controls the position of each finger
• Equipped with a dual-axis joystick controller that allows users to explore the various menu options in the application

Simulation software

The virtual reality-based simulation software gives the user a 360-degree view of 3D organs. Along with the use of a few open source 3D models of human organs, many 3D models were built using Blender.

This software simulates the shape and responses from each organ. For example, if users are holding a heart in a virtual environment, the haptic glove positions all the fingers as though they are actually holding a beating human heart, giving a realistic look and feel. The beating sensation of the heart is felt by users through the tactile feedback mechanism.

Application

The GUI of Project H has been built using Unity, C# as the programming language, and an Arduino Pro Micro (programmed in C). Serial communication has been used for communication between the hardware and software.

The application allows users to perform virtual surgeries like cutting through layers of skin using scissors and scalpels, injecting fluids, etc. Since the haptic glove is integrated, when users hold a syringe or a pair of scissors, their fingers automatically move into position by using the actuator mechanism. Therefore, users feel that they are actually holding an instrument.

Challenges

Here are some of the challenges that the team faced while building this application:

• Setting up a communication protocol to integrate the haptic glove with the simulation software, which was fast with low latency
• Designing accurate 3D models of human organs
• Bulky and uncomfortable (initial) prototypes, which caused concerns about the structural aspects of the haptic glove

What’s next?

Project creators Gagan G. and Pratik R. have great plans in store for their project. Here are some of their ideas:

• Piezoelectric actuators will be used to improve the tactile feedback.
• Organ systems and tools for surgeries will be increased to improve the content on virtual surgery simulation.
• Actuator mechanism will be improved by using poly magnets and programmable magnets to control the movement of the gloves and increase the precision, look, and feel of holding tools/organs

The team plans on extending the usage of the application to new and exciting fields, such as e-commerce and gaming.

Excited about VR??

Register at the UnitedByHCl hackathon

Happy Mixing Reality!!