Of all the constructions in nature that are hard to replicate artificially, the human hand is the most complex. It is nature’s most efficient appendage, blending maximum mobility, strength, and function together with a sense of touch that can detect a wide range of sensory aspects. Skilled performers, athletes, and craftsmen will frequently opine that their hands even have a “mind of their own,” able to respond to stimulus and perform actions before the conscious thought is registered. There is even a name for this phenomenon, called “muscle memory.”
Replicating anything close to the human hand experience in virtual reality is one of the greatest challenges in all of AI. The goal is to be able to accompany the canonical VR headset with a set of haptic gloves which will allow the user to interact with objects in the virtual environment. The user should be able to hold objects, flip switches, open doors, pet a cat, and any number of other common actions naturally and with as little intrusion from the technology as possible.
If we could perfectly replicate all the sensations the human hand can feel, the list would extend to:
- Motion – Vibration is so far the easiest to peg.
- Texture – Distinguishing between soft, hard, rough, or smooth.
- Pressure – Being able to feel an object gripped in the hand.
- Temperature – Hot or cold.
- Wind – Being able to feel air blowing on the skin.
- Moisture – Wet or dry.
- Pain – Pricks, cuts, and other hazards.
That’s just the most basic list. As it is, haptic technology is making progress in replicating a surprising chunk of this list.
The Current State of Haptic AI
Haptic technology isn’t just limited to what the user feels. It’s a two-way link between the user’s hands and their virtual reality equivalents, using motion capture methods to track the gross and fine motor skills expressed in the human hand. When the user moves their hands, the simulated hands in the virtual environment also have to mirror the motion.
So far the company farthest ahead in the haptics game is HaptX, which develops a set of haptic gloves and also software and accessories for haptic technology. Their gloves boast the following features:
- 130 feedback points embedded within the gloves.
- Can simulate the sensations of shape, texture, and motion of grasped objects.
- Lightweight exoskeleton applying light force for resistance sensations when grasping and squeezing objects.
- Magnetic motion tracking.
- Software development kit based on open source and proprietary technologies.
Though haptic technology at this level is impressive, there’s still much more progress left open to the future. For one thing, determining all of what we feel and how we process it is in the domain of neuroscience, which still makes new discoveries and encounters new questions every year.
Haptic Glove Applications
The most obvious application of haptics is in virtual and augmented reality gaming. Gaming is the domain where haptics first showed up, after all, in the form of vibrating feedback in game controllers. But haptic gloves have come a long way since then. Haptic gloves in games will allow players to feel the recoil when firing a gun, the resistance when firing a bow, and other touches of reality.
But well beyond the domain of gaming, haptic systems are very useful in practical industrial applications as well. They are valuable for training simulation scenarios, allowing surgeons and dentists to practice on virtual patients, emergency workers to train in scenarios where tactile feedback counts, or gardeners and veterinarians to feel responses in the simulation of their tasks. Drivers in training simulations use haptic gloves to feel the pull on the steering wheel as they round the corner of the racetrack.
The sense of touch is also beneficial to simulate in industrial scenarios. For example, a robot controlled with motion capture can give feedback for delicate tasks where pressure on the hands would indicate information about the object the user is manipulating.