The researchers at Facebook Reality Labs have set themselves the goal of developing technologies for natural and effective interaction in augmented reality (AR) environments. One of the projects Facebook Reality Labs is working on consists of a wrist strap. The integrated AI should dynamically adapt to the user and his or her environment.
According to the researchers, a bracelet was the ideal solution. Contrasted with a voice command system that is “not private enough for the public domain or unreliable due to background noise”. A device that is in a pocket adds an additional layer of friction between the user and the environment. A wearable worn on the wrist also has the advantage that it combines computing power, battery and antennas in one housing and at the same time supports a wide range of sensors, as the researchers explain.
Recognize the smallest movements
In addition, the bracelet must be able to convey a sufficient amount of information. Facebook Reality Labs uses EMG (electromyography) technology, which uses sensors to translate electrical signals from nerves and muscles. The technology should be able to detect a finger movement of just under a millimeter.
The prototype of the Facebook bracelet can initially detect one or two control movements, such as a finger pinch or the letting go of the thumb and forefinger. But the researchers want to go much further in order to capture complex movements that would allow objects to be moved remotely in augmented reality or to type anywhere on an imaginary keyboard. Ultimately, the goal is to recognize even the simple intention of moving a finger. It’s not about reading minds, emphasize the researchers, but rather capturing the nerve signal of an intention on the wrist.
AI knows what you want to do
Another goal of the project is to supplement the nerve signal recognition system with predictive AI functions. “The underlying AI has an idea of what you want to do in the future,” said Tanya Jonker, chief scientist at Facebook Reality Labs. She adds that the system selects the most relevant interaction options based on the previous behavior of the user, which can be confirmed or changed with a micro-gesture.
Looking even further into the future, the researchers plan to supplement this futuristic interface with a haptic feedback system. The latter would also be transmitted via a vibro-tactile device and a pressure mechanism on the wrist. These haptic sensory impressions would be used, for example, to guide the wearer when performing an action with virtual objects. Or to signal the receipt of e-mails differently depending on the importance of the sender.
The development of the bracelet is still in its infancy. But researchers at Facebook Reality Labs say they are also making their progress public to spark a debate about the ethical, security, and privacy issues that projects like this one raise.
Facebook should also work on other wrist devices. Rumor has it that the social media giant is developing its own smartwatch. It should be able to independently connect to mobile networks, measure health data and send messages.