Remember J.A.R.V.I.S. from Iron Man? In the movie franchise, J.A.R.V.I.S. is an artificially intelligent computer system with a holographic user interface, allowing Tony Stark to control its graphical elements with his hands easily. As it turns out, we could be closer to experiencing this digitally advanced reality than we thought.
Currently being studied and developed, Facebook Reality Labs’ (FRL) new endeavor aims to give people a new way to interact with the digital world. We don’t have to move our attention away from our surroundings to our devices anymore.
Through augmented reality (AR) glasses and an AI-powered wristband, Facebook is reportedly paving the way for a future where the digital world is deeply embedded in our day-to-day life. With the AI-powered wristband, users can give their input — or commands — to the device, which will be reflected in the AR glasses.
According to their report on Thursday last week, FRL says this is how it will look like when both devices work together:
Imagine you’re browsing through the web. But instead of viewing the digital world through a handheld device, you’re looking at your immediate surroundings overlaid with the graphical elements you often see on your gadget’s screen. And by strapping your wrist with the AI-powered wristband, you get to interact with these elements using your fingers. It’s almost like being in an Avengers movie — but with less glitz and more utility.
The Goals of Facebook in Developing AR Glasses with an AI-Powered Wristband
Tech companies tend to take leaps toward creating trailblazing technology, and Facebook is no exception. After Google’s and Snapchat’s previous attempts to break into this new approach to human-computer interaction (HCI), Facebook is now also setting its sights on AR-based wearables as an innovative means to engage with the digital space.
According to Facebook’s post, they offered two reasons for why they want to see the cooperative technologies of AR glasses and AI-powered wristbands materialize in the future. The first reason is innovation and the second efficiency.
FLR’s director of neuromotor interfaces Thomas Reardon said in FRL’s post, “We want computing experiences where the human is the absolute center of the entire experience.” And to FRL, this means integrating technology into people’s daily lives in a way that’s more intimate and intuitive.
What if, rather than clicking through menus to do the thing you’d like to do, the system offered that thing to you and you could confirm it with just a simple ‘click’ gesture?
Facebook Reality Labs
In conjunction with this goal, the Facebook group also stresses how the AR glasses and AI wristband can be an attention-preserving and time-saving tool. The group suggests with the enhanced HCI pairing, users can better engage with the present moment, allowing them the space to think more clearly and, paradoxically, not be distracted by technology.
FLR notes the tools they’re presently researching should be able to adapt to our needs, taking our attention only when necessary. “This may only save you a few seconds per interaction, but all those seconds add up,” Facebook says. “And perhaps more importantly, these subtle gestures won’t derail you from your train of thought or flow of movement.”
The Reason Behind the Wrist-Worn Input Device & How It Works with the AR Glasses, According to Facebook
In last year’s Facebook Connect, the tech giant shared its research goals, which included its vision for developing the AR glasses. According to their post published in September 2020, the group wanted to introduce a proactive approach to the way we communicate with digital devices. Or rather, to the way our digital devices communicate with us.
In partnership with Ctrl-Labs, Facebook’s recently acquired tech startup, Facebook has now made strides toward their AR glasses. The glasses should be able to fuse the real and virtual worlds, according to FRL’s VR/AR head Andrew Bosworth.
Conceptually, the AR glasses make for a good tool with which you can view the system’s GUI. But the group lacked another important tool by which users can interact with the digital world. After some consideration, they finally came up with the idea of having the input device strapped around a person’s wrist.
A wristband nearly satisfies all of FRL’s requirements for the ideal input device to pair with their AR glasses. To FRL, what constitutes the ideal input device would be something “intuitive, always available, unobtrusive, and easy to use.” Moreover, it needs to be able to perform a wide range of controls, their post says.
How the Wrist-Worn Input Device Works
With the AI-powered wristband idea, Facebook has found a technique to convert nerve signals from the wrist into digital commands. It’s the same concept as pressing an icon on our screen, which tells our device we want to interact with it.
Through electromyography (EMG), which “measures muscle response or electrical activity in response to a nerve’s stimulation of the muscle” according to hopkinsmedicine.org, the device can understand a user’s movement and decode it into the proper command.
In their post, FRL says this approach is not the same as Facebook being able to read a user’s mind and immediately reflecting their commands in the AR glasses’ screen. It’s about designating movement to a certain type of action carried out in the interface.
For example, the input device can translate a simple clicking gesture to mean you want to open an application or exit from it. Hrvoje Benko, FRL’s director of research science, likens the AI wrist-worn input device to a mouse. It’s a piece in the broader system of the interface, a tool for instructing through gestures.
Such gestures are similar to mouse-clicking, too. Except in the context of the AR glasses and AI wristband, Facebook intends to transform mouse-clicking into what it calls the “intelligent click,” their post says.
The intelligent click…surfaces something that’s relevant based on your personal history and choices, and it allows you to do that with minimal input gestures.
Facebook Reality Labs
Are You Willing to Share More Than Just Your Online Data with Facebook?
What Facebook offers is an exciting piece of technology. If their research pans out as intended, the digital sphere will be in closer proximity to our daily life. This will require a new level of trust we are willing to give Facebook’s ever-growing database. Instead of only our online data, we are going to surrender a part of our neural activity to Facebook’s system for their technology to work.
One safeguard Facebook has claimed so far is the devices will store our information locally instead of the cloud, The Washington Post points out. Only the AR glasses and AI wristband will keep our neural data. But as it’s still under development, it’s too early to say whether this is enough to ensure strict data protection.
The Verge says it best, “Like most wearable technology, EMG bands offer an intimate look at how our bodies are moving — and while it’s not quite as creepy-sounding as a band that reads your thoughts, it still requires a lot of trust.”
Aside from the ethical implications of this new technology, which Facebook is reportedly rigorously reviewing, it will also take some time for the AR glasses and AI wristband pairing to be accepted as the norm. Beyond the learning curve users need to surmount, the question of how accessible it will be comes into play as well.
Nevertheless, Facebook appears to be working to make sure all these potential holes are addressed. And we can only look forward to their updates to see what happens.
In Summary
Facebook Reality Labs (FRL) is researching and developing technology that will integrate a device’s user interface into the real world. Through AR glasses and an AI-powered wristband, Facebook intends to provide an advanced way of interacting with the digital space.
The AI-powered wristband will serve as the input device or a user’s way to communicate with the system. A user’s commands will then be reflected in the AR glasses in real-time. FRL announced that later this year, they will be giving updates on the project and other similar research.
Looking for augmented reality artists who can create 3D graphics for your AR application? NarraSoft offers augmented reality art development and 3D digital art creation services.
Integrate our team of experienced artists into yours. Email us at sales@narrasoft.com or message our contact form or
Sources: Facebook Reality Labs | Bloomberg | hopkinsmedicine.org | The Washington Post | The Verge