Over the past couple of years, we have seen numerous attempts at turning the smartwatch on your wrist into more of an input device. One of those approaches was an app called Doublepoint, which lets users control a cursor and click using finger gestures while connected to a Mac. Apple itself has experimented with finger-based gestures such as wrist flick and double tap between your thumb and index finger. But so far, it’s mostly been a fairly rudimentary exercise.
Researchers at North Carolina State University have now developed a radar-based system that essentially turns your forearm into a trackpad. The tech, which the team calls EpiPad, relies on a compact millimeter-wave (mmWave) radar that can read what’s happening on your forearm. For example, it can detect the tiny dents your fingertips make when you press the skin. The system can also detect multi-finger gestures, and even sense how hard you’re pressing, assigning different tasks to each.
According to Yuliang Fu, lead author of the paper, the team essentially wanted to create a system that lets people use their smartwatch the same way they would use a laptop’s trackpad. The EpiPad prototype looks more like a wristband, and comes equipped with a sensor array measuring roughly 5 x 4 millimeters that can track finger movement as well as force touch. To test it, the researchers focused on three core performance areas: gesture recognition, force detection, and finger tracking on the skin.
When it comes to hardware innovations like this, prototypes usually look bulky, but as they move toward integration in mass-market hardware, they tend to shrink pretty quickly. The researchers are hoping that EpiPad will follow a similar path. And here’s the most interesting part. The team behind EpiPad notes that the components powering the radar system are pretty cheap, and that the cost could be less than $10.
Another notable aspect is that clothing doesn’t necessarily break the system, as long as it fits snugly. As per the research paper, when a person wears a skin-tight sleeve, gesture recognition accuracy is 88.4%, while force recognition accuracy is 89.9%. The accuracy drops to 72.3% and 63.2%, respectively, when users wear something loose, such as a jacket. A skin-tight garment fares much better because it follows the natural movement of your skin, so it doesn’t introduce much interference.
As part of the study, the team recruited 20 participants, who rated gentle inputs, with either one or two fingers, as the most intuitive. As far as power consumption goes, the radar module’s draw is well below what a smartwatch screen consumes. In the paper, the team notes that the interaction experience is comparable to that of a traditional touchpad. They also say the compact antenna design could be integrated into a smartwatch’s bezel, which means that if the tech eventually makes its way into a mainstream device like an Apple Watch or a Google Pixel Watch, it could blend into the hardware instead of sticking out.
The biggest hurdle is calibration. Without it, accuracy for new users drops to 56.7%. But after a one-time calibration that takes under four minutes, gesture recognition accuracy climbs to 92.8%, which is not surprising. I recently built an app that uses the motion sensors inside AirPods to control my Mac, and I noticed that once I calibrated the degree of my head movement and recentered my natural resting position, accuracy was much, much higher.
With EpiPad, latency is also not too bad. For gestures, it stands at 90 milliseconds, which is comparable to commercial devices, while for force detection, it is about 470 milliseconds, or roughly half a second. The other big engineering hurdle is the height the sensor needs to do its job accurately. In the current design, the sensor sits 14 millimeters above the skin, modeled on the thickness of the Apple Watch Ultra 2.
That works for chunkier watches like the Apple Watch Ultra, but for slimmer smartwatches, the hardware will need to be reworked, with the researchers pointing to custom designs with multiple transmitters as a possible fix. Either way, it’s a massive breakthrough, and the research (which has been published in the Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies journal) also notes that the tech can be even integrated within smartglasses down the road, solving a huge interaction hurdle for users.