Your smartwatch might be more capable than you think. Researchers at Cornell University and KAIST in South Korea have developed a system called WatchHand that turns any regular smartwatch into a hand-tracking device.
The best part is that it doesn’t require any extra hardware to do so. According to the published paper, the technology uses AI-powered micro sonar to read your hand movements and finger positions in real time, and it works using hardware your watch already has.
So, how does WatchHand actually work?
Your smartwatch has a built-in speaker and microphone. WatchHand puts both to work in a way you’d never expect. The speaker emits inaudible sound waves that bounce off your hand and back into the microphone, creating a unique echo pattern.
An AI algorithm running directly on the watch reads that pattern and maps your hand position in 3D. Think of it like the echolocation bats use to navigate, except it fits on your wrist. Because everything is processed on the watch itself, none of your movement data gets sent anywhere else.
What can WatchHand do for you?
The practical applications are genuinely exciting. Double-tapping your fingers could skip a track. Subtle hand gestures could move your cursor without touching your mouse or trackpad.
Down the line, WatchHand could potentially track your typing, serve as a controller in augmented or virtual reality, and support people with limited mobility who struggle with traditional input devices.
Researchers also believe a software update alone could bring this capability to millions of existing smartwatches already in people’s hands.
Although there are some limitations worth knowing. WatchHand currently only works on Android, not Apple watches, and it has trouble keeping up when you’re walking. But for a first-of-its-kind system built entirely on hardware you already own, that’s a pretty strong start.
Smartwatches are clearly becoming a lot more ambitious than anyone expected, as researchers are also working on a wrist-worn device that can detect microplastics inside your body.