Blog
Notes from the people making the wand: air-writing, the data behind it, and what we learn on the way.
Three problems. One thing in the hand.
The elderly are left out of the family chat; phone keyboards are too small for the hands that use them; and there is no good text interface for smart glasses. Three problems that turn out to be the same problem, and what a handheld that types what you draw in the air does about it. Plus nine things to take away, including the one that surprised us — the wand reads the over-sixties better than the under-25s.
450 paid volunteers have written in the air for our AI. Here is how well it reads a stranger — and how to teach it your own hand in five minutes.
Since May, 450 paid volunteers have written in the air for the wand across 24 studies. How well the recogniser reads a hand it has never seen — 85% of words exactly right, cold; by handedness, sex and age; half get 95%, the worst tenth 65% — what five minutes of your own writing changes (28 people, before and after, every number), where the data came from, what more data buys, and the five-minute version you can run on your own phone.
Your bird-song app and the wand use the same trick. Here is what a neural network sees when you write in the air.
Merlin turns bird-song into a picture and reads it with a convolutional network; the wand reads the motion of your hand the same way. Watch real recordings from our study light up a bank of lamps, and see that a network that had never met the writer reads the over-40s better than the under-25s.
Our neural net reads the over-60s better than the under-25s
We paid 349 strangers to write 52,191 letters in the air to teach the wand to read air-writing. The recogniser reads left- and right-handers, women and men, and the over-60s equally — the 18-to-24s were the ones it struggled with. Plus: men and women write the same in the air, and older hands don't shake more, they slow down.
Glasses everywhere, keyboards nowhere
It's been quite a summer for face computers — and none of them has a keyboard. On the Glass years, the backlash that was really about the camera, and why the face computer still needs handwriting.
Why we're building the wand
The idea came in lockdown, 2020: kids growing up staring at screens, elderly parents thumb-typing WhatsApp, and a camera in the living room never an option. What the wand is, why the TV comes first, and where it goes.