Blog
Notes from the people making the wand: air-writing, the data behind it, and what we learn on the way.
A motion sensor in a ring reads handwriting in the air
We have been quiet for 3 weeks, making a ring. Its own board is not made until November, so John printed a test ring around an off-the-shelf motion sensor, wired it to the bench and wrote 891 letters on it in 3 days. It read his handwriting after a few minutes, because 450 people on Prolific had already taught The Wand theirs with their phones. How the test ring is made, what the phone study is worth to a ring, and a film traced from John's own recordings.
Inside The Wand, a swinging crystal
Your phone contains an inertial measurement unit — 2 sensors that between them measure 6 degrees of freedom, for about $3. Inside one of them is a piece of silicon machined into a tuning fork, sealed in a vacuum and swinging at 10–30 kHz, and the sideways lean of that swing is measured to 16 femtometres. How a gyroscope actually works, why it needs an accelerometer beside it, what one costs, and how the pair follows a hand writing in mid-air.
Three problems. One thing in the hand.
The elderly are left out of the family chat; phone keyboards are too small for the hands that use them; and there is no good text interface for smart glasses. Three problems that turn out to be the same problem, and what a handheld that types what you draw in the air does about it. Plus nine things to take away, including the one that surprised us — the wand reads the over-sixties better than the under-25s.
450 paid volunteers have written in the air for our AI. Here is how well it reads a stranger — and how to teach it your own hand in five minutes.
Since May, 450 paid volunteers have written in the air for the wand across 24 studies. How well the recogniser reads a hand it has never seen — 85% of words exactly right, cold; by handedness, sex and age; half get 95%, the worst tenth 65% — what five minutes of your own writing changes (28 people, before and after, every number), where the data came from, what more data buys, and the five-minute version you can run on your own phone.
Your bird-song app and the wand use the same trick. Here is what a neural network sees when you write in the air.
Merlin turns bird-song into a picture and reads it with a convolutional network; the wand reads the motion of your hand the same way. Watch real recordings from our study light up a bank of lamps, and see that a network that had never met the writer reads the over-40s better than the under-25s.
Our neural net reads the over-60s better than the under-25s
We paid 349 strangers to write 52,191 letters in the air to teach the wand to read air-writing. The recogniser reads left- and right-handers, women and men, and the over-60s equally — the 18-to-24s were the ones it struggled with. Plus: men and women write the same in the air, and older hands don't shake more, they slow down.
Glasses everywhere, keyboards nowhere
It's been quite a summer for face computers — and none of them has a keyboard. On the Glass years, the backlash that was really about the camera, and why the face computer still needs handwriting.
Why we're building the wand
The idea came in lockdown, 2020: kids growing up staring at screens, elderly parents thumb-typing WhatsApp, and a camera in the living room never an option. What the wand is, why the TV comes first, and where it goes.
Get the next one by email.
What we measure, what breaks, what we ship. Roughly monthly, and once on the day the first run opens.
One email to confirm, then roughly monthly. One click to leave, and we never sell addresses. What we keep